LeaderRebalancerTest.RestartTserver: /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:467: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260924 14:31:48.704905 32453 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.791735 32368 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:48.796073 32464 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:48.847641 32466 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:48.889286 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:48.869127 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 145296us
I20260924 14:31:48.722960 32457 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.845722 32462 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:48.874845 32390 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.309s	user 0.005s	sys 0.016s
I20260924 14:31:48.902240 32473 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.903054 32486 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260924 14:31:48.891703 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 86621.3us
I20260924 14:31:48.932811 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:48.845347 31924 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:48.897073 32475 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:48.803336 31677 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:48.908181 32470 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:48.907779 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1633) took 1863 ms. Trace:
I20260924 14:31:48.843678 32192 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:48.906365 32317 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:48.808610 32382 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.873520 32182 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:48.941421 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 79706.1us
I20260924 14:31:48.942932 27185 rpcz_store.cc:276] 0924 14:31:47.043958 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.046731 (+  2773us) service_pool.cc:227] Handling call
0924 14:31:47.047028 (+   297us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.445107 (+398079us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.907280 (+1462173us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400260a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:48.907693 (+   413us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":895078272}
I20260924 14:31:48.924510 32075 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.943909 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:48.968109 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65394.7us
I20260924 14:31:48.973192 26399 raft_consensus.cc:2396] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260924 14:31:48.892710 32259 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.907734 32368 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.968302 23135 tablet_server.cc:178] TabletServer@127.22.151.193:38943 shutting down...
I20260924 14:31:48.940464 32466 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:48.980036 27186 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:31:48.968918 32236 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.259s	user 0.012s	sys 0.004s
I20260924 14:31:48.976086 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:48.976439 32455 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
W20260924 14:31:48.982707 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 91318.3us
I20260924 14:31:48.941797 32461 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:48.979772 32395 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.239s	user 0.000s	sys 0.015s
I20260924 14:31:48.934880 32473 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.868891 32469 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:48.995551 32236 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:48.996007 32311 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.974059 32317 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.997334 26403 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260924 14:31:49.003750 26403 pending_rounds.cc:76] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 9
I20260924 14:31:49.004109 26403 pending_rounds.cc:97] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260924 14:31:49.004420 26403 raft_consensus.cc:2866] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:49.010828 32472 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:49.011720 32472 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:48.999897 32456 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260924 14:31:48.990489 32326 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:48.985565 32454 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.992066 31980 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.005928 32277 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.034677 32192 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.036108 32259 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.036787 32479 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260924 14:31:49.038231 32236 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:48.996908 32461 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:49.009733 32182 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.988034 32484 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260924 14:31:49.044904 32453 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.045131 32473 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:49.045832 32446 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.047760 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1300) took 1514 ms. Trace:
I20260924 14:31:49.050874 32469 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:48.962409 32470 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.008291 32190 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:49.120211 32401 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260924 14:31:49.047801 32380 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.161288 32447 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:49.173708 32334 consensus_meta.cc:287] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.993s	user 0.008s	sys 0.005s
I20260924 14:31:49.122555 32277 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.175194 32472 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.176451 32382 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:49.177631 32467 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260924 14:31:49.223176 32473 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.184866 31688 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:49.222779 32446 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.168717 26324 rpcz_store.cc:276] 0924 14:31:47.533173 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.819023 (+285850us) service_pool.cc:227] Handling call
0924 14:31:47.819431 (+   408us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.859844 (+ 40413us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:47.859894 (+    50us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:47.898601 (+ 38707us) log.cc:842] Serialized 26 byte log entry
0924 14:31:48.546869 (+648268us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:48.606882 (+ 60013us) log.cc:842] Serialized 14 byte log entry
0924 14:31:48.611644 (+  4762us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.611720 (+    76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:49.047521 (+435801us) raft_consensus.cc:1671] finished
0924 14:31:49.047545 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:49.047690 (+   145us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44307,"prepare.queue_time_us":59310,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":4500,"raft.queue_time_us":283690,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":18677,"spinlock_wait_cycles":1075456,"thread_start_us":489245,"threads_started":1,"wal-append.queue_time_us":654030}
I20260924 14:31:49.214460 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1670) took 1478 ms. Trace:
I20260924 14:31:49.253927 26403 rpcz_store.cc:276] 0924 14:31:47.735981 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.886015 (+150034us) service_pool.cc:227] Handling call
0924 14:31:47.917623 (+ 31608us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.005004 (+1087381us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.214137 (+209133us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001c1e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:49.214378 (+   241us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2481948032}
I20260924 14:31:49.130510 32238 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.235819 26332 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260924 14:31:49.180593 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1769) took 2107 ms. Trace:
I20260924 14:31:49.262115 26330 rpcz_store.cc:276] 0924 14:31:47.073450 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.165447 (+ 91997us) service_pool.cc:227] Handling call
0924 14:31:47.165778 (+   331us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:47.259940 (+ 94162us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:47.591628 (+331688us) log.cc:842] Serialized 14 byte log entry
0924 14:31:48.106157 (+514529us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:48.147757 (+ 41600us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:48.148031 (+   274us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.180160 (+1032129us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b5400558cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:49.180234 (+    74us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:49.180501 (+   267us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":47780,"prepare.queue_time_us":269748,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":371684,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":8326,"spinlock_wait_cycles":215935360,"thread_start_us":728668,"threads_started":2,"wal-append.queue_time_us":789325}
I20260924 14:31:49.247392 32457 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:49.251056 31688 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:49.276191 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 302990us
I20260924 14:31:49.250469 31924 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.270107 31677 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.268107 32166 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.350392 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:49.250099 32473 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.274035 32368 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.363022 32289 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260924 14:31:49.364279 32289 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:49.364691 32289 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Advancing to term 24
I20260924 14:31:49.364751 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.365957 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.366173 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.366926 26403 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:49.367854 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.369609 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.372249 32461 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.313097 32259 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.373483 32259 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.373495 32289 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.376327 32470 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:49.349651 32317 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.376363 32289 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 24 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.378589 32473 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.380268 32446 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.267529 32238 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.381174 26404 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:49.382397 26400 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
W20260924 14:31:49.362118 32458 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.626s	user 0.004s	sys 0.008s
I20260924 14:31:49.362551 31688 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:49.363024 32488 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260924 14:31:49.382025 32326 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Stepping down as leader of term 20
I20260924 14:31:49.406190 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:49.383023 32368 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:49.357529 26251 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.000s	sys 0.014s
W20260924 14:31:49.354836 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 331573us
W20260924 14:31:49.410153 26251 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.000s	sys 0.015s
W20260924 14:31:49.384615 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 449549us
W20260924 14:31:49.356724 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 446830us
I20260924 14:31:49.414621 27126 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:49.405656 32470 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:49.406843 32271 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.011s	sys 0.004s
I20260924 14:31:49.417698 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:49.420549 26397 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 23 FOLLOWER]: Advancing to term 24
I20260924 14:31:49.417361 32470 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.423858 32470 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.406570 32326 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:49.413358 32424 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.414852 31688 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:49.412760 32259 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.416719 32391 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:49.428325 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:49.390784 32458 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.438001 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 2232) took 2184 ms. Trace:
I20260924 14:31:49.443336 32322 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.443534 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1977) took 2332 ms. Trace:
I20260924 14:31:49.444082 32192 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:49.445727 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1292) took 2090 ms. Trace:
I20260924 14:31:49.457703 26332 rpcz_store.cc:276] 0924 14:31:47.355708 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.761576 (+405868us) service_pool.cc:227] Handling call
0924 14:31:47.761841 (+   265us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.261997 (+1500156us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.445502 (+183505us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54005bb8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:49.445667 (+   165us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3388535168}
I20260924 14:31:49.444772 31979 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:49.445185 26326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1765) took 2390 ms (client timeout 2157 ms). Trace:
W20260924 14:31:49.462234 26326 rpcz_store.cc:269] 0924 14:31:47.054571 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.054684 (+   113us) service_pool.cc:227] Handling call
0924 14:31:49.444978 (+2390294us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b5400505854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:49.445092 (+   114us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":700235648}
I20260924 14:31:49.445727 26331 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:49.454094 26401 rpcz_store.cc:276] 0924 14:31:47.111004 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.111149 (+   145us) service_pool.cc:227] Handling call
0924 14:31:47.324639 (+213490us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:47.356904 (+ 32265us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.443328 (+2086424us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400290750. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:49.443478 (+   150us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":333028608}
I20260924 14:31:49.453868 26254 rpcz_store.cc:276] 0924 14:31:47.253904 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.254123 (+   219us) service_pool.cc:227] Handling call
0924 14:31:47.289476 (+ 35353us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:47.327814 (+ 38338us) raft_consensus.cc:1520] Early marking committed up to index 12
0924 14:31:47.578699 (+250885us) log.cc:842] Serialized 14 byte log entry
0924 14:31:47.585516 (+  6817us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:47.585571 (+    55us) raft_consensus.cc:1631] Marking committed up to 12
0924 14:31:47.585614 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.558415 (+972801us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b54006c53a8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bcc95361 00007f09bcd2cebb 00007f09bcd2ce09 00007f09bcd2cd99 00007f09bcd2cd61 00007f09bcd2c05d 00007f09bf68a204 00007f09bf68a039 00007f09b64266b6 00007f09b6429b11 00007f09b6429ac9
0924 14:31:49.437721 (+879306us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b54004c9d54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:49.437775 (+    54us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:49.437915 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":199912,"prepare.run_cpu_time_us":23695,"prepare.run_wall_time_us":1030604,"raft.queue_time_us":4123,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":84304,"spinlock_wait_cycles":101265024,"thread_start_us":250591,"threads_started":1}
I20260924 14:31:49.425629 32481 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.460340 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:49.426829 32306 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260924 14:31:49.426468 32271 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.011s	sys 0.004s
I20260924 14:31:49.416882 32368 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.451346 32277 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.452863 32391 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:49.469695 31979 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:49.452059 32290 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:49.468997 26397 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 24.
W20260924 14:31:49.480728 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 466471us
I20260924 14:31:49.481894 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:49.485713 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:49.430326 32332 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260924 14:31:49.470718 32306 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:49.470434 32494 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260924 14:31:49.483157 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.011s	sys 0.007s
I20260924 14:31:49.484387 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:49.490279 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:49.510571 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.495008 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.507822 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:49.480347 32496 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:49.511184 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.011s	sys 0.007s
I20260924 14:31:49.511462 26257 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:49.511693 26332 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:49.512135 26251 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:49.514479 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:49.456907 32424 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.484259 32290 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:49.461932 32322 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.484009 32391 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:49.515027 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 539569us
I20260924 14:31:49.518388 26330 raft_consensus.cc:2396] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260924 14:31:49.520804 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:49.557132 26397 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:49.481948 32238 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.508929 32306 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:49.514588 32486 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:49.541252 32186 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260924 14:31:49.522900 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.006s	sys 0.013s
W20260924 14:31:49.583021 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.006s	sys 0.018s
W20260924 14:31:49.583464 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 779168us
W20260924 14:31:49.488425 32441 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.040s	sys 0.093s
I20260924 14:31:49.555059 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.558296 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.556659 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:49.514804 31984 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 24 FOLLOWER]: Leader election won for term 24
I20260924 14:31:49.469949 32481 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:49.522130 32345 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260924 14:31:49.585579 32497 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:49.587169 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.593964 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.594604 26402 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:49.588161 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:49.594616 26401 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:49.482237 32326 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:49.585088 32442 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.042s	user 0.001s	sys 0.015s
I20260924 14:31:49.565161 32153 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:49.640607 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 84040.6us
I20260924 14:31:49.648001 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.641407 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:49.587432 32441 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.041s	sys 0.098s
I20260924 14:31:49.650547 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.643257 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.652465 26249 raft_consensus.cc:2396] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260924 14:31:49.651603 26250 raft_consensus.cc:2396] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260924 14:31:49.657528 32479 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:49.657998 32479 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:49.658782 32153 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.661147 32153 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.650712 31984 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 24 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:49.662684 31984 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.667539 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 1065) took 2582 ms. Trace:
I20260924 14:31:49.592756 32497 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:49.669539 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 84661.3us
I20260924 14:31:49.667963 27194 rpcz_store.cc:276] 0924 14:31:47.084906 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.086924 (+  2018us) service_pool.cc:227] Handling call
0924 14:31:47.087369 (+   445us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:48.465864 (+1378495us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.667238 (+1201374us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54002c51d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:49.667477 (+   239us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3162191104}
I20260924 14:31:49.670248 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.680169 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:49.678175 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.678404 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:49.678833 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:49.636430 32433 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.641605 32440 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.693281 32440 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.694300 32306 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.695060 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:49.673727 32485 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.044s	sys 0.087s
I20260924 14:31:49.658810 32479 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.607597 32447 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:49.695881 32485 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.044s	sys 0.088s
I20260924 14:31:49.698509 32497 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:49.698840 32330 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:49.699311 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:49.700186 32440 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.701241 32447 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.676813 26402 raft_consensus.cc:2396] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260924 14:31:49.684379 26295 leader_election.cc:334] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:49.702695 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:49.681429 26252 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
W20260924 14:31:49.703294 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:49.703495 32447 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.639284 32470 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:49.704236 32271 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:49.704838 32481 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.706367 32271 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:49.706959 32271 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:49.687016 26298 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:49.712917 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:49.517581 32496 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.689824 32334 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.689980 32441 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:49.690606 32322 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.691987 32290 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.692678 32345 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:49.640141 31980 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.701187 32433 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.671835 26397 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:49.719959 31335 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:49.651588 32442 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:49.652659 32331 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:49.717737 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.720635 32493 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:49.695811 32458 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.679934 32382 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.725642 32345 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:49.593550 32486 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.699249 32330 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:49.726359 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:49.642845 32449 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.020s	sys 0.055s
I20260924 14:31:49.826066 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 1998) took 1040 ms. Trace:
I20260924 14:31:49.826629 26399 rpcz_store.cc:276] 0924 14:31:48.785380 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:48.786089 (+   709us) service_pool.cc:227] Handling call
0924 14:31:49.760311 (+974222us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54001e41d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:49.825987 (+ 65676us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":399833984}
I20260924 14:31:49.746845 31335 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:49.723424 32334 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.723645 32441 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:49.749027 32331 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:49.824419 32449 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.021s	sys 0.055s
I20260924 14:31:49.752674 32368 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.737556 31980 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.843086 32330 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:49.915450 32166 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.933298 31335 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:49.939638 32331 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:49.770731 32382 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.825201 32271 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.963814 32345 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.050899 32382 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.042217 32290 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:50.042464 32496 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:50.052757 32433 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.063015 32441 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:50.072113 32486 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:50.068881 32345 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:50.123184 32306 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:50.142829 32391 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.002s
I20260924 14:31:50.150374 32391 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.153853 32391 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.682147 26257 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:50.232299 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.234042 26250 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:50.234814 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.232345 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.236083 26252 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:50.231069 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.752066 26298 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:50.247488 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 495326us
W20260924 14:31:50.249424 26295 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:50.251736 26326 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.005s	sys 0.001s
W20260924 14:31:50.252383 26326 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.005s	sys 0.001s
W20260924 14:31:50.256551 26369 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.011s	sys 0.028s
W20260924 14:31:50.257027 26369 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.011s	sys 0.028s
W20260924 14:31:50.257547 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 702386us
I20260924 14:31:50.258618 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.266358 26330 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:50.269712 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.272936 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.274319 26332 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:50.274972 26399 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:50.275755 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.276273 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.277102 26399 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:50.276897 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.277503 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.278148 26401 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:49.799417 26402 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:50.279410 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:50.280685 26404 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.002s	sys 0.019s
I20260924 14:31:50.278158 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:50.279721 26398 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.000s	sys 0.017s
W20260924 14:31:50.298906 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.001s	sys 0.010s
W20260924 14:31:50.300387 26404 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.003s	sys 0.019s
W20260924 14:31:50.300800 31984 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.068s	sys 0.116s
W20260924 14:31:50.301025 27125 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.009s	sys 0.004s
W20260924 14:31:50.301286 31984 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.068s	sys 0.117s
W20260924 14:31:50.301558 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.009s	sys 0.004s
W20260924 14:31:50.280424 26403 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.002s	sys 0.019s
W20260924 14:31:50.302218 26403 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.003s	sys 0.020s
W20260924 14:31:50.302156 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 605398us
I20260924 14:31:50.279876 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:50.363101 26398 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.000s	sys 0.018s
W20260924 14:31:50.300760 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.002s	sys 0.010s
W20260924 14:31:50.380261 26400 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.004s	sys 0.003s
W20260924 14:31:50.380887 26400 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.005s	sys 0.004s
I20260924 14:31:50.387236 26399 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:50.396642 32238 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.397964 32238 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:50.399997 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 692404us
I20260924 14:31:50.405381 32380 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.408001 32368 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.408634 32380 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.409071 32368 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.411641 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.413697 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.414705 26403 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:50.386626 31882 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
W20260924 14:31:50.419665 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 116208us
I20260924 14:31:50.425453 31980 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:50.426365 31980 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.976538 32166 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.438331 32166 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.438485 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.439456 26330 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:50.440196 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.441428 26324 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:50.443073 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.444926 32382 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.445981 32382 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.447811 31945 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:50.448963 31945 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.449788 32425 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.452311 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.467715 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.468858 32466 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.468981 26323 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:50.469597 32466 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:50.471068 32166 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.471674 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.471920 32166 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.472563 26401 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:50.473564 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.475008 32368 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:50.459308 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.479583 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.479804 32290 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:50.482201 32290 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.485301 32153 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.486279 32153 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.487579 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.488672 26332 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:50.489476 31882 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:50.492013 32290 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:50.468047 32380 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.492883 32382 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.493323 32380 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.494156 32380 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.496254 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.496585 32466 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:50.496822 32475 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:50.491511 31882 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.492297 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:50.498399 26403 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:50.450786 32425 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:50.587085 26397 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:50.497733 32475 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.591248 32153 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.596979 26401 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260924 14:31:50.611788 26401 pending_rounds.cc:76] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 10
I20260924 14:31:50.599766 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:50.600167 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.603987 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.604727 26403 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260924 14:31:50.605713 32425 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.427151 31980 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.607124 32433 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.588610 32166 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.625857 26401 pending_rounds.cc:97] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260924 14:31:50.638895 32235 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.639904 32235 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.641270 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1784) took 1129 ms. Trace:
I20260924 14:31:50.641798 26326 rpcz_store.cc:276] 0924 14:31:49.511762 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:49.516757 (+  4995us) service_pool.cc:227] Handling call
0924 14:31:49.551910 (+ 35153us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.552172 (+   262us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:49.580709 (+ 28537us) log.cc:842] Serialized 14 byte log entry
0924 14:31:50.631666 (+1050957us) log.cc:842] Serialized 14 byte log entry
0924 14:31:50.640681 (+  9015us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:50.640744 (+    63us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:50.640834 (+    90us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.640946 (+   112us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:50.641161 (+   215us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":33485,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":163045,"raft.run_cpu_time_us":486,"raft.run_wall_time_us":490,"thread_start_us":1035093,"threads_started":2,"wal-append.queue_time_us":561048}
I20260924 14:31:50.627149 31945 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:50.643640 32331 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.621s	user 0.008s	sys 0.003s
I20260924 14:31:50.669783 32331 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:50.671691 32433 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.642530 32235 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.652927 32153 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.655920 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.674312 32433 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.673707 32153 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.656375 26401 raft_consensus.cc:2866] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:31:50.658524 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:50.677263 32238 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.677824 32238 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:50.678345 26295 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:50.678438 32238 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:50.679657 32290 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260924 14:31:50.661924 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 169846us
I20260924 14:31:50.681151 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:50.682152 32166 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:50.682497 32331 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:50.682940 32166 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.648389 32380 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.663439 32475 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.683916 32331 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:50.685477 32166 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:50.685551 26295 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:50.686371 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:50.707384 26332 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:50.687345 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:50.712051 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1264) took 3537 ms. Trace:
I20260924 14:31:50.712669 26327 rpcz_store.cc:276] 0924 14:31:47.175459 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.342557 (+167098us) service_pool.cc:227] Handling call
0924 14:31:47.342941 (+   384us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:48.689111 (+1346170us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.708866 (+2019755us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540001e0d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:50.711826 (+  2960us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3094374784}
I20260924 14:31:50.723937 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:50.725507 32235 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:50.726241 32235 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:50.726819 32235 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:50.688598 32153 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:50.763770 32306 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:50.764461 32306 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:50.769259 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 83759.5us
I20260924 14:31:50.746162 32433 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:50.764954 32306 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:50.773864 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 108370us
I20260924 14:31:50.781605 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1992) took 1218 ms. Trace:
W20260924 14:31:50.785279 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:50.798007 26298 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:50.799981 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 116709us
W20260924 14:31:50.801642 32438 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 2.362s	user 0.030s	sys 0.043s
I20260924 14:31:50.804060 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1975) took 3317 ms. Trace:
I20260924 14:31:50.805497 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1998) took 1153 ms. Trace:
I20260924 14:31:50.804464 26405 rpcz_store.cc:276] 0924 14:31:47.486169 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.488880 (+  2711us) service_pool.cc:227] Handling call
0924 14:31:47.505201 (+ 16321us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.516172 (+ 10971us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:47.516216 (+    44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:47.569449 (+ 53233us) log.cc:842] Serialized 26 byte log entry
0924 14:31:48.074026 (+504577us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:48.074064 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.074143 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:49.043109 (+968966us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b4800088d58. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:31:50.749885 (+1706776us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:50.803628 (+ 53743us) raft_consensus.cc:1671] finished
0924 14:31:50.803652 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:50.803840 (+   188us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31627,"raft.queue_time_us":62833,"raft.run_cpu_time_us":3389,"raft.run_wall_time_us":1463703,"spinlock_wait_cycles":22954240,"thread_start_us":403147,"threads_started":1,"wal-append.queue_time_us":799935}
I20260924 14:31:50.805904 26398 rpcz_store.cc:276] 0924 14:31:49.652737 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:49.653024 (+   287us) service_pool.cc:227] Handling call
0924 14:31:49.653366 (+   342us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.653594 (+   228us) raft_consensus.cc:1520] Early marking committed up to index 11
0924 14:31:50.431687 (+778093us) log.cc:842] Serialized 14 byte log entry
0924 14:31:50.742716 (+311029us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:50.747564 (+  4848us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:50.775824 (+ 28260us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.786709 (+ 10885us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:50.805435 (+ 18726us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":29260,"prepare.queue_time_us":483032,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":39,"raft.queue_time_us":59560,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":19785,"thread_start_us":967678,"threads_started":2,"wal-append.queue_time_us":314889}
I20260924 14:31:50.812340 32238 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:50.814597 32507 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.115s	user 0.012s	sys 0.030s
I20260924 14:31:50.818121 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.822423 31984 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:50.828686 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1293) took 3452 ms. Trace:
I20260924 14:31:50.829229 26331 rpcz_store.cc:276] 0924 14:31:47.375891 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.800664 (+424773us) service_pool.cc:227] Handling call
0924 14:31:49.445557 (+1644893us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.467296 (+ 21739us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.828238 (+1360942us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400505850. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:50.828492 (+   254us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3774703360}
W20260924 14:31:50.844182 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47558.2us
W20260924 14:31:50.852473 31335 consensus_meta.cc:287] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.846s	user 0.008s	sys 0.008s
I20260924 14:31:50.852993 31335 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.787001 26404 rpcz_store.cc:276] 0924 14:31:49.555935 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:49.556051 (+   116us) service_pool.cc:227] Handling call
0924 14:31:49.597602 (+ 41551us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.597852 (+   250us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:50.301306 (+703454us) log.cc:842] Serialized 14 byte log entry
0924 14:31:50.709753 (+408447us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:50.736596 (+ 26843us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:50.745437 (+  8841us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.753244 (+  7807us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:50.773929 (+ 20685us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11821,"prepare.queue_time_us":535809,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":315,"raft.queue_time_us":243503,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":48208,"thread_start_us":1025091,"threads_started":3,"wal-append.queue_time_us":161013}
I20260924 14:31:50.860703 26369 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:50.861477 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.862308 26404 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:50.863478 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.865078 32424 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:50.815279 32330 consensus_meta.cc:287] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.840s	user 0.012s	sys 0.006s
I20260924 14:31:50.868669 32330 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:50.850111 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 49194.1us
I20260924 14:31:50.855198 31335 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:50.900089 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48718.5us
I20260924 14:31:50.892478 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:50.914256 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:50.863222 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:50.930219 26332 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:50.931798 32530 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:50.936940 32526 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.104s	user 0.002s	sys 0.000s
W20260924 14:31:50.952389 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 89355.1us
I20260924 14:31:50.958339 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.958989 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.960511 26250 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:50.961971 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.962108 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.962863 26251 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:50.963161 26256 raft_consensus.cc:2396] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:50.963965 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.964591 32442 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.965342 32442 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.967085 26252 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:50.965725 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:50.967664 26258 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260924 14:31:50.969143 26258 pending_rounds.cc:76] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 5
I20260924 14:31:50.948704 32153 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:50.940546 32235 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.975152 32334 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.977563 26399 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260924 14:31:50.977982 26258 pending_rounds.cc:97] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260924 14:31:50.978346 26258 raft_consensus.cc:2866] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:50.990557 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.993194 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.999123 31677 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260924 14:31:50.997769 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.002094 26404 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:50.988826 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.002071 32184 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:50.972206 32306 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:51.013810 26399 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:51.003669 32442 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:51.015938 26329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1771) took 3938 ms (client timeout 1657 ms). Trace:
I20260924 14:31:51.016076 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:51.016474 26329 rpcz_store.cc:269] 0924 14:31:47.077534 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.139725 (+ 62191us) service_pool.cc:227] Handling call
0924 14:31:51.015485 (+3875760us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b540001dbd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:51.015659 (+   174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3649623424}
I20260924 14:31:51.017042 26251 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:51.026142 32192 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260924 14:31:51.028515 32517 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:51.031608 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.032421 26254 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:31:51.008425 32460 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.231s	user 0.026s	sys 0.030s
W20260924 14:31:51.036005 32460 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.027s	sys 0.030s
W20260924 14:31:51.037529 31979 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.484s	user 0.008s	sys 0.023s
I20260924 14:31:51.038178 31979 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.040728 31979 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.041898 31980 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.042192 26255 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 18. Current term is 19. Ops: []
I20260924 14:31:51.044746 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.044948 32435 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.045459 26402 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 17
I20260924 14:31:51.047626 32424 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:51.048421 32424 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:51.062765 32325 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:51.074137 32325 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.077179 32330 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:51.079945 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 72722.4us
I20260924 14:31:51.066069 32306 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.072405 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1050) took 2636 ms. Trace:
I20260924 14:31:51.062835 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1650) took 2406 ms. Trace:
I20260924 14:31:51.083612 27192 rpcz_store.cc:276] 0924 14:31:48.655806 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:48.656103 (+   297us) service_pool.cc:227] Handling call
0924 14:31:48.656454 (+   351us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:48.795680 (+139226us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:48.795713 (+    33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:48.795756 (+    43us) raft_consensus.cc:1631] Marking committed up to 12
0924 14:31:48.795788 (+    32us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:51.062182 (+2266394us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400453b54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:51.062269 (+    87us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:51.062584 (+   315us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":319728384}
I20260924 14:31:51.007390 32184 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:51.094527 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.094854 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:51.090273 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 99878us
I20260924 14:31:51.056080 32424 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.098815 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.050647 31335 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:51.092043 32235 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.096508 32330 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.097875 32535 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:51.120340 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:51.120427 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 56701.1us
I20260924 14:31:51.122638 26399 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:51.123356 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:51.130364 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:51.132314 26295 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.136807 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.140162 26253 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:51.121104 32535 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.083302 27181 rpcz_store.cc:276] 0924 14:31:48.435842 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:48.435980 (+   138us) service_pool.cc:227] Handling call
0924 14:31:48.457531 (+ 21551us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:48.543855 (+ 86324us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:48.543905 (+    50us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:48.544531 (+   626us) log.cc:842] Serialized 26 byte log entry
0924 14:31:48.694434 (+149903us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:48.694471 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:51.071604 (+2377133us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b540000ee54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:51.071692 (+    88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:51.071799 (+   107us) raft_consensus.cc:1671] finished
0924 14:31:51.071816 (+    17us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:51.072178 (+   362us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6755,"raft.queue_time_us":10604,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":38479,"spinlock_wait_cycles":181891456,"thread_start_us":149551,"threads_started":1,"wal-append.queue_time_us":183829,"wal-append.run_cpu_time_us":1853,"wal-append.run_wall_time_us":1006740}
I20260924 14:31:51.111658 32306 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260924 14:31:51.155303 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 2004) took 1791 ms. Trace:
I20260924 14:31:51.158179 26406 rpcz_store.cc:276] 0924 14:31:49.363757 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:49.366474 (+  2717us) service_pool.cc:227] Handling call
0924 14:31:49.366754 (+   280us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.389393 (+ 22639us) raft_consensus.cc:1520] Early marking committed up to index 11
0924 14:31:49.413784 (+ 24391us) log.cc:842] Serialized 14 byte log entry
0924 14:31:49.414072 (+   288us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:49.414139 (+    67us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:49.414201 (+    62us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.742362 (+328161us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b1800243160. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11
0924 14:31:50.743964 (+1001602us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b1800243160. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9
0924 14:31:51.154915 (+410951us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b540023fd54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:51.154963 (+    48us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:51.155127 (+   164us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":24123,"raft.queue_time_us":103,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3835,"spinlock_wait_cycles":63040128}
I20260924 14:31:51.157670 32184 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.160569 32325 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.163548 32330 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.164897 26402 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:51.165596 32473 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:51.143392 32535 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.171329 32184 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:51.156378 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.155118 32479 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.127942 31335 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:51.184630 32380 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:51.185336 32380 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:51.160699 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:51.186328 32326 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:31:51.143724 26295 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.187435 32380 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.185627 31335 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:51.187834 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:51.188983 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50387.1us
W20260924 14:31:51.189642 26295 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:51.190866 26295 leader_election.cc:334] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.192144 32153 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:51.200966 32153 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:51.200626 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.201905 26397 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:51.203018 32529 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.371s	user 0.001s	sys 0.001s
I20260924 14:31:51.193958 32440 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:51.205860 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:51.207237 32473 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:51.213402 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.212901 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:51.218210 32538 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260924 14:31:51.205937 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.211243 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.218432 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.213614 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:51.191414 26295 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:51.237370 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45973us
I20260924 14:31:51.237944 26258 raft_consensus.cc:2396] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260924 14:31:51.227927 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 87786.5us
I20260924 14:31:51.218815 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.233736 26251 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
W20260924 14:31:51.238631 26295 leader_election.cc:334] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.240756 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.238862 26254 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:51.246817 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:51.244660 26295 leader_election.cc:334] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:51.248970 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.253908 26255 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:51.250101 31945 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:51.259807 31945 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.260154 31945 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.266637 26253 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:51.269685 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:51.272104 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 157366us
I20260924 14:31:51.298692 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.307498 26397 raft_consensus.cc:2396] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260924 14:31:51.306190 26249 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:51.282498 31335 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:51.314976 32450 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:51.315718 32330 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.315745 32450 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:51.317306 32450 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.317708 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:51.319777 32424 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:51.321601 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.322487 26332 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:51.323614 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.323921 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.322384 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.322288 32517 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Stepping down as leader of term 18
I20260924 14:31:51.333993 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.351938 26397 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260924 14:31:51.355993 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.357893 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.360239 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.360924 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:51.361778 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.362594 26401 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:51.365592 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.367873 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.361033 26404 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:51.369992 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 2005) took 1098 ms. Trace:
I20260924 14:31:51.373836 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.375140 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.334072 32166 raft_consensus.cc:2741] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260924 14:31:51.381827 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:51.352401 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:51.356487 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:51.374521 26332 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:51.374315 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.380321 26400 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:51.322510 32259 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:51.385443 26331 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260924 14:31:51.370566 26403 rpcz_store.cc:276] 0924 14:31:50.271187 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:50.728993 (+457806us) service_pool.cc:227] Handling call
0924 14:31:50.729544 (+   551us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:50.730088 (+   544us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:50.730135 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:50.731003 (+   868us) log.cc:842] Serialized 26 byte log entry
0924 14:31:50.890723 (+159720us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:50.890847 (+   124us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.890950 (+   103us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:51.369695 (+478745us) raft_consensus.cc:1671] finished
0924 14:31:51.369728 (+    33us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:51.369919 (+   191us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":159129,"threads_started":1,"wal-append.queue_time_us":558842}
I20260924 14:31:51.340313 32517 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:51.388294 26295 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.404533 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.382737 32530 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:51.406013 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:51.406281 32530 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.407986 32527 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:51.408658 32530 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.408583 32527 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.409360 32530 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:51.409776 32530 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:51.410197 32530 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:51.410673 32527 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.393541 32466 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:51.411916 32235 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.412640 32235 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.413657 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.412631 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.414714 26397 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:51.415946 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.417178 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.417598 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.418174 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:51.421937 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:51.422966 26295 leader_election.cc:334] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:51.421857 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 147536us
W20260924 14:31:51.398543 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 73439us
I20260924 14:31:51.418268 26403 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:51.415205 32235 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.398093 32517 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.436349 32494 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.436961 32494 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.437418 32517 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:51.440757 32494 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:51.433573 32153 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.445209 32153 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.411250 32527 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260924 14:31:51.446781 32527 raft_consensus.cc:3032] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:51.447176 32527 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:51.447019 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.448484 32527 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.449220 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.450428 32166 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:51.451061 32166 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.453503 32166 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:51.427583 26398 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
W20260924 14:31:51.423609 26295 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.456125 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.384543 32259 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:51.457845 32259 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
W20260924 14:31:51.458613 26295 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.461553 32330 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:51.462324 32330 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:51.463944 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.465258 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.469264 26249 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:51.437457 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:51.481452 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:51.456491 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:51.423291 32497 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.018s	sys 0.041s
I20260924 14:31:51.493702 26256 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:31:51.499934 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.506871 26298 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:51.513264 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 75968.1us
W20260924 14:31:51.500800 32497 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.019s	sys 0.041s
I20260924 14:31:51.468104 32494 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.524825 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:51.504945 32527 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:51.531826 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 169767us
W20260924 14:31:51.534209 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 82748.4us
I20260924 14:31:51.513091 32153 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:51.541915 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.520990 32331 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:51.526248 32442 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:51.558748 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:51.559592 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.567755 26404 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:51.560223 26405 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:51.572908 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:51.496024 32330 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.554665 32424 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:51.554189 32325 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:51.587210 32458 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:51.587955 32458 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:51.589100 32458 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.562145 32331 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:51.592566 32331 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.583668 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.571614 32473 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.551795 32494 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:51.606115 32494 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.599964 26255 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:51.579514 32424 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.607308 32424 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:51.602381 32535 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.610306 32535 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.574833 32440 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:31:51.633488 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 183835us
I20260924 14:31:51.643994 32390 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:51.644547 32390 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.644924 32390 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.647923 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.648764 26254 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:51.650306 31945 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.614233 32473 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.642259 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.659713 26255 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 16
I20260924 14:31:51.662323 26400 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 23. Current term is 24. Ops: []
I20260924 14:31:51.665565 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.664454 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.668308 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.666568 26404 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:51.672627 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.673288 26324 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:51.674207 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.617426 32325 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.676527 26252 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:51.679184 26298 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:51.683303 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.684162 26328 raft_consensus.cc:2354] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18.
I20260924 14:31:51.685387 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.688282 31945 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.688891 32473 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.686244 26331 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:51.698709 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:51.703994 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.704879 26328 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.624716 32546 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260924 14:31:51.707434 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:51.712908 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:51.642640 32461 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.182s
I20260924 14:31:51.718211 32503 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:51.717324 32530 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.714243 31335 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:51.709404 31945 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:51.661270 32547 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.078s
I20260924 14:31:51.736289 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:51.718487 32153 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:51.740502 32535 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:51.741014 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:51.730968 32462 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.014s	sys 0.017s
I20260924 14:31:51.768397 26399 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:51.745785 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.726027 32503 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.752131 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.786031 26257 raft_consensus.cc:2396] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260924 14:31:51.759001 32153 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:51.768826 32462 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.016s	sys 0.019s
I20260924 14:31:51.743733 32325 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.763026 31335 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:51.784186 31945 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:51.788197 32503 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:51.767371 32535 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.798944 32153 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:51.843257 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 261217us
I20260924 14:31:51.833592 32535 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:51.847949 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:51.848892 32530 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:51.850831 32326 consensus_meta.cc:287] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.664s	user 0.008s	sys 0.012s
I20260924 14:31:51.859540 26329 raft_consensus.cc:2396] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260924 14:31:51.861217 32259 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.863540 31980 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.877014 31980 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.878639 32380 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:51.879143 32380 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:51.864384 32550 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
W20260924 14:31:51.872566 32184 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.000s
I20260924 14:31:51.874683 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:51.889082 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.887887 26255 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:51.892210 26328 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:51.895988 26252 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:51.902287 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.910192 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.909857 26369 leader_election.cc:398] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18.
I20260924 14:31:51.912016 26369 leader_election.cc:401] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260924 14:31:51.913872 26256 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:51.915767 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.915849 31979 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260924 14:31:51.916790 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.917615 32535 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.919018 32259 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.921362 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.921162 26331 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 23 LEADER]: Stepping down as leader of term 23
I20260924 14:31:51.921906 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:51.909969 26295 leader_election.cc:334] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.922246 26331 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:51.924062 26327 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:51.930701 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.933703 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:51.934957 26295 leader_election.cc:334] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.857474 32424 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.936640 32390 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.938568 32424 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.939513 32390 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.944216 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.944496 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.945458 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.946022 26253 raft_consensus.cc:2362] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16. Re-sending same reply.
I20260924 14:31:51.946887 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.947836 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.949105 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.950704 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.933952 26252 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:51.952576 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.955224 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.928681 32473 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.966485 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.968885 26327 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260924 14:31:51.964476 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.958622 32535 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.966588 26255 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:31:51.975067 26249 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:51.982177 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:51.989794 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.990736 26404 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
I20260924 14:31:51.997581 26398 raft_consensus.cc:2396] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260924 14:31:52.001816 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 111674us
I20260924 14:31:51.911372 31335 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.007148 26400 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:51.972741 32075 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
W20260924 14:31:52.025260 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 150675us
I20260924 14:31:51.993268 32326 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:52.003687 32462 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:52.005106 32442 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:51.994841 31980 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.930104 31945 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.030694 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:52.016215 32527 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:52.040948 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:52.037285 26224 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:52.047415 32497 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:52.047904 32497 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:52.047807 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:52.048321 32497 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:52.048533 32332 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:52.049528 32326 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:52.052232 32535 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:52.052801 32535 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:52.053608 32535 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.032116 32462 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:52.036988 32563 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:52.081399 26249 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:52.068086 32424 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260924 14:31:52.078478 26295 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:52.095713 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 98494.6us
I20260924 14:31:52.042964 32527 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.065063 32153 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:52.110328 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 81401.7us
I20260924 14:31:52.091013 32424 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:52.115408 26324 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 18: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:52.110351 32171 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
I20260924 14:31:52.126031 32462 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.127298 32462 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:52.116421 32424 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:52.135972 32557 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.272s	user 0.003s	sys 0.000s
I20260924 14:31:52.113031 32326 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:52.143678 32534 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:52.144304 26331 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.145915 26331 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 23 FOLLOWER]: Advancing to term 24
I20260924 14:31:52.148821 32527 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:52.154994 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:52.157902 26331 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260924 14:31:52.098584 32450 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260924 14:31:52.159865 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 117493us
I20260924 14:31:52.165899 26325 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:52.106096 32497 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:52.160219 32325 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 24 current term: 24 is higher.
I20260924 14:31:52.177527 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.178395 26398 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260924 14:31:52.178717 26331 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:52.182740 32497 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:52.184312 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:52.186067 32502 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:52.186717 32502 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:52.176628 32517 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.739s	user 0.011s	sys 0.014s
I20260924 14:31:52.187991 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:52.189293 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:52.191239 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:52.197059 26295 leader_election.cc:334] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:52.198933 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:52.191957 26406 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
W20260924 14:31:52.207842 26371 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.865s (SENT)
I20260924 14:31:52.208534 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.189235 32424 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.212496 32424 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:52.225754 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:52.227630 26331 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260924 14:31:52.227784 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:52.232801 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:52.233537 26399 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 19
I20260924 14:31:52.235737 32497 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:52.237885 26255 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:52.201828 32473 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18.
I20260924 14:31:52.240743 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:52.241755 32558 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:52.242779 32558 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:52.243116 32558 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:52.244007 32558 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.244768 32534 consensus_queue.cc:1046] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:52.245586 32558 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:52.246260 32534 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:52.246753 32534 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:52.251907 32547 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:52.252408 32547 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:52.252769 32547 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:52.259868 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 148631us
I20260924 14:31:52.261140 32153 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:52.262382 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:52.263759 32330 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:52.264281 32330 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:52.264739 32330 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:52.267403 32442 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:52.267904 32442 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:52.268694 32442 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.270123 32442 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:52.270179 32534 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.272338 32534 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:52.275218 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 146328us
I20260924 14:31:52.236054 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:52.284981 26399 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:52.290397 26403 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:31:52.291146 26405 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:52.291640 26398 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260924 14:31:52.295112 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 64655.2us
I20260924 14:31:52.297609 32458 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:31:52.298691 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:52.300496 32424 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:52.301232 32424 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 19 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:52.303936 32568 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.151s	user 0.002s	sys 0.000s
I20260924 14:31:52.305686 26406 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:52.306209 32380 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.318351 32380 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:52.312089 32565 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260924 14:31:52.283803 32503 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:52.326342 32503 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:52.293217 32332 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
W20260924 14:31:52.333884 32562 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260924 14:31:52.315672 32204 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:52.302453 32424 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:52.339372 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:52.343058 26256 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:31:52.343551 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46635.7us
I20260924 14:31:52.352046 32204 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:52.351471 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 85612.2us
I20260924 14:31:52.380795 26257 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260924 14:31:52.383919 32204 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:52.389550 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:52.392059 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:52.400482 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 53158.2us
I20260924 14:31:52.396224 26256 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
W20260924 14:31:52.410696 26297 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.724s (SENT)
I20260924 14:31:52.411123 26297 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:52.417820 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:52.416689 32325 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:52.419400 32506 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:52.421131 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:52.423038 32506 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:52.424945 32458 raft_consensus.cc:2741] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260924 14:31:52.424314 32506 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:52.447762 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 58349.8us
I20260924 14:31:52.448585 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:52.452453 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47099.7us
I20260924 14:31:52.453928 32330 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:52.420382 32450 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:52.458377 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:52.459306 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 362604us
W20260924 14:31:52.468044 32580 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260924 14:31:52.469444 32559 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.427s	user 0.000s	sys 0.001s
I20260924 14:31:52.471333 32184 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Stepping down as leader of term 19
I20260924 14:31:52.471753 32184 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:52.472620 32184 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:52.473387 32200 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:52.474275 32184 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:52.463249 32325 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260924 14:31:52.477206 32325 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:52.477540 32325 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Advancing to term 22
W20260924 14:31:52.494415 32569 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.351s	user 0.002s	sys 0.000s
I20260924 14:31:52.489197 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.501771 31945 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260924 14:31:52.504709 26327 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:52.505085 32326 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:31:52.506075 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.506733 26327 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:52.506830 32473 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:52.507386 32473 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:52.507941 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:52.508639 26398 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:52.508520 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.509552 32473 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:52.509949 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.510470 32330 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:52.510561 26327 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:52.511919 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.513236 26327 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:52.513275 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:52.502858 32573 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:52.514990 32573 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:52.515877 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.515946 32573 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.517328 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.519059 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:52.520907 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 72374.4us
I20260924 14:31:52.524626 32547 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.528579 32547 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:52.532023 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.533057 32153 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:52.533732 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:52.537415 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.538357 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:52.543618 26402 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:52.548763 31335 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:52.442281 32380 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.055s
I20260924 14:31:52.550556 32380 consensus_queue.cc:1225] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260924 14:31:52.550642 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.551146 32380 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:52.551497 32380 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:52.552374 32380 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.553521 32380 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:52.554054 31335 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:52.555593 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:52.556408 31335 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:52.558265 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.561064 32204 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:52.563794 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:52.569803 32204 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:52.570319 32204 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:52.570699 32204 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:52.572333 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:52.573179 31335 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:52.583638 31335 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:52.584091 31335 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:52.560366 26402 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:52.582064 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.586145 32153 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:52.586539 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:52.590847 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.583648 31984 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:52.590936 32424 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:52.586549 32153 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:52.600819 32543 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:52.632337 32462 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260924 14:31:52.636931 32543 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:52.640056 31984 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:52.642356 31984 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:52.643496 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:52.644174 26257 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:52.604497 32442 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:52.646878 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:52.592157 32517 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:52.616228 32534 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260924 14:31:52.647567 26406 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:52.645619 27125 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:52.650882 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:52.604594 26402 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260924 14:31:52.648437 31335 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:52.636066 32153 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:52.639277 32543 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:52.655133 32424 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:52.631099 32325 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.703016 31984 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:52.753634 31984 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:52.756948 31984 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:52.656816 26402 pending_rounds.cc:76] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 7
I20260924 14:31:52.823648 26402 pending_rounds.cc:97] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260924 14:31:52.824333 26402 raft_consensus.cc:2866] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:31:52.988426 26371 leader_election.cc:334] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.680s (SENT)
I20260924 14:31:52.994102 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:52.996181 31335 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:52.997082 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 298773us
I20260924 14:31:53.003644 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:52.999940 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 395626us
I20260924 14:31:52.986439 32380 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:52.994764 32075 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:53.020236 32075 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:53.029249 32075 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:53.019434 32153 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.008416 31945 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:53.049000 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:53.004690 32543 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.060014 31945 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.021174 32380 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.062407 31945 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:53.063131 32380 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:53.064092 32380 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:53.067809 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:53.068552 31984 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.069576 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:53.070919 31984 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.071412 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:53.073141 32204 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.013s
W20260924 14:31:53.115362 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 125207us
I20260924 14:31:53.112514 32200 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.119395 32200 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.121632 32200 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.076859 32458 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:53.118623 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.123533 32547 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:53.125473 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.126345 32547 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:53.115123 31945 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:53.127692 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:53.071627 31979 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:53.071817 32326 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:53.071923 32573 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:31:53.136708 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 65376.6us
I20260924 14:31:53.138464 26297 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:53.141387 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:53.144403 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.087411 32330 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.077189 32543 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.136168 32547 raft_consensus.cc:2741] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260924 14:31:53.155747 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:53.139672 32326 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:53.145670 32558 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:53.162235 32547 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:53.162701 32547 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:53.163051 32547 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.166015 32075 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.152043 32506 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.150300 31979 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:53.179883 32153 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.189881 26073 catalog_manager.cc:5495] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:53.190513 32506 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.110523 32424 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.140332 32204 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:53.197633 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 59286.1us
I20260924 14:31:53.204541 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.153299 32330 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.153211 32380 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:53.158761 32326 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:53.168326 32558 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:53.168069 32325 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:53.222988 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 67351.9us
I20260924 14:31:53.224927 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:53.184214 32473 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.191335 31979 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:53.218511 32153 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:53.193851 32506 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.243952 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:53.248334 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.249212 26406 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.226982 32558 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:53.250931 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:53.240164 32461 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:53.256453 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:53.251948 26331 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:53.257198 26330 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:53.257922 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:53.259865 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:53.259919 26325 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:53.237305 31979 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.264537 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:53.269102 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:53.256394 32380 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260924 14:31:53.278980 26369 leader_election.cc:334] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.279541 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.235947 32473 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.259493 32204 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:53.286692 26326 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260924 14:31:53.286960 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.256634 32506 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260924 14:31:53.289402 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:53.290596 32458 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:31:53.292167 26369 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.293078 26369 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.293445 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.294148 32458 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260924 14:31:53.295141 26369 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.259146 32461 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:53.296152 26369 leader_election.cc:334] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.296926 26330 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
W20260924 14:31:53.297374 26369 leader_election.cc:334] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.298066 26323 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260924 14:31:53.298286 26369 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.281268 32075 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.301066 32473 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:53.303604 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:31:53.304409 26369 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.305109 26369 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.305760 26369 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.306391 26369 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.306747 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.297412 32461 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:53.311126 26406 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
W20260924 14:31:53.312691 26369 leader_election.cc:334] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.313241 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 100896us
I20260924 14:31:53.314919 26404 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:53.313144 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.311831 32473 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260924 14:31:53.281968 32380 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:53.321681 26369 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.324366 26369 leader_election.cc:334] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.325845 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.317786 32458 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:53.333889 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.321054 32380 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:53.343916 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.344509 26257 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.345340 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:53.348490 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:53.346020 26402 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
W20260924 14:31:53.365410 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 76813.1us
W20260924 14:31:53.338816 26295 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 121 similar messages]
I20260924 14:31:53.366269 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:53.328912 32543 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:53.369589 26328 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:31:53.339811 32473 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:53.376360 26256 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260924 14:31:53.377625 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.345198 32330 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.380882 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.391834 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.401221 26399 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 19: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.397135 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.385077 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.398339 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:53.403090 26254 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
W20260924 14:31:53.386397 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47619.9us
I20260924 14:31:53.379416 26256 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260924 14:31:53.409842 32596 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:53.413681 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.415938 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:53.416843 26323 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:53.416437 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:53.408656 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:53.420892 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 54707.6us
I20260924 14:31:53.424640 32598 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:53.425694 32598 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:53.426046 32598 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:53.428251 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 439818us
W20260924 14:31:53.429039 26371 leader_election.cc:334] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.721s (SENT)
I20260924 14:31:53.429602 32506 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:53.405462 32332 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260924 14:31:53.431659 32517 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:53.433318 32479 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:53.430884 32503 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:53.424142 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.434419 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.434634 26257 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:53.421761 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.435693 26404 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
W20260924 14:31:53.439414 26295 leader_election.cc:334] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.394452 32547 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.442005 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:53.443231 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.385510 32594 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260924 14:31:53.421012 32330 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader election won for term 20
W20260924 14:31:53.446696 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 104107us
I20260924 14:31:53.408381 32543 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
W20260924 14:31:53.445425 26295 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.451171 26295 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.451838 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:53.452239 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:53.455788 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:53.457816 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.461352 32204 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:31:53.462910 32599 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:53.463438 32599 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:53.463832 32599 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:53.462954 26075 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 142) took 33855 ms (client timeout 15000 ms). Trace:
I20260924 14:31:53.469635 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:53.388162 26073 catalog_manager.cc:5495] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:53.456880 32458 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:53.491776 32458 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:53.492182 32458 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:53.492514 26073 catalog_manager.cc:5495] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 15, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:53.494550 32547 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.465968 32517 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:53.453274 32600 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:53.455193 32332 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:53.497890 31979 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.500804 26329 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:53.503474 32200 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:53.452885 32424 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:53.504117 32534 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Stepping down as leader of term 19
I20260924 14:31:53.504586 32534 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:53.453248 32330 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 20 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:53.505446 32534 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.505900 32330 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.506917 32534 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.507268 32171 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260924 14:31:53.476202 26075 rpcz_store.cc:269] 0924 14:31:19.553112 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:19.553408 (+   296us) service_pool.cc:227] Handling call
0924 14:31:20.342433 (+789025us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b540000ab90. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eed92 00007f09bf33f7c9 00007f09bf2d7c36 00007f09bf2a8c28 00007f09bf3a2c3e 00007f09bd6b4fb4 00007f09bd6b4f42 00007f09bd6b4e71 00007f09bd6b4dec
0924 14:31:53.408084 (+33065651us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:46.493288 (+     0us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b64085fe 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6423155 00007f09bcd5c714
0924 14:31:46.736380 (+243092us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:46.749321 (+ 12941us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:46.749379 (+    58us) write_op.cc:435] Acquired schema lock
0924 14:31:46.749397 (+    18us) tablet.cc:660] Decoding operations
0924 14:31:46.750746 (+  1349us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:46.750903 (+   157us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:46.750960 (+    57us) tablet.cc:683] Acquiring locks for 9 operations
0924 14:31:46.883101 (+132141us) tablet.cc:699] Row locks acquired
0924 14:31:46.896184 (+ 13083us) write_op.cc:260] PREPARE: finished
0924 14:31:46.927647 (+ 31463us) write_op.cc:270] Start()
0924 14:31:46.943597 (+ 15950us) write_op.cc:276] Timestamp: P: 1790260306917637 usec, L: 0
0924 14:31:46.950559 (+  6962us) op_driver.cc:331] REPLICATION: starting
0924 14:31:46.992740 (+ 42181us) log.cc:842] Serialized 3672 byte log entry
0924 14:31:48.333028 (+1340288us) op_driver.cc:444] REPLICATION: finished
0924 14:31:48.711889 (+378861us) write_op.cc:301] APPLY: starting
0924 14:31:48.728270 (+ 16381us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:49.021195 (+292925us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:49.031463 (+ 10268us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:49.666458 (+634995us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:49.666747 (+   289us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0924 14:31:49.666808 (+    61us) write_op.cc:312] APPLY: finished
0924 14:31:49.696872 (+ 30064us) log.cc:842] Serialized 88 byte log entry
0924 14:31:50.762287 (+1065415us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b64085fe 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb
0924 14:31:51.036672 (+274385us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:51.037195 (+   523us) write_op.cc:454] Released schema lock
0924 14:31:51.037325 (+   130us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:51.037802 (+   477us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:51.037845 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":119640,"reported_tablets":43,"spinlock_wait_cycles":5177088,"child_traces":[["op",{"apply.queue_time_us":334072,"apply.run_cpu_time_us":87345,"apply.run_wall_time_us":2389071,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41783,"num_ops":9,"prepare.queue_time_us":339734,"prepare.run_cpu_time_us":22949,"prepare.run_wall_time_us":672295,"raft.queue_time_us":621451,"raft.run_cpu_time_us":15180,"raft.run_wall_time_us":358471,"replication_time_us":1308737,"spinlock_wait_cycles":93513472,"thread_start_us":2369379,"threads_started":5,"wal-append.queue_time_us":1773188,"wal-append.run_cpu_time_us":72539,"wal-append.run_wall_time_us":2584810}]]}
W20260924 14:31:53.523561 26295 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.501619 32326 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.533610 26324 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:53.534732 32200 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:53.535043 32326 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.535256 32200 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:53.536163 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 66629.8us
I20260924 14:31:53.536558 32596 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:53.556915 27125 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:53.559993 32517 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:53.560177 32200 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.560863 26323 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260924 14:31:53.561352 32200 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:53.524962 32558 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.494880 26073 catalog_manager.cc:5495] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:53.502162 32380 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.565840 26073 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:53.566967 32380 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.543433 31979 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.567725 26249 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:53.558961 32332 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.569968 32171 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:53.570196 32332 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.572930 32332 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.582193 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.584445 26402 raft_consensus.cc:2396] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260924 14:31:53.559083 26369 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.565071 32558 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:53.551347 32598 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.607547 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:53.600512 26369 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.610527 26369 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.611014 26323 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 22: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:53.583560 32597 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260924 14:31:53.616779 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 168019us
I20260924 14:31:53.625013 26249 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260924 14:31:53.611210 26369 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.633561 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 187477us
I20260924 14:31:53.634689 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:53.640317 26403 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:53.640399 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:53.641964 26224 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:53.643422 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.644357 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.648468 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.649479 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.645613 26257 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:53.663857 26255 raft_consensus.cc:2379] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.667052 26258 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:31:53.676082 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61397.8us
I20260924 14:31:53.677666 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.677914 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.677603 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.680976 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.681713 26401 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Advancing to term 22
I20260924 14:31:53.681087 26406 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:31:53.685431 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.686390 26397 raft_consensus.cc:2354] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16.
I20260924 14:31:53.687307 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.688479 32326 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260924 14:31:53.689247 26371 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.621s (SENT)
I20260924 14:31:53.689718 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.688537 32442 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260924 14:31:53.691789 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.688457 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.689714 32326 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:53.695763 26401 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 22.
I20260924 14:31:53.696202 32326 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:53.698279 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.679527 26257 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:53.699301 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.700091 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.700976 26258 raft_consensus.cc:2396] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260924 14:31:53.700560 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.698330 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.700452 26251 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:53.701704 26252 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:53.705188 26254 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 20: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.706142 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.707093 26250 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:53.639798 32380 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:53.716629 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.717913 26399 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260924 14:31:53.720041 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.720325 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.723839 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.723839 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.724848 26253 raft_consensus.cc:2379] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.725431 26257 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:53.721335 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.727610 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.728334 26255 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:53.734812 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:53.735033 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45815.5us
I20260924 14:31:53.735249 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.735515 26254 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:53.636185 32543 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260924 14:31:53.741254 26252 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:53.741336 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.637768 32458 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.745267 26257 raft_consensus.cc:2354] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20.
I20260924 14:31:53.747388 26298 leader_election.cc:398] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16.
I20260924 14:31:53.747779 26298 leader_election.cc:401] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260924 14:31:53.721480 32330 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:53.725014 32380 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:53.734653 32595 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.728421 32473 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:53.732586 32558 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:53.640242 32600 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260924 14:31:53.724385 32607 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:53.587644 26073 catalog_manager.cc:5495] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:53.758195 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.758649 26404 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260924 14:31:53.763352 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.697794 32326 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.761281 32558 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:53.772840 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.773054 32598 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:53.757447 32599 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.757329 32380 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:53.756721 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:53.797781 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.752995 32479 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:31:53.778129 26254 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:53.790449 26249 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260924 14:31:53.762208 32442 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:53.788552 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.801044 26249 pending_rounds.cc:76] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 10
I20260924 14:31:53.799816 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.804158 26249 pending_rounds.cc:97] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260924 14:31:53.804433 26252 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 20: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.804494 26249 raft_consensus.cc:2866] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:53.806677 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.807497 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.808220 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.808871 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.809516 26252 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:53.809433 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.810662 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.812013 32599 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.774987 32607 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:53.823885 26297 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.630s (SENT)
I20260924 14:31:53.824342 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.817499 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.790762 32461 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:53.827091 26254 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:53.826313 26257 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:53.791849 32558 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:53.835966 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.832741 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.835296 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.838572 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.843014 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.844211 26258 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:53.838927 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:53.839483 26371 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.512s (SENT)
I20260924 14:31:53.845952 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.845117 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.820689 32326 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:53.761610 32330 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.853302 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.856040 26257 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260924 14:31:53.761901 32595 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.865175 26254 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:53.871145 32171 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:53.854852 31979 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:53.838070 32440 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader election won for term 22
I20260924 14:31:53.853649 32473 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:53.875080 32595 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:53.875824 26073 catalog_manager.cc:5495] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 18 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:53.877808 26073 catalog_manager.cc:5495] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 16, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:31:53.880033 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:53.884178 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.829090 32607 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.886901 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.894356 26252 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.894146 32380 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.902865 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:53.906098 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 106506us
I20260924 14:31:53.929477 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:53.970011 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 45815.5us
I20260924 14:31:53.971771 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.972496 26402 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:53.973353 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.974222 26257 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260924 14:31:53.986474 23135 thread.cc:536] Waited for 1000ms trying to join with heartbeat (tid 27223)
W20260924 14:31:54.015793 26297 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.085s (SENT)
W20260924 14:31:54.022351 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 50439.7us
I20260924 14:31:54.023753 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.024580 26252 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 20: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.902251 32440 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:54.030866 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.039733 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.040647 26332 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:54.043083 32479 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:31:54.044100 32440 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.017827 32461 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:54.047586 32458 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:54.052194 26403 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:54.055691 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.056483 26257 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 15
I20260924 14:31:54.057449 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.065179 26257 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
W20260924 14:31:54.075902 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 275995us
I20260924 14:31:54.018571 32573 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260924 14:31:54.091334 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.115940 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.116513 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.097544 26329 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:54.120867 26327 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 20: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:54.067766 32632 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:54.126499 26252 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:54.127324 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.128177 26328 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 18: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:54.047339 32593 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260924 14:31:54.120016 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.158991 32380 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:54.096016 32600 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:54.167794 26324 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:54.069476 32624 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.063s
I20260924 14:31:54.172185 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:54.171547 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.173079 26402 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:54.179131 32595 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:54.179711 32595 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:54.180099 32595 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:54.164985 32380 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.181811 32380 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.183933 32596 raft_consensus.cc:3032] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:54.184211 32380 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.184384 32596 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:54.185355 32596 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.186592 32596 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:54.189162 26404 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260924 14:31:54.191860 32592 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:54.192291 32592 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:54.161191 32424 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.131165 32632 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:54.173720 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 116368us
I20260924 14:31:54.191509 32595 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.199245 26371 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:54.193886 26369 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.204763 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.213943 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.214851 26404 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:31:54.224597 26224 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:31:54.225675 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 182507us
I20260924 14:31:54.224586 27125 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:54.174618 32624 consensus_queue.cc:1225] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260924 14:31:54.228981 32625 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:54.233176 32623 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.236176 26326 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:54.019680 26073 catalog_manager.cc:5495] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 14, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 14 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.238217 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.211050 26402 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:54.193845 32631 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.216463 32458 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:54.264175 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.264050 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.218447 32600 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:54.202100 32632 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:54.222932 32204 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.193066 32592 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.264705 32479 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:54.263389 32380 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260924 14:31:54.300925 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62895.2us
I20260924 14:31:54.279109 32330 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.214787 32424 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.281639 32461 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:54.311053 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.315575 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.283273 32204 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.318457 32204 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.297561 32458 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:54.274173 32473 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260924 14:31:54.309149 32380 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:54.278172 32645 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260924 14:31:54.266738 32623 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:54.347661 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 120659us
W20260924 14:31:54.359153 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 131626us
I20260924 14:31:54.340209 32592 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:54.353127 26402 pending_rounds.cc:76] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 8
I20260924 14:31:54.364092 26402 pending_rounds.cc:97] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260924 14:31:54.364445 26402 raft_consensus.cc:2866] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:54.349933 32380 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:54.273638 32631 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:54.370672 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 55169.8us
W20260924 14:31:54.370051 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 105966us
W20260924 14:31:54.372233 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 136952us
I20260924 14:31:54.348644 32597 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260924 14:31:54.381369 32633 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:54.381928 26254 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260924 14:31:54.386009 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.386622 26251 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:54.366317 31979 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:54.393430 32326 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260924 14:31:54.394192 32623 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.394088 32632 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.395067 32643 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260924 14:31:54.395568 32442 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260924 14:31:54.383929 26371 leader_election.cc:398] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20.
I20260924 14:31:54.407768 26371 leader_election.cc:401] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:54.410018 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.401927 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.412864 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:54.427006 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48948.3us
I20260924 14:31:54.428336 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.431097 26399 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:54.433148 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:54.437664 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 273329us
I20260924 14:31:54.347289 32642 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260924 14:31:54.413568 32473 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20.
I20260924 14:31:54.465081 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.430202 32645 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:54.424355 32610 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 19 LEADER]: Stepping down as leader of term 19
I20260924 14:31:54.436057 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.468123 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.470954 32591 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:54.468537 32646 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260924 14:31:54.469278 32650 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260924 14:31:54.468730 32607 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.468585 32649 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:54.470508 32648 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260924 14:31:54.424078 32633 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:54.473215 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:54.474982 32517 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260924 14:31:54.475806 32604 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:54.477244 32506 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:54.479218 32330 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:54.436030 32651 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:54.497360 32631 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.417826 32442 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16.
W20260924 14:31:54.505689 26221 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.507296 32610 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:54.414682 26073 catalog_manager.cc:5495] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 16, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:54.522476 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.575793 26404 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:54.520208 32646 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:54.533653 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.522336 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.579659 32646 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:54.574620 26073 catalog_manager.cc:5495] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:54.437074 32632 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:54.514333 32654 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:31:54.536988 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 63846.4us
I20260924 14:31:54.539774 32649 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.536485 32506 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:54.584721 32506 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.586040 32506 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:54.530495 31979 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:54.587718 31979 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.589792 31979 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:54.536177 32604 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:54.543613 32461 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:54.544301 32652 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260924 14:31:54.544992 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 66604.5us
I20260924 14:31:54.591943 32652 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:54.592350 32652 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:54.594515 26406 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260924 14:31:54.534139 32424 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.596886 32649 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.590837 32604 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.597996 32604 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:54.600555 32461 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:54.601225 26403 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
W20260924 14:31:54.538471 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 102523us
I20260924 14:31:54.547269 32171 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.549075 32619 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.513151 32479 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:54.549551 32558 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.713s	user 0.012s	sys 0.000s
W20260924 14:31:54.550148 26295 leader_election.cc:334] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.549885 32651 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.539244 32330 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:54.551386 32458 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:54.557090 32607 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:54.568140 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.422348 32595 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:54.583055 32610 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.607082 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.546638 32380 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.581461 26073 catalog_manager.cc:5495] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:54.610445 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.615352 32461 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260924 14:31:54.614059 26332 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
W20260924 14:31:54.615123 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 65017.3us
W20260924 14:31:54.639631 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:54.615622 32619 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:54.619509 26404 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:54.619976 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:54.626719 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 121070us
I20260924 14:31:54.644484 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:54.617211 32458 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:54.625101 32330 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.621637 32442 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:31:54.616338 32558 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:54.648048 32461 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:54.648375 32621 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:54.649366 31979 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:54.649240 32621 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.649921 31979 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:54.650929 31979 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.647168 32603 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:31:54.651593 26256 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:54.626346 32651 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.651278 32621 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.652635 32619 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:54.653322 32461 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:54.654270 32442 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:54.616207 32171 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.654253 32461 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:54.643213 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:54.647060 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:54.656028 26221 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:54.656965 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:54.657050 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.657182 32595 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260924 14:31:54.657537 26221 leader_election.cc:334] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.659068 32442 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:54.657176 32461 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:54.659215 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.669698 32458 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.663594 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.670831 32458 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:54.649278 32380 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.671528 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:54.672633 32171 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.673436 32424 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:54.674584 26221 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.674989 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.676828 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.677711 26405 raft_consensus.cc:2379] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:54.667657 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.678238 32442 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:54.678140 32652 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.681317 32610 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:54.681607 32424 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:54.664379 32558 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:54.663198 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.677515 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.653941 32479 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:54.684161 26398 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:54.685372 26254 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:54.685596 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.692400 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.694783 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.695319 26403 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:54.698273 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:54.701092 26221 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:54.702020 26221 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.702346 26326 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:54.704421 32619 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:31:54.704869 32619 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:54.705186 32619 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:54.699007 32664 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:54.708549 32645 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:54.708359 32664 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.709342 26401 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:54.702361 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:54.715476 26295 leader_election.cc:334] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:54.719740 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:31:54.720427 26295 leader_election.cc:334] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.696149 32599 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:31:54.729857 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 49576.7us
I20260924 14:31:54.732725 32664 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.731783 26073 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.736236 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.735857 26073 catalog_manager.cc:5495] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 19. New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.736650 32646 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.737866 26399 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260924 14:31:54.746079 32442 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260924 14:31:54.749461 32662 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:31:54.749770 32330 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:54.750597 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48599.9us
W20260924 14:31:54.777557 26221 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.751852 32594 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:54.755946 26224 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:54.757851 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.737315 26073 catalog_manager.cc:5495] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:54.738996 32646 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:54.779156 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.802110 26256 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:54.785032 32380 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:54.803952 32380 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:54.804858 32380 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:54.789345 26221 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.790123 32619 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:54.812520 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.813305 32594 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:54.813834 32619 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:54.793015 32595 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:54.814487 32595 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:31:54.814726 26221 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.815346 32595 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.816612 32595 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:54.818820 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.819525 26250 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:54.821153 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.821738 26401 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:54.823319 26256 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:54.825682 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:54.843885 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.844589 26255 raft_consensus.cc:2362] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18. Re-sending same reply.
I20260924 14:31:54.829777 26403 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:54.844467 26073 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.845719 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.846107 26073 catalog_manager.cc:5495] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.846988 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.848728 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.848024 26073 catalog_manager.cc:5495] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.850899 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.851213 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.852545 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.852011 26073 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 15, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 15 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.853376 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.854029 32652 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.854732 26073 catalog_manager.cc:5495] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:54.855611 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.857060 26402 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 12
I20260924 14:31:54.857745 26073 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 23, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 23 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.842310 32619 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:54.860805 26251 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:54.870931 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.871344 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.825157 32517 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.862787 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.863402 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.872936 26255 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:54.865185 26401 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 21.
I20260924 14:31:54.867933 26224 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:54.869542 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.871552 26330 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:54.875309 32380 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:54.875823 32380 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 19 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:54.875962 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:54.876791 32380 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.877811 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.883415 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.867199 26073 catalog_manager.cc:5495] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.885627 26332 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 20: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:54.867178 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:54.889137 26330 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:54.885789 26256 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:54.888315 26255 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:54.890631 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.890901 26332 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:54.892736 26297 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:54.892976 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.893143 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.894752 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.895356 26328 raft_consensus.cc:2362] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19. Re-sending same reply.
I20260924 14:31:54.895488 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.895731 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.896195 26256 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:54.900027 32171 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:31:54.900137 32631 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:31:54.900791 32462 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:54.901132 32619 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:54.904114 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.910569 26402 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:54.913348 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.903041 31979 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:54.885300 26073 catalog_manager.cc:5495] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 9, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 9 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.911046 32610 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:54.928387 32610 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:54.912838 32171 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:54.929279 32610 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.929155 26073 catalog_manager.cc:5495] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 15, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.930015 32171 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:54.911482 32631 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.931175 26073 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.931850 32631 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.933063 26073 catalog_manager.cc:5495] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 10, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 10 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.895582 32517 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.935711 32652 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:54.910924 32646 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:54.926832 26327 raft_consensus.cc:2396] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260924 14:31:54.938149 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.939049 26257 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:54.902213 32619 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.942961 26406 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260924 14:31:54.944031 26251 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260924 14:31:54.944396 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.945603 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.945694 32618 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:54.946341 32649 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:54.947079 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.945744 32631 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.952566 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.957974 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:54.958801 32479 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:54.959704 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.956802 26329 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:54.963145 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.963882 26257 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
W20260924 14:31:54.987234 23135 thread.cc:536] Waited for 2000ms trying to join with heartbeat (tid 27223)
I20260924 14:31:54.965276 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.007109 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.031788 26328 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:54.960000 32517 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.012950 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:54.962633 32664 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:31:55.036963 32664 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:55.037388 32664 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:31:55.032212 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 67028.1us
I20260924 14:31:55.034266 26250 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 21.
I20260924 14:31:55.035526 32604 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:55.036226 32646 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:55.050065 32646 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:55.046260 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.934752 26073 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:55.048869 32380 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:54.963866 32599 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:55.078955 32380 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:55.079427 32380 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:55.085873 32440 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:55.086477 32440 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.963640 32594 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
I20260924 14:31:55.051198 32646 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.076819 32604 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:31:55.104249 32479 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.104961 32479 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:55.107131 32479 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.111716 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.112206 32184 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.112947 32184 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:55.115190 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.122694 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.129570 26073 catalog_manager.cc:5495] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 15, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:55.132844 32440 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:55.136029 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.140833 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:55.080255 32380 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.146627 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:31:55.147820 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.151024 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.134832 32649 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:55.137511 32442 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.154214 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:55.156348 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:55.165555 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:55.145669 32517 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:31:55.171545 26295 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.172227 32461 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:55.156543 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.144421 32594 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:55.164992 32458 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:55.156720 32632 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:55.176971 32461 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:55.184499 32458 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:55.192364 32461 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:55.193109 32458 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.182960 32632 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:55.172463 32442 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.200160 32632 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:55.200575 32442 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.178537 32595 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:55.147754 32424 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260924 14:31:55.153044 26221 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.176889 32184 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.175045 32649 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:55.204754 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 51761.4us
W20260924 14:31:55.207325 26221 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.207767 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:55.208940 26221 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:55.209628 26221 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.209941 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.180416 32610 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:55.214080 26221 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.223896 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:55.225528 26221 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.225996 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:55.227056 26221 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.227851 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:55.228842 26221 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:55.229935 26221 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.230463 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:55.219758 32607 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:55.229218 32603 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:55.225198 32594 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:55.243929 32424 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:55.214090 32604 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:55.231066 32558 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:31:55.210953 32649 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:55.261451 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:55.268497 32649 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.274878 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.276309 26257 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:55.281646 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:55.275386 32380 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:55.286655 32618 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:55.288571 32618 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:55.289088 32618 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:55.305819 26406 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:55.307715 32604 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:55.319000 26257 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:55.322315 26399 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:55.352370 32607 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:55.353330 32607 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.358204 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.361224 32607 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.368410 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.341753 32603 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:55.325417 32424 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260924 14:31:55.387624 26401 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:55.341113 32649 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260924 14:31:55.373601 32594 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:55.405479 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 231103us
I20260924 14:31:55.407941 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:55.411211 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 181287us
W20260924 14:31:55.412132 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:55.413621 32640 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:55.414105 32640 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:55.414389 32640 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:55.395310 32603 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:55.417460 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:31:55.422094 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:31:55.422730 26221 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:55.423730 26221 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.424180 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.416030 32594 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:55.425490 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.425940 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.426505 32594 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.426784 32604 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:55.428831 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.429705 26255 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:55.407039 32442 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260924 14:31:55.432215 26399 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260924 14:31:55.433393 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.427205 32594 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:55.439014 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:55.446810 26221 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.447379 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:55.518014 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 71238.3us
W20260924 14:31:55.518863 26221 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.519332 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:55.520622 26221 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.415979 32461 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:55.579483 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 129781us
I20260924 14:31:55.579851 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.580762 26329 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:55.568929 32424 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260924 14:31:55.586117 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 150727us
I20260924 14:31:55.586671 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.600136 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.600481 32603 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.602137 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:55.602627 32664 consensus_meta.cc:287] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.565s	user 0.013s	sys 0.000s
I20260924 14:31:55.602732 26323 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:55.617352 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.622725 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.631001 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.633857 26323 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:55.568621 32685 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:55.652143 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.688263 26332 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260924 14:31:55.688578 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:55.696309 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.637652 32597 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.603359 32664 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:55.637120 32604 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:55.617553 32632 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.641279 32506 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:55.707581 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 83445.6us
I20260924 14:31:55.644290 32649 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:55.698764 32424 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:55.698474 31979 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:55.724721 32693 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.723058 32688 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.698204 32597 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:55.728371 32594 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:55.715112 32604 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.760257 32654 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.779484 32597 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:55.754226 32646 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260924 14:31:55.756937 32693 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:55.790264 32693 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.791630 32664 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.766957 32688 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.794924 32688 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.754866 32506 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:55.808259 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.809128 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:55.809746 26323 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:31:55.797842 32380 consensus_meta.cc:287] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.000s
I20260924 14:31:55.816845 26257 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:55.772771 32618 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:55.826697 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 130496us
I20260924 14:31:55.827968 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.781608 32654 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:55.824811 32632 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.850953 32380 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:55.857017 26255 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260924 14:31:55.861364 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:55.862108 26250 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260924 14:31:55.871730 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 84365.9us
I20260924 14:31:55.868019 26256 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:55.873979 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.876412 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:55.877722 26251 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:55.874912 26249 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
W20260924 14:31:55.875978 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 154818us
I20260924 14:31:55.878875 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.888934 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:55.887415 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.894320 26332 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260924 14:31:55.860543 32685 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260924 14:31:55.909825 26332 pending_rounds.cc:76] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 8
I20260924 14:31:55.917415 26332 pending_rounds.cc:97] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260924 14:31:55.917172 26323 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 21.
I20260924 14:31:55.917786 26332 raft_consensus.cc:2866] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:55.920035 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.920816 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:55.921419 26327 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:55.920876 26399 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:55.921715 26323 raft_consensus.cc:2354] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15.
I20260924 14:31:55.925454 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.927313 26402 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:55.933768 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.934121 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.930423 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.939824 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:55.943073 26256 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:55.880282 32632 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260924 14:31:55.946979 27123 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:55.916605 32604 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.896704 32696 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:55.928689 32688 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:55.954422 26404 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:55.893739 32646 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
I20260924 14:31:55.958781 32708 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:31:55.944062 32631 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:55.947676 32506 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.948218 32707 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.955947 32688 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:55.932446 32664 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260924 14:31:55.970598 32631 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:55.975343 26256 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 21.
I20260924 14:31:56.008317 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:56.024519 23135 thread.cc:536] Waited for 3000ms trying to join with heartbeat (tid 27223)
I20260924 14:31:56.051307 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.121304 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:56.002592 32640 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.588s	user 0.006s	sys 0.006s
W20260924 14:31:56.122833 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 166579us
I20260924 14:31:56.127291 26400 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:31:56.046890 32689 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.236s	user 0.003s	sys 0.000s
W20260924 14:31:56.029790 32699 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260924 14:31:56.150568 32461 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:56.151305 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:56.047317 32618 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.053555 32696 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:56.157390 32696 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.023931 31979 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:56.166304 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:56.167109 26256 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:55.978286 32688 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:55.993253 32654 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:56.004822 32631 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:55.990790 32424 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:56.184140 26256 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:56.123306 32708 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:56.192495 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:56.128283 32683 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:56.195161 26255 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260924 14:31:56.196897 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.123171 32640 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:56.201399 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:31:55.979254 32707 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:56.217495 32641 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:56.218405 32641 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.218883 32707 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.199915 32631 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:56.228034 32631 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:56.229022 32631 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.229763 32717 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260924 14:31:56.231738 32708 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.232519 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.233974 32506 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:56.236734 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:56.202205 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 260259us
I20260924 14:31:56.237874 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.198554 32683 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:56.238575 26403 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:56.197906 32709 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:56.241732 26224 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:56.239066 26256 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:56.246320 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:56.247076 26330 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 18. Current term is 19. Ops: []
I20260924 14:31:56.255236 32703 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:56.255733 32703 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:56.256420 32703 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:56.260576 32716 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.261127 32716 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.267110 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.268178 26403 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:56.272418 26369 leader_election.cc:398] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15.
I20260924 14:31:56.238996 32641 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.243902 32683 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:56.349036 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 75119us
W20260924 14:31:56.349565 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 97116.7us
I20260924 14:31:56.263266 32716 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.248196 32697 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:56.357316 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 82261.5us
I20260924 14:31:56.350931 31979 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:56.244661 32695 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.275694 32707 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:56.379786 32707 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.379673 32695 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.382897 32707 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.368894 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:56.384263 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.385005 26256 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:31:56.386080 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:56.274561 32610 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
W20260924 14:31:56.378376 26295 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:56.392728 32664 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:56.378360 32697 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.385442 32695 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.367522 32595 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.402004 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:56.425705 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47375us
I20260924 14:31:56.378367 31979 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:56.438158 31979 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:56.439333 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:56.440776 32705 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:56.441174 32705 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:56.441506 32705 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:56.428588 32595 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:56.434271 32720 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260924 14:31:56.428992 32695 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:31:56.429531 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:56.270478 32709 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.402807 32707 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:56.454908 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:56.457029 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:56.457289 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 69229.2us
W20260924 14:31:56.463485 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 89692.6us
I20260924 14:31:56.475409 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:56.476143 26327 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:31:56.455385 32695 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:56.461448 32618 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:56.470760 32707 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:56.498962 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.498483 31979 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.482385 32640 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.485253 32424 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:56.494980 32695 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:31:56.552073 32727 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260924 14:31:56.552963 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.553646 26257 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:56.554162 32595 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.495426 32733 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:56.542054 32723 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260924 14:31:56.544888 32732 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:56.523262 32506 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.548408 32709 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.549070 32424 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:56.581574 32732 consensus_queue.cc:1225] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260924 14:31:56.557535 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:56.581741 32506 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:56.582489 32594 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.560760 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:56.526790 32715 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
W20260924 14:31:56.583951 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 83230us
I20260924 14:31:56.590406 26298 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:56.590606 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:56.591523 32461 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:31:56.598536 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.583055 32594 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.589936 32709 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:56.592900 32721 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.592628 32558 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:56.609613 32558 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:56.610100 32697 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:56.607936 32709 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:56.611814 32709 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:56.611872 32558 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:56.613161 32558 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:56.613631 32558 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.616178 32705 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:56.621819 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:56.622450 26328 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:56.638101 26249 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:56.642524 26256 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:31:56.637465 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 133476us
I20260924 14:31:56.647897 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:56.630605 32708 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.648612 26330 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:56.650508 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.651467 26254 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:56.651137 32688 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.650203 32708 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:56.658430 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:56.653153 26402 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:56.653738 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.663079 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.668094 26328 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:56.673501 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:56.685748 26398 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:56.689797 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:56.638482 31979 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.651563 32705 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:56.715277 26221 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:56.719054 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:56.672079 32646 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260924 14:31:56.732823 32558 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:56.734668 32728 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.255s	user 0.000s	sys 0.000s
I20260924 14:31:56.628173 32721 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.739526 32703 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.740701 26257 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:56.741390 32721 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.741781 32703 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.728077 32683 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.750743 32683 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:56.752570 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 256173us
I20260924 14:31:56.755119 32700 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:56.755796 32700 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:56.760646 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.761309 26249 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:56.761489 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:56.762107 26251 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:56.762976 32708 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.763856 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.764443 26329 raft_consensus.cc:2354] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:56.764616 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.769264 26258 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
W20260924 14:31:56.775944 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 93478.5us
I20260924 14:31:56.777068 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:56.777963 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106800us
I20260924 14:31:56.779027 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:56.798609 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 129427us
I20260924 14:31:56.795897 32506 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:56.803598 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.763906 32424 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:56.805119 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.811553 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.813093 26401 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:56.808707 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.737278 32695 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.814670 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:56.819123 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.819394 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.826263 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.843606 27125 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:56.847488 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 165743us
I20260924 14:31:56.849044 26401 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:56.815424 32687 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:56.815411 32746 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:56.793553 32709 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.845618 32597 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:56.827240 32688 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:56.919793 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 71415.6us
I20260924 14:31:56.869244 32594 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.841490 32700 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:56.873373 32479 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:56.959980 26404 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:56.969230 32479 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:56.975071 32458 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Stepping down as leader of term 18
I20260924 14:31:56.975785 32458 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:56.976771 32458 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.978118 32458 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:31:57.046485 23135 thread.cc:536] Waited for 4000ms trying to join with heartbeat (tid 27223)
I20260924 14:31:56.941872 32597 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:56.969755 32479 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:56.937534 32687 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:57.072459 26404 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
W20260924 14:31:57.063645 32720 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260924 14:31:57.067045 32597 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:57.090034 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 217547us
I20260924 14:31:57.112568 32709 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:57.118736 32739 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
W20260924 14:31:57.120994 32750 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260924 14:31:57.129108 26251 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:57.128626 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:57.140936 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 190428us
I20260924 14:31:57.142063 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:57.143348 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 325485us
I20260924 14:31:57.170691 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.172655 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:57.173533 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.175467 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:57.176115 26328 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:57.174396 26403 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:57.179519 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:57.145179 32517 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:57.177907 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 232478us
I20260924 14:31:57.185777 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:57.190778 26325 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:57.182816 26324 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260924 14:31:57.169684 32695 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.199357 26328 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 21.
I20260924 14:31:57.206830 26369 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:57.177927 32709 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.208335 32707 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
W20260924 14:31:57.214466 32742 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260924 14:31:57.215617 32748 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:57.216078 32748 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:57.216310 32748 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:57.217139 32597 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:57.219992 32725 log.cc:924] Time spent T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Append to log took a long time: real 0.618s	user 0.003s	sys 0.004s
I20260924 14:31:57.224124 26325 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
W20260924 14:31:57.227803 32758 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260924 14:31:57.219249 32597 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.238246 32709 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:57.273789 26298 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:57.275120 26369 leader_election.cc:398] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:57.274461 27125 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:57.141242 32687 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.276269 26369 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:57.199095 32517 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:57.293977   302 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:57.295749 32709 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:57.299817 32731 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.301247   300 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.301462 32330 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:57.301779 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62572.3us
W20260924 14:31:57.312856 32766 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260924 14:31:57.313840 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.314455 26254 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:57.316910 32517 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:57.319550 32731 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.320600 32330 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.321455 32697 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:57.321903 32731 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:57.322144 32697 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:57.283608 32707 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:57.305356 26332 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:57.323215 32697 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.292639 32184 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:57.322780 32330 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.235940 32695 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:31:57.288424 32607 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:57.335602 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.308645 32711 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:57.311239 32632 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:57.345125 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.348408 26254 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:57.345974 32607 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:57.351648 32764 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260924 14:31:57.352602 32764 consensus_queue.cc:1225] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260924 14:31:57.353551 32607 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:57.355942 31979 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:57.356582 31979 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.358924 31979 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:57.350464 32632 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:57.360703 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.361083 32632 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.364283 32607 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:57.364917 32607 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.367180 32607 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.368505 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1469) took 1510 ms. Trace:
I20260924 14:31:57.368952 26255 rpcz_store.cc:276] 0924 14:31:55.858216 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:55.858327 (+   111us) service_pool.cc:227] Handling call
0924 14:31:55.858577 (+   250us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:56.197134 (+338557us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:57.368233 (+1171099us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54002a1054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:57.368415 (+   182us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":773141760}
I20260924 14:31:57.368448 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:57.347102 32733 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.760s	user 0.004s	sys 0.009s
I20260924 14:31:57.374910 32733 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.375650 26323 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:57.329244   300 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.377143 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:57.379050 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 208752us
I20260924 14:31:57.382475 26325 raft_consensus.cc:2354] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17.
I20260924 14:31:57.377854 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.384404 26332 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:57.386029 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.386663 26256 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:57.387583 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.389961 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.332407 32184 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.392900 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.393508 32184 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.382654 32746 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:57.394191 32732 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:57.392967 32698 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.395187 32698 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:57.396895 26251 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:57.347394 32695 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 21 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:57.405109 32479 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.413005 32695 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.413887 32479 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.382537 32707 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.410043 32746 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.417770 26256 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:57.421646 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.421679 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.425793 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.426762 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.418357 32746 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.427484 26255 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:57.437537 26255 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 21.
I20260924 14:31:57.441331   300 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:57.441874 26400 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Stepping down as leader of term 14
W20260924 14:31:57.442342 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61898.9us
I20260924 14:31:57.442353 26400 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:57.443562 26400 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.443778 32748 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.446017 32748 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.447204 32698 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:57.449980 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 82111.7us
I20260924 14:31:57.451363 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:57.452792 26400 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:57.452961 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.453677 26332 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:57.455612 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.456903 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:57.457562 26328 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:57.456959 26325 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260924 14:31:57.428175 32632 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.462934 32754 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:57.463434 26254 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260924 14:31:57.465202 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.465824 26249 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:57.463522 32754 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:57.466274 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.462984 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.469563 26250 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260924 14:31:57.453783 26369 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.473819 26255 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260924 14:31:57.476253 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.477094 26251 raft_consensus.cc:2354] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:57.472815 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.478458 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:57.478895 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 133943us
I20260924 14:31:57.479041 26332 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 18
I20260924 14:31:57.479597 26328 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:57.483245 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.481041 32696 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:57.484409 32754 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.492164 32171 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:57.492797 32171 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.493948 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.462468 32632 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.512181 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:57.532466 26249 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:57.486140 32184 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260924 14:31:57.496838 32748 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.537392 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.537557 32748 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:57.535780 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.539817 31979 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:31:57.501325 32719 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:57.541553 32171 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.542591 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.550577 26400 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260924 14:31:57.564230 26371 leader_election.cc:398] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:57.558081 26332 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:57.571621 26400 pending_rounds.cc:76] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 8
I20260924 14:31:57.572302 26371 leader_election.cc:401] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260924 14:31:57.573565 26400 pending_rounds.cc:97] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260924 14:31:57.574283 26400 raft_consensus.cc:2866] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:57.542605 32748 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:57.576005 32691 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:57.587800 27125 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:57.588712 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.570560 32171 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:57.559721 32719 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:57.584285 32747 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260924 14:31:57.554113 32732 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:57.604194 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:57.570293 32761 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260924 14:31:57.612730 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.615183 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 2118) took 1020 ms. Trace:
I20260924 14:31:57.615617 26402 rpcz_store.cc:276] 0924 14:31:56.594404 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:56.594543 (+   139us) service_pool.cc:227] Handling call
0924 14:31:57.615009 (+1020466us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b54001e4bd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:57.615133 (+   124us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":123667840}
W20260924 14:31:57.616559 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52202.5us
I20260924 14:31:57.583606 32631 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260924 14:31:57.591003 32619 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260924 14:31:57.621502 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.609411 32711 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:57.626012 27126 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:57.625353 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 741609us
I20260924 14:31:57.614060 32171 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:57.634253 32711 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:57.636044 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:31:57.634229 26221 leader_election.cc:398] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17.
I20260924 14:31:57.639240 26221 leader_election.cc:401] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260924 14:31:57.639374 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.640443 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:57.641623 32741 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:57.661199 32632 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:57.666795 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.667689 32748 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.641881 32719 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.668967 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.669662 32748 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260924 14:31:57.670831   316 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260924 14:31:57.671522 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1872) took 1115 ms. Trace:
I20260924 14:31:57.671985 26257 rpcz_store.cc:276] 0924 14:31:56.556231 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:56.560418 (+  4187us) service_pool.cc:227] Handling call
0924 14:31:57.671320 (+1110902us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54000c17d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:57.671421 (+   101us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":360561536}
I20260924 14:31:57.609920 32732 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.677845 32732 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.678402 32711 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.721120 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:57.728906 26221 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:57.730415 26221 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.643369 32767 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:57.731240 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 92639.2us
I20260924 14:31:57.654371 32746 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260924 14:31:57.737457 26295 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.698519 32506 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:57.742471 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 106610us
I20260924 14:31:57.732785 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.744016 27126 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.749711   324 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:57.753265 32654 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.753890 32654 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.744887 32682 raft_consensus.cc:2741] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260924 14:31:57.740130 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:57.770174 26221 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.770879 26221 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.738504 32746 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:57.761719   319 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260924 14:31:57.780318 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 302085us
W20260924 14:31:57.772821   320 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.101s	user 0.002s	sys 0.000s
I20260924 14:31:57.772954 32631 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:57.788086 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 44198.8us
I20260924 14:31:57.774840 32698 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.778290 32746 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:57.762686 32506 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.798305 27126 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.779778 32717 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:57.809141 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.810184 32717 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.797586 32755 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.821327 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:57.825644 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52924.9us
I20260924 14:31:57.827507 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:57.828285 26401 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:57.828755 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.829938 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.828219 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.841733 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:57.826521 26221 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.845816 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:57.846807 26221 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.852697 26401 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:31:57.847862 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.804111 32654 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:57.811625 32693 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:57.864162 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.867555 26329 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:57.869383 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:31:57.871824 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47901.5us
I20260924 14:31:57.863348 32717 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.872174 32748 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:57.874589 32698 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:57.901660 26255 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:57.902642 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.911281 32698 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:57.921320   329 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260924 14:31:57.920907 32698 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.910892 32506 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.908840 32646 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.931345 32171 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:57.931959 32171 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.933923 32171 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.934392 32746 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.938787 26252 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:57.939813 26332 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260924 14:31:57.944628 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.946553 32746 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.956492 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:57.957818 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61580.7us
I20260924 14:31:57.958241 32741 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17.
W20260924 14:31:57.962248 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.969188 26295 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.904793 32693 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:57.970161 32693 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.972671 32693 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:57.921455   323 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260924 14:31:57.975181 32687 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:57.976990 26295 leader_election.cc:334] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.978577 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.980052 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.943185 32740 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260924 14:31:57.993261 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:57.993942 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:57.994442 26369 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.994951 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.994804 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.996351 32752 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:57.997016 26369 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.996862 32752 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.997427 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.915143 32682 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260924 14:31:57.998813 32752 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:58.000802 32740 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:57.962870   305 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:58.004793 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:58.002787 27126 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:58.003463 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.922860 32747 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260924 14:31:58.019893 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:57.980289 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:58.028620 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.999331 32752 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.992640 32687 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:58.036118 26295 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:58.036422 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 123898us
W20260924 14:31:58.037062 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 56793.2us
W20260924 14:31:58.037721 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:58.040977 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:58.043246 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:58.047199 23135 thread.cc:536] Waited for 5000ms trying to join with heartbeat (tid 27223)
I20260924 14:31:58.048627 32682 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:58.051879 32171 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:58.053519 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:58.054428 32184 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:58.058434 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:58.063279 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260924 14:31:58.072009 26221 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:58.035646 32696 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:58.076128 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:58.075040 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:58.085865 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:58.031039   305 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:58.072965 32754 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260924 14:31:58.097855 26250 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260924 14:31:58.098285 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:58.080767 32748 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:58.080039 32685 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:31:58.103261 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:58.141846 26329 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:58.145285 26255 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:58.151635 32597 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260924 14:31:58.151619 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48457.3us
I20260924 14:31:58.160321 26397 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:58.161844 32747 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260924 14:31:58.162453 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57949.7us
I20260924 14:31:58.192199 32618 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260924 14:31:58.184682 32682 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:58.169305 32663 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260924 14:31:58.171362 32654 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260924 14:31:58.220453   335 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260924 14:31:58.231362 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:58.207031 32687 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260924 14:31:58.260273 26252 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260924 14:31:58.255661 32712 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:58.256820 26400 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:58.208801 32682 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:58.265823 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 119357us
I20260924 14:31:58.270861 32707 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:58.272882 32707 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:58.273425 32707 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:58.276113 26252 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:58.280112 32632 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:58.281152   324 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:58.281685   324 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:58.283715 26221 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:58.285285 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:58.286134 32703 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:58.293195 32682 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:58.264961   305 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:58.296169 32610 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:58.296705 32610 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:58.298799 32610 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:58.262849 32712 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:58.308843 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:58.309685 26253 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:58.311690 26249 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:58.313211 26253 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260924 14:31:58.314131 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:58.315884 32440 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:58.316638 32440 consensus_queue.cc:1225] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 6c22052275034c369b7f7dca086fbd1e log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260924 14:31:58.319626 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:58.320122 26329 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260924 14:31:58.320370 26323 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:31:58.323530 26397 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:58.335837 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:58.337397 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:58.341630 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:58.342119 26295 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 131 similar messages]
I20260924 14:31:58.342644 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:58.303346 32707 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:58.343286 26330 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 19: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:58.334924 32654 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:58.355556 26403 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:31:58.348780 26327 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:58.349020 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:58.349447 32693 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:58.353488 26323 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260924 14:31:58.364943 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:58.302259 32712 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:58.343223   340 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260924 14:31:58.338348   324 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:58.361038 32654 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:58.385577 26295 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:58.391435 32711 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:58.393885   325 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:58.396685 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:58.398183 32710 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:58.398766 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:58.391830   336 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260924 14:31:58.384944 32744 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:58.398964 32506 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:58.363559 32687 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260924 14:31:58.391197 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:58.370347 32700 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260924 14:31:58.388535 32654 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
W20260924 14:31:58.428725 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:31:58.416671 32712 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:58.411759 32744 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:58.420873 32767 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:58.427174   309 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260924 14:31:58.438265   341 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260924 14:31:58.444684   345 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260924 14:31:58.512550 32744 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:58.595911 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 81614.8us
I20260924 14:31:58.611356   303 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:58.612047   303 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:58.662441 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:58.677987 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 75349.8us
W20260924 14:31:58.682637 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 113687us
I20260924 14:31:58.691098 26327 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:58.719318 32744 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:58.709913   303 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:58.758010   349 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260924 14:31:58.719784   310 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260924 14:31:58.760238   353 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260924 14:31:58.787217 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 86007us
W20260924 14:31:58.818954 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:58.823110   361 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260924 14:31:58.849084 32654 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:58.909271 32654 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:58.910293 26404 raft_consensus.cc:2354] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 19. Current term is 20.
I20260924 14:31:58.929777   305 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:58.930601   305 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:58.936978   305 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:59.005045 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:59.008653 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:59.013471 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:59.026163 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 80543us
I20260924 14:31:59.033082 26224 leader_election.cc:398] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 19. Current term is 20.
I20260924 14:31:59.034122 26224 leader_election.cc:401] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:59.034785 32700 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 20
I20260924 14:31:59.027875 26324 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260924 14:31:59.040169 26324 pending_rounds.cc:76] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 10
I20260924 14:31:59.040534 26324 pending_rounds.cc:97] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260924 14:31:59.040885 26324 raft_consensus.cc:2866] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:59.057325 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:31:59.058759 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.060322 23135 raft_consensus.cc:2229] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.060983 23135 pending_rounds.cc:62] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Trying to abort 1 pending ops.
I20260924 14:31:59.061337 23135 pending_rounds.cc:68] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Aborting op as it isn't in flight: id { term: 16 index: 10 } timestamp: 7332906214000791552 op_type: NO_OP noop_request { }
I20260924 14:31:59.061930 23135 raft_consensus.cc:2866] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260924 14:31:59.062400 23135 raft_consensus.cc:2258] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.077708   363 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:59.079202 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:59.080297 26324 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
W20260924 14:31:59.083976 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 51032.9us
I20260924 14:31:59.092435 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.093722 23135 raft_consensus.cc:2229] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.094515 23135 raft_consensus.cc:2258] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.097748 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.098542 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:59.099118 23135 raft_consensus.cc:2229] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.101601 23135 raft_consensus.cc:2258] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260924 14:31:59.104847 26073 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55200 (request call id 131) took 15081 ms (client timeout 15000 ms). Trace:
W20260924 14:31:59.105458 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 44177.3us
W20260924 14:31:59.105459 26073 rpcz_store.cc:269] 0924 14:31:44.023865 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:44.027477 (+  3612us) service_pool.cc:227] Handling call
0924 14:31:59.104690 (+15077213us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:55.932325 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:55.999158 (+ 66833us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:55.999219 (+    61us) write_op.cc:435] Acquired schema lock
0924 14:31:55.999240 (+    21us) tablet.cc:660] Decoding operations
0924 14:31:56.001353 (+  2113us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:56.001460 (+   107us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:56.001506 (+    46us) tablet.cc:683] Acquiring locks for 28 operations
0924 14:31:56.203990 (+202484us) tablet.cc:699] Row locks acquired
0924 14:31:56.224377 (+ 20387us) write_op.cc:260] PREPARE: finished
0924 14:31:56.224623 (+   246us) write_op.cc:270] Start()
0924 14:31:56.224841 (+   218us) write_op.cc:276] Timestamp: P: 1790260316224533 usec, L: 0
0924 14:31:56.224890 (+    49us) op_driver.cc:331] REPLICATION: starting
0924 14:31:56.225875 (+   985us) log.cc:842] Serialized 11093 byte log entry
0924 14:31:56.984726 (+758851us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:57.352025 (+367299us) op_driver.cc:444] REPLICATION: finished
0924 14:31:57.536091 (+184066us) write_op.cc:301] APPLY: starting
0924 14:31:57.536196 (+   105us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:57.977848 (+441652us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:57.977893 (+    45us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:58.477004 (+499111us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:58.477410 (+   406us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=56,mrs_lookups=0
0924 14:31:58.477472 (+    62us) write_op.cc:312] APPLY: finished
0924 14:31:58.478024 (+   552us) log.cc:842] Serialized 243 byte log entry
0924 14:31:58.611287 (+133263us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:58.648480 (+ 37193us) write_op.cc:454] Released schema lock
0924 14:31:58.655269 (+  6789us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:58.683945 (+ 28676us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:58.693445 (+  9500us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":452805,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":183749,"apply.run_cpu_time_us":53728,"apply.run_wall_time_us":1175821,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121279,"num_ops":28,"prepare.queue_time_us":256950,"prepare.run_cpu_time_us":28738,"prepare.run_wall_time_us":388354,"raft.queue_time_us":130460,"raft.run_cpu_time_us":10535,"raft.run_wall_time_us":72648,"replication_time_us":1127016,"spinlock_wait_cycles":19827072,"thread_start_us":558012,"threads_started":5,"wal-append.queue_time_us":389966,"wal-append.run_cpu_time_us":44515,"wal-append.run_wall_time_us":1905397}]]}
I20260924 14:31:59.141741 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:59.160333 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:31:59.170464 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:59.178865 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.179749   303 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:59.184322 26324 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:31:59.162567   305 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:59.208603 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:59.210186   364 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:31:59.210871   364 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:59.211993   364 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:59.209185 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:59.220263 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:59.220896 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:59.215210 32700 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 19. Current term is 20.
I20260924 14:31:59.192691 23135 raft_consensus.cc:2229] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.232964 26252 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
W20260924 14:31:59.233848 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:59.242806 23135 raft_consensus.cc:2258] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.227360 26074 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 11 to 13, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.253269 26074 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.266245 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.267284 23135 raft_consensus.cc:2229] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.268407 23135 raft_consensus.cc:2258] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.269011 26074 catalog_manager.cc:5495] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.270778 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.273871 23135 raft_consensus.cc:2229] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.275511 23135 raft_consensus.cc:2258] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.273383 26074 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.278385 26074 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 9 to 11, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.281302 26074 catalog_manager.cc:5495] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 4 to 10, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 10 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.284955 26074 catalog_manager.cc:5495] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 8 to 10, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 10 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.287211 26074 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
W20260924 14:31:59.299021 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:59.303153 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.317054 23135 raft_consensus.cc:2229] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.318645 23135 raft_consensus.cc:2258] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.324316 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.325362 23135 raft_consensus.cc:2229] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.326591 23135 raft_consensus.cc:2258] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.330325 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.331271 23135 raft_consensus.cc:2229] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.332288 23135 raft_consensus.cc:2258] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.337823 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.338840 23135 raft_consensus.cc:2229] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.340344 23135 raft_consensus.cc:2258] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.346447   310 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:59.347251   310 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:59.350274 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:59.351539 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:59.353084 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:59.353971   310 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:59.357777 32479 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:59.360085 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.361668 23135 raft_consensus.cc:2229] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.362533 23135 raft_consensus.cc:2258] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.376912 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.378788 23135 raft_consensus.cc:2229] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.393222 23135 raft_consensus.cc:2258] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260924 14:31:59.412871 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:59.421988 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.435914 23135 raft_consensus.cc:2229] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.440164 23135 raft_consensus.cc:2258] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.447286 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.454746 23135 raft_consensus.cc:2229] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.456396 23135 raft_consensus.cc:2258] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.473380 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.474622 23135 raft_consensus.cc:2229] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.481525 23135 raft_consensus.cc:2258] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.506628 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.507944 23135 raft_consensus.cc:2229] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.513573 23135 raft_consensus.cc:2258] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.516485 23135 tablet_replica.cc:330] stopping tablet replica
W20260924 14:31:59.518661 26074 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 142) took 38805 ms (client timeout 15000 ms). Trace:
I20260924 14:31:59.519668 23135 raft_consensus.cc:2229] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.520332 23135 raft_consensus.cc:2258] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.530756 26078 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 8 to 11, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:59.532796 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.533999 23135 raft_consensus.cc:2229] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.533537 26078 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 10 to 11, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 11 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.535332 23135 raft_consensus.cc:2258] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.537791 26078 catalog_manager.cc:5495] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 8 to 9, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 9 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.547060 26078 catalog_manager.cc:5495] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.551126 26078 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.558844 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.560034 23135 raft_consensus.cc:2229] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.560788 23135 raft_consensus.cc:2258] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.563995 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.565464 23135 raft_consensus.cc:2229] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.567386 23135 raft_consensus.cc:2258] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.571818 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.600965   300 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:59.601730   300 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:59.611683 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:59.612727 23135 raft_consensus.cc:2229] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.613169 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:59.614137 26255 raft_consensus.cc:2379] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:59.615674 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:59.616555 32641 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:59.619837   300 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:59.625650 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:59.519168 26074 rpcz_store.cc:269] 0924 14:31:20.713395 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:20.713700 (+   305us) service_pool.cc:227] Handling call
0924 14:31:59.518576 (+38804876us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:59.332221 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:59.332715 (+   494us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:59.332744 (+    29us) write_op.cc:435] Acquired schema lock
0924 14:31:59.332759 (+    15us) tablet.cc:660] Decoding operations
0924 14:31:59.333636 (+   877us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:59.333720 (+    84us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:59.333746 (+    26us) tablet.cc:683] Acquiring locks for 8 operations
0924 14:31:59.334403 (+   657us) tablet.cc:699] Row locks acquired
0924 14:31:59.334425 (+    22us) write_op.cc:260] PREPARE: finished
0924 14:31:59.334581 (+   156us) write_op.cc:270] Start()
0924 14:31:59.334696 (+   115us) write_op.cc:276] Timestamp: P: 1790260319334536 usec, L: 0
0924 14:31:59.334718 (+    22us) op_driver.cc:331] REPLICATION: starting
0924 14:31:59.359608 (+ 24890us) log.cc:842] Serialized 3281 byte log entry
0924 14:31:59.375038 (+ 15430us) op_driver.cc:444] REPLICATION: finished
0924 14:31:59.413512 (+ 38474us) write_op.cc:301] APPLY: starting
0924 14:31:59.413607 (+    95us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:59.418385 (+  4778us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:59.418496 (+   111us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:59.423216 (+  4720us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:59.423487 (+   271us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0924 14:31:59.423541 (+    54us) write_op.cc:312] APPLY: finished
0924 14:31:59.423813 (+   272us) log.cc:842] Serialized 80 byte log entry
0924 14:31:59.424600 (+   787us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:59.424756 (+   156us) write_op.cc:454] Released schema lock
0924 14:31:59.424808 (+    52us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:59.425089 (+   281us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:59.425119 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":64253,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":38171,"apply.run_cpu_time_us":11184,"apply.run_wall_time_us":12430,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37599,"num_ops":8,"prepare.queue_time_us":38676,"prepare.run_cpu_time_us":3822,"prepare.run_wall_time_us":31800,"replication_time_us":40208,"thread_start_us":59963,"threads_started":2}]]}
I20260924 14:31:59.614161 23135 raft_consensus.cc:2258] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.630756 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.631718 23135 raft_consensus.cc:2229] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.644155 32641 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:59.644996 32479 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:59.645663 32479 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:59.649425 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:59.652915 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:59.654452 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:59.655315 32705 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:59.656903 32479 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:59.663971 26249 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260924 14:31:59.665537   364 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:59.674373 23135 raft_consensus.cc:2258] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.675689 26073 catalog_manager.cc:5495] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 21, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 21 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.677354 26073 catalog_manager.cc:5495] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:59.678861 26073 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 18, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 18 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
W20260924 14:31:59.682614 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260924 14:31:59.682251 26073 catalog_manager.cc:5495] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 22, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 22 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:59.684309 26073 catalog_manager.cc:5495] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 6 to 19, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:31:59.675190 26078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 8) took 41216 ms (client timeout 15000 ms). Trace:
I20260924 14:31:59.686568 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.685400 26073 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.688144 26073 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 15, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.689040 26073 catalog_manager.cc:5495] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.690066 26073 catalog_manager.cc:5495] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 19, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:59.687502 23135 raft_consensus.cc:2229] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.691159 26073 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 16 to 17, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 17 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
W20260924 14:31:59.686269 26078 rpcz_store.cc:269] 0924 14:31:18.458565 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.463681 (+  5116us) service_pool.cc:227] Handling call
0924 14:31:59.675104 (+41211423us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:59.555881 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:59.556402 (+   521us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:59.556442 (+    40us) write_op.cc:435] Acquired schema lock
0924 14:31:59.556459 (+    17us) tablet.cc:660] Decoding operations
0924 14:31:59.557170 (+   711us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:59.557228 (+    58us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:59.557256 (+    28us) tablet.cc:683] Acquiring locks for 5 operations
0924 14:31:59.557840 (+   584us) tablet.cc:699] Row locks acquired
0924 14:31:59.557860 (+    20us) write_op.cc:260] PREPARE: finished
0924 14:31:59.558026 (+   166us) write_op.cc:270] Start()
0924 14:31:59.558143 (+   117us) write_op.cc:276] Timestamp: P: 1790260319557976 usec, L: 0
0924 14:31:59.558165 (+    22us) op_driver.cc:331] REPLICATION: starting
0924 14:31:59.559045 (+   880us) log.cc:842] Serialized 2108 byte log entry
0924 14:31:59.562262 (+  3217us) op_driver.cc:444] REPLICATION: finished
0924 14:31:59.563516 (+  1254us) write_op.cc:301] APPLY: starting
0924 14:31:59.563592 (+    76us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:59.566222 (+  2630us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:59.566318 (+    96us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:59.571734 (+  5416us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:59.571918 (+   184us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0924 14:31:59.571964 (+    46us) write_op.cc:312] APPLY: finished
0924 14:31:59.572182 (+   218us) log.cc:842] Serialized 56 byte log entry
0924 14:31:59.572346 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:59.572452 (+   106us) write_op.cc:454] Released schema lock
0924 14:31:59.572495 (+    43us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:59.572776 (+   281us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:59.572805 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":634269,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":1005,"apply.run_cpu_time_us":7238,"apply.run_wall_time_us":9599,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20920,"num_ops":5,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":3385,"prepare.run_wall_time_us":3873,"replication_time_us":4003}]]}
I20260924 14:31:59.696018 26073 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 18, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 18 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.699728 23135 raft_consensus.cc:2258] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.699499 26073 catalog_manager.cc:5495] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 16 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 18 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:59.701205 26073 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 19, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.702493 26073 catalog_manager.cc:5495] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 17, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 17 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.703709 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.703881 26073 catalog_manager.cc:5495] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:59.704694 23135 raft_consensus.cc:2229] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.705698 26073 catalog_manager.cc:5495] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 15, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.706549 23135 raft_consensus.cc:2258] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.707319 26073 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.708638 26073 catalog_manager.cc:5495] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 18, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 18 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.709764 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.709913 26073 catalog_manager.cc:5495] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.710749 23135 raft_consensus.cc:2229] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.711148 26073 catalog_manager.cc:5495] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:59.712411 26073 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.713518 26073 catalog_manager.cc:5495] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 16 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.714707 26073 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.715991 26073 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 18, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 18 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.717268 26073 catalog_manager.cc:5495] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 17, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 17 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.718726 26073 catalog_manager.cc:5495] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 16, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 16 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.720268 26073 catalog_manager.cc:5495] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 18 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:59.721501 26073 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 23 to 24, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 24 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:59.722718 26073 catalog_manager.cc:5495] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 15, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 15 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.724009 26073 catalog_manager.cc:5495] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.725561 26073 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 16 to 19, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 19 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.727075 23135 raft_consensus.cc:2258] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.727152 26073 catalog_manager.cc:5495] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 17. New cstate: current_term: 17 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.738744 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.736095 26073 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 16 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:59.740633 26073 catalog_manager.cc:5495] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.740072 23135 raft_consensus.cc:2229] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 24 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.745393 23135 raft_consensus.cc:2258] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.751292 32618 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:59.756558 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.757580 23135 raft_consensus.cc:2229] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.761626 23135 raft_consensus.cc:2258] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.757104 32618 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:59.777320 32618 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:59.778070 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:59.778673 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:59.778860 26324 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:59.779394 26255 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:59.780254 27123 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:59.781261 32705 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:59.781652 32705 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:59.781993 32705 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:59.789549 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.790551 23135 raft_consensus.cc:2229] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.792140 23135 raft_consensus.cc:2258] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.798003 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.798908 23135 raft_consensus.cc:2229] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.799978 23135 raft_consensus.cc:2258] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.802495 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.803280 23135 raft_consensus.cc:2229] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.804292 23135 raft_consensus.cc:2258] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.806829 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.807421 23135 raft_consensus.cc:2229] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.808455 23135 raft_consensus.cc:2258] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.809237 26324 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260924 14:31:59.811113   364 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:59.811832   364 consensus_queue.cc:1225] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260924 14:31:59.814563 26324 pending_rounds.cc:76] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 10
I20260924 14:31:59.809711 32705 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:59.814895 26324 pending_rounds.cc:97] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260924 14:31:59.815196 26324 raft_consensus.cc:2866] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:59.817230 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:59.817929 26255 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:59.826028 23135 tablet_replica.cc:330] stopping tablet replica
W20260924 14:31:59.826566 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260924 14:31:59.829109 23135 raft_consensus.cc:2229] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.835069 26255 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:59.837249 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:59.835090 23135 raft_consensus.cc:2258] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.838701 32618 raft_consensus.cc:2684] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Received election callback for term 17 while not running: Illegal state: RaftConsensus is not running: State = Stopped
I20260924 14:31:59.849648 32705 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:59.850653 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:59.851356 26330 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:59.852945 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.853885 23135 raft_consensus.cc:2229] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.855054 23135 raft_consensus.cc:2258] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.860517 26330 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:59.871650 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.889585 23135 raft_consensus.cc:2229] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.892211 23135 raft_consensus.cc:2258] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.897037 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.897876 23135 raft_consensus.cc:2229] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.898499 23135 raft_consensus.cc:2258] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.921303 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.922066 23135 raft_consensus.cc:2229] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.922658 23135 raft_consensus.cc:2258] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.936365 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.937196 23135 raft_consensus.cc:2229] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.952353 23135 raft_consensus.cc:2258] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.967664 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.968497 23135 raft_consensus.cc:2229] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.969058 23135 raft_consensus.cc:2258] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.993840 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.004220 23135 raft_consensus.cc:2229] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:00.005406 23135 raft_consensus.cc:2258] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.009266 26079 catalog_manager.cc:5495] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.013952 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.014938 23135 raft_consensus.cc:2229] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:00.023595 23135 raft_consensus.cc:2258] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.039083 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.040066 23135 raft_consensus.cc:2229] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:00.041229 23135 raft_consensus.cc:2258] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.044410 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.045253 23135 raft_consensus.cc:2229] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:00.045846 23135 raft_consensus.cc:2258] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.064944 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.064328 26073 catalog_manager.cc:5495] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 20, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 20 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:32:00.065811 23135 raft_consensus.cc:2229] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:32:00.067098 23135 raft_consensus.cc:2258] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.078900 26077 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 13 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 15 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:00.078959 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.092299 23135 raft_consensus.cc:2229] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:00.092974 23135 raft_consensus.cc:2258] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.093605 26077 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 15 to 17, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 17 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.095130 26077 catalog_manager.cc:5495] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.097079 26077 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 6 to 11, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 11 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:00.099123 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.099840 23135 raft_consensus.cc:2229] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Raft consensus shutting down.
I20260924 14:32:00.101162 23135 raft_consensus.cc:2258] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:00.102612 26079 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 141) took 67038 ms (client timeout 15000 ms). Trace:
W20260924 14:32:00.103137 26079 rpcz_store.cc:269] 0924 14:30:53.063866 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.064054 (+   188us) service_pool.cc:227] Handling call
0924 14:32:00.102536 (+67038482us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:32:00.027722 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:32:00.028216 (+   494us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:32:00.028263 (+    47us) write_op.cc:435] Acquired schema lock
0924 14:32:00.028284 (+    21us) tablet.cc:660] Decoding operations
0924 14:32:00.028630 (+   346us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:32:00.028681 (+    51us) write_op.cc:641] Partition lock acquired for write op
0924 14:32:00.028705 (+    24us) tablet.cc:683] Acquiring locks for 1 operations
0924 14:32:00.028886 (+   181us) tablet.cc:699] Row locks acquired
0924 14:32:00.028906 (+    20us) write_op.cc:260] PREPARE: finished
0924 14:32:00.029043 (+   137us) write_op.cc:270] Start()
0924 14:32:00.029153 (+   110us) write_op.cc:276] Timestamp: P: 1790260320029002 usec, L: 0
0924 14:32:00.029171 (+    18us) op_driver.cc:331] REPLICATION: starting
0924 14:32:00.029756 (+   585us) log.cc:842] Serialized 543 byte log entry
0924 14:32:00.032215 (+  2459us) op_driver.cc:444] REPLICATION: finished
0924 14:32:00.032709 (+   494us) write_op.cc:301] APPLY: starting
0924 14:32:00.032797 (+    88us) tablet.cc:1365] starting BulkCheckPresence
0924 14:32:00.033660 (+   863us) tablet.cc:1368] finished BulkCheckPresence
0924 14:32:00.033767 (+   107us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:32:00.034483 (+   716us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:32:00.034652 (+   169us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0924 14:32:00.034696 (+    44us) write_op.cc:312] APPLY: finished
0924 14:32:00.034887 (+   191us) log.cc:842] Serialized 24 byte log entry
0924 14:32:00.035034 (+   147us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.035116 (+    82us) write_op.cc:454] Released schema lock
0924 14:32:00.035153 (+    37us) write_op.cc:341] FINISH: Updating metrics
0924 14:32:00.036578 (+  1425us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.036609 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2567379,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":264,"apply.run_cpu_time_us":3055,"apply.run_wall_time_us":4132,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":1432,"prepare.run_cpu_time_us":2318,"prepare.run_wall_time_us":2318,"replication_time_us":2950}]]}
I20260924 14:32:00.103688 26077 catalog_manager.cc:5495] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.122366 26077 catalog_manager.cc:5495] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 11 to 15, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 15 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:00.141039 26077 catalog_manager.cc:5495] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:32:00.151891 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:32:00.151197 26077 catalog_manager.cc:5495] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 10 to 14, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.155625 26077 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.157032 26077 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 10 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 15 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
W20260924 14:32:00.200532 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:32:00.309703   385 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.310503   385 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.317582   385 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:00.319186 26221 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:32:00.325271 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.328053 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.328984 32700 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:00.336370   385 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.337077   385 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.341188   385 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:00.347587 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:32:00.354223 26221 leader_election.cc:334] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:32:00.354833 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:00.357515   385 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:00.372154   364 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.372917   364 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.376665 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:32:00.382437 26369 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:32:00.383553   364 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.382963 26369 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.384934   364 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:32:00.386842 26076 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 14 to 18, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 18 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
W20260924 14:32:00.402971 26077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 143) took 13443 ms (client timeout 15000 ms). Trace:
W20260924 14:32:00.406057 26077 rpcz_store.cc:269] 0924 14:31:46.959951 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:46.969425 (+  9474us) service_pool.cc:227] Handling call
0924 14:31:47.550136 (+580711us) spinlock_profiling.cc:243] Waited 89.3 ms on lock 0x7b540015fed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3af0 00007f09b9356f2a 00007f09bf2a8d6c 00007f09bf3a2c3e 00007f09bd6b4fb4 00007f09bd6b4f42 00007f09bd6b4e71 00007f09bd6b4dec 00007f09bd6b40a2 00007f09b777894c 00007f09b7777d86
0924 14:32:00.402887 (+12852751us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:32:00.187293 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:32:00.187802 (+   509us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:32:00.187837 (+    35us) write_op.cc:435] Acquired schema lock
0924 14:32:00.187856 (+    19us) tablet.cc:660] Decoding operations
0924 14:32:00.188884 (+  1028us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:32:00.188951 (+    67us) write_op.cc:641] Partition lock acquired for write op
0924 14:32:00.188986 (+    35us) tablet.cc:683] Acquiring locks for 10 operations
0924 14:32:00.189783 (+   797us) tablet.cc:699] Row locks acquired
0924 14:32:00.189800 (+    17us) write_op.cc:260] PREPARE: finished
0924 14:32:00.189946 (+   146us) write_op.cc:270] Start()
0924 14:32:00.190059 (+   113us) write_op.cc:276] Timestamp: P: 1790260320189900 usec, L: 0
0924 14:32:00.190081 (+    22us) op_driver.cc:331] REPLICATION: starting
0924 14:32:00.190698 (+   617us) log.cc:842] Serialized 4063 byte log entry
0924 14:32:00.194731 (+  4033us) op_driver.cc:444] REPLICATION: finished
0924 14:32:00.197042 (+  2311us) write_op.cc:301] APPLY: starting
0924 14:32:00.197131 (+    89us) tablet.cc:1365] starting BulkCheckPresence
0924 14:32:00.204681 (+  7550us) tablet.cc:1368] finished BulkCheckPresence
0924 14:32:00.204823 (+   142us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:32:00.220577 (+ 15754us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:32:00.220802 (+   225us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0924 14:32:00.220846 (+    44us) write_op.cc:312] APPLY: finished
0924 14:32:00.221163 (+   317us) log.cc:842] Serialized 96 byte log entry
0924 14:32:00.296601 (+ 75438us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.311607 (+ 15006us) write_op.cc:454] Released schema lock
0924 14:32:00.311654 (+    47us) write_op.cc:341] FINISH: Updating metrics
0924 14:32:00.311928 (+   274us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.311961 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21244,"reported_tablets":44,"spinlock_wait_cycles":205277952,"child_traces":[["op",{"apply.queue_time_us":2087,"apply.run_cpu_time_us":14111,"apply.run_wall_time_us":115225,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41840,"num_ops":10,"prepare.queue_time_us":104,"prepare.run_cpu_time_us":3695,"prepare.run_wall_time_us":15888,"replication_time_us":4549}]]}
W20260924 14:32:00.473238 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:00.563427   385 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.594177   385 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.606133   385 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.607424 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.616017   364 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.617326   364 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.621049   300 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.621464   386 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.622458   300 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.622754   386 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.625401   364 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.625420   385 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.626516   385 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.630779   385 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.632103   300 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.632818 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.634330 26400 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:32:00.635586 26224 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:00.636597   385 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:32:00.637058   385 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.637511   385 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:32:00.640125 26295 leader_election.cc:334] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:00.640949 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.636626   386 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:00.643726 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:00.644438 26369 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.648967   390 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.650087   364 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.650034   390 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.650906   364 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.652971   390 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:00.653414   364 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.660694   364 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.661514   364 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:00.668143 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.678534   364 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.679764   394 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.680645   394 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.681453   385 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.684497 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:32:00.686323 26369 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.686671   385 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.687983 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:00.688789 26400 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:32:00.690411   394 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:00.697921 26295 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.698781 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.707645   392 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.708180 32610 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.708663   392 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.709192 32610 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.712410   392 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.712989 32610 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.718570   386 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.721904   386 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:00.724280 26369 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:00.725023 26369 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.725266   386 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.725786   300 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.726611   300 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.727627 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.728366 26406 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:32:00.729044   300 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.730083 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:00.730769 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.731289   394 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:00.731674 26324 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:32:00.732543   394 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260924 14:32:00.733547 26076 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 143) took 18375 ms (client timeout 15000 ms). Trace:
I20260924 14:32:00.733734   300 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.733191   394 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.736112   394 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:32:00.734776   300 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.741004   385 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.741916   385 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.752948   385 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:00.754818   394 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.756402 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:00.742974 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45103.5us
I20260924 14:32:00.757704 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:00.757973   355 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:32:00.758466   355 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.758823   355 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:32:00.755635   300 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.762903   394 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.764436 32641 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.765007   364 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.765326 32641 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.765838   364 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.768194 32641 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.770341   364 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:00.774118 26369 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.775144 26075 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.780128 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.780977 26255 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:32:00.782169 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.782898 26256 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:32:00.784761 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:32:00.785552 26080 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 7) took 71569 ms (client timeout 15000 ms). Trace:
I20260924 14:32:00.771183   355 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:00.786020 26080 rpcz_store.cc:269] 0924 14:30:49.215997 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:49.216201 (+   204us) service_pool.cc:227] Handling call
0924 14:32:00.785300 (+71569099us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":50133,"reported_tablets":45}
W20260924 14:32:00.787794 26067 connection.cc:441] server connection from 127.0.0.1:34948 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 7) could send its response
I20260924 14:32:00.789819   355 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.793597 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.794430 26253 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
W20260924 14:32:00.795403 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.795975 26075 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.796615 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.797523 26324 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:32:00.797729 26075 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 15 to 16, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 16 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:00.799065 26406 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:32:00.795356 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.800446 26255 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:32:00.802129 26075 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 13, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
W20260924 14:32:00.808756 26295 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:00.809875 26295 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:00.810510 26295 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:00.811038 26295 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.811473 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.812335   394 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260924 14:32:00.734316 26076 rpcz_store.cc:269] 0924 14:31:42.312225 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:42.404867 (+ 92642us) service_pool.cc:227] Handling call
0924 14:32:00.687351 (+18282484us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:32:00.416961 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:32:00.423072 (+  6111us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:32:00.423474 (+   402us) write_op.cc:435] Acquired schema lock
0924 14:32:00.423870 (+   396us) tablet.cc:660] Decoding operations
0924 14:32:00.424765 (+   895us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:32:00.424823 (+    58us) write_op.cc:641] Partition lock acquired for write op
0924 14:32:00.424848 (+    25us) tablet.cc:683] Acquiring locks for 1 operations
0924 14:32:00.425045 (+   197us) tablet.cc:699] Row locks acquired
0924 14:32:00.425059 (+    14us) write_op.cc:260] PREPARE: finished
0924 14:32:00.425206 (+   147us) write_op.cc:270] Start()
0924 14:32:00.425318 (+   112us) write_op.cc:276] Timestamp: P: 1790260320425162 usec, L: 0
0924 14:32:00.425334 (+    16us) op_driver.cc:331] REPLICATION: starting
0924 14:32:00.425935 (+   601us) log.cc:842] Serialized 543 byte log entry
0924 14:32:00.427609 (+  1674us) op_driver.cc:444] REPLICATION: finished
0924 14:32:00.427933 (+   324us) write_op.cc:301] APPLY: starting
0924 14:32:00.428011 (+    78us) tablet.cc:1365] starting BulkCheckPresence
0924 14:32:00.429257 (+  1246us) tablet.cc:1368] finished BulkCheckPresence
0924 14:32:00.429353 (+    96us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:32:00.430003 (+   650us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:32:00.430153 (+   150us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0924 14:32:00.430190 (+    37us) write_op.cc:312] APPLY: finished
0924 14:32:00.430361 (+   171us) log.cc:842] Serialized 24 byte log entry
0924 14:32:00.430499 (+   138us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.430578 (+    79us) write_op.cc:454] Released schema lock
0924 14:32:00.430613 (+    35us) write_op.cc:341] FINISH: Updating metrics
0924 14:32:00.430847 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.430871 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":417490,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":100,"apply.run_cpu_time_us":2663,"apply.run_wall_time_us":3139,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":1985,"prepare.run_cpu_time_us":2753,"prepare.run_wall_time_us":9841,"replication_time_us":2159}]]}
I20260924 14:32:00.816911   355 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.817086   364 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:00.817512 26295 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.817770   355 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.817891   364 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.819798   394 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.820492   364 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.820495   394 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.823416   394 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.824404 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.825204 26401 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:32:00.826431 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:32:00.826858 26369 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.827484   394 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:32:00.828037   394 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:00.827564 26369 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.827942 26075 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.832237   364 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.832902   364 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.834122   385 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.834980   364 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.834862   385 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.829007   355 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.837510   385 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:00.827564 26295 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.829720 26400 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:32:00.828665   394 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:00.837968 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:32:00.841578 26369 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.842355 26224 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:00.843559   385 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:32:00.844108   385 raft_consensus.cc:686] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:00.845345   385 consensus_queue.cc:237] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.846621 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.847370 26406 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 21.
I20260924 14:32:00.848383 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:00.849450   390 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260924 14:32:00.849844   390 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.850216   390 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:32:00.858985   394 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.849721 26075 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 18 to 19, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.870237 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.869908 26075 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.870910 26323 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:32:00.872519 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.873818   355 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:00.873375 26075 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 15 to 17, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 17 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.874298   355 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.863251 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.864941   389 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.875138   355 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:00.875478   389 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.877977   389 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:00.865764 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:00.878877 26324 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 17: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:00.867110 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.880148 26330 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:00.867825 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:00.881026 26400 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:00.881361 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:00.882330   364 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:32:00.884033   364 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:00.884439   364 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.884853   364 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:00.885804 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.890177 32610 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:32:00.890552 32610 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.890941 32610 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:00.893735 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:00.899187   300 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.875049 26075 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.899854   300 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.890702 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.901957   300 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.901414 26075 catalog_manager.cc:5495] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 15 to 16, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 16 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:00.903419 26075 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 19, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 19 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:00.905385 26075 catalog_manager.cc:5495] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:32:00.908340   397 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:32:00.909485 26295 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.915171 26400 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:32:00.860668   390 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.903864 32610 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.874365 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:00.920557   364 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.922979   392 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:00.924898 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:00.898394   355 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:00.933688 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 59537.9us
I20260924 14:32:00.935784 32700 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:32:00.936223 32700 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.936654 32700 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:32:00.936805 32641 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.937698 32641 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.938313   364 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.932350   300 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:32:00.931998   389 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.945022 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.945449 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.946468 26253 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:00.947433   386 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.948120   386 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.948607 32610 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:00.950467 26369 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.950855   386 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.951548   389 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:00.953728 26369 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.957041 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.957317 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.958272 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.958971 26252 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:32:00.958811 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:32:00.950774 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:00.952162 32641 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.961297 26256 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:32:00.963486 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.965224 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:32:00.969455 26295 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.943481   300 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:00.978129   389 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:00.969300   364 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.979858 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.980029   364 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.981036 32610 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:00.981308 26400 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 24.
I20260924 14:32:00.976917   300 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.983058 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.983517 32610 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.983762 26330 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 24.
I20260924 14:32:00.983973 32610 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
W20260924 14:32:00.984992 26295 leader_election.cc:334] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.985510 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.985622 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:00.986402 32641 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:00.987069   392 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260924 14:32:00.987372 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.987510   392 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.987926   398 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.988024   392 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Advancing to term 25
W20260924 14:32:00.988530 26369 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.988745   397 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:32:00.989190   397 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.989233   364 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.989625   397 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:32:00.988576   398 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.992600   398 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.988745 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:00.995697 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.996352 26330 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:32:00.998457   397 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.000686   397 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.999372 32700 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.001880 26256 raft_consensus.cc:2379] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.002162 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.002883   397 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:00.941313   394 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.003240   397 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:00.987018 26221 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.011042   398 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.012794 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.005302 26295 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.997942 32641 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260924 14:32:01.013401   398 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.014073 32641 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.014711   397 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.014827 32641 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Advancing to term 25
I20260924 14:32:01.015671   398 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:01.004858 26369 leader_election.cc:334] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.017493 26369 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.019013 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.019730 26330 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.019902   389 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:01.005905   355 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.020977 32610 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.004379 32700 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.026307   392 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.033257 32700 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.033555 26330 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
W20260924 14:32:01.019868 26221 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.038640 26221 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.036161 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.023520 32610 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.039672 32700 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:01.040995 26221 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.041819 32700 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.043004 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.044282 26076 catalog_manager.cc:5495] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:01.045742 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.046010 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.046694 26323 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:32:01.047947 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.048056 32610 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.062250   300 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.075590   300 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.078019   300 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.066745   389 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.037197   392 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.080065   389 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.082227   389 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.068037 32700 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.068822 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.094916 26256 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 22: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.094717 32700 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.069674 26406 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:32:01.095644 32641 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.096992 32700 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.097381 26324 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 25: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:01.075276   397 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.071655 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.099756   397 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.100472 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.072320   385 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.102869   386 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260924 14:32:01.074108 26369 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.109416 26369 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.109845 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:32:01.110901 26369 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.063210   390 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.102984 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.111831 26256 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.113665 32641 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.113750   392 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:01.114400 32641 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:32:01.114586   392 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.115013 32610 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.115180   392 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.116256 32610 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.118011   392 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.072531   394 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.102449   385 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.106139 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.126394 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.107049 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.126662 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.127406 26255 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:32:01.127943 26324 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:32:01.128656 26253 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.127331 26400 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 25: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:32:01.130841   385 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.108362 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.119935   386 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.133299   386 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.134212   397 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.134864   397 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:01.136523 26295 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.132831 26249 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:01.137907   385 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:01.138967 26295 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.138749   385 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.142184   385 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.143097 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.143404 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.143967 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.144047 26400 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:32:01.144143 26401 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
W20260924 14:32:01.144415 26369 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.137034   397 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.145018 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.139689   394 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.148200   394 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.152096 26255 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:32:01.126670 26256 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:32:01.161181 26253 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:32:01.160522 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.163096 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.164038   390 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:32:01.164424   390 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.164798   390 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:01.164881   385 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:32:01.165457   385 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.165829   385 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:32:01.169926   386 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.158324 26406 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 22.
I20260924 14:32:01.171350 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.172039 26324 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:32:01.174870 26249 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
W20260924 14:32:01.177150 26221 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.177898 26369 leader_election.cc:334] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.178100 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:32:01.178514 26369 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.177320 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 76948.6us
I20260924 14:32:01.179723 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.180531 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.184195 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.179965 32700 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:32:01.176496   386 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:01.191803 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.193681 26295 leader_election.cc:334] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.189350   389 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Leader election won for term 22
I20260924 14:32:01.198057 26324 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:32:01.200631 32641 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:32:01.207551 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.204927   386 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260924 14:32:01.214720 32700 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.217284   392 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:32:01.217468 32700 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.222828 32700 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.224015 32700 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.224591 32700 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.213068 32641 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:01.225412 26369 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.225843 32641 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:32:01.226805 32700 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.233090   390 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.235281   390 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.263571   364 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.264364   364 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:01.268541 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 88097.8us
I20260924 14:32:01.271602 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.272215 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.272655 26324 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:32:01.272957 26406 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:32:01.268110   364 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.269924   394 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.274056   390 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.274389   394 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.274773   390 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.276650   394 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.240641   389 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:32:01.279574 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 95432.7us
I20260924 14:32:01.280283 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.280429 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.280910 26406 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:01.287591   364 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.287976   389 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.288221   364 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.284307   390 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.290287   364 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.291316   385 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.284437   394 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:32:01.296371 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.298702 26406 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:32:01.302136 26401 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:32:01.303862   385 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:01.305557 26369 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.301010 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.300349 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.307170 26406 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:01.308745 32641 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:01.310227 26369 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.311684 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.312554   409 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:32:01.314384   409 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.314939   409 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:32:01.313899 32641 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.312862 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.318683 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.319731 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.299728   394 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.320329   390 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:32:01.321976   385 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:32:01.323148   390 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.323746   385 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.324164   390 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:32:01.324266   385 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:32:01.322958 32700 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:32:01.322754   394 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:32:01.322693 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.321899 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.326841 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.328069 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.329857 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.330418   397 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.330821 26401 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:32:01.322398 32641 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260924 14:32:01.329155 26295 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.329485   364 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:32:01.341997   410 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:32:01.344457   397 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.345052 32610 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.348969   412 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:32:01.349223 26401 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
W20260924 14:32:01.360293 26295 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.328485 26221 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.349321 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.333346 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.362370 26221 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.363165 26221 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.364081 26221 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.365063 26221 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.368654 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46855.6us
I20260924 14:32:01.369699 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.357582   364 raft_consensus.cc:686] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.372401 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.373693 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:01.368726   355 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:32:01.372227   386 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:32:01.391474 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.369141   410 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.392479 26324 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 21.
I20260924 14:32:01.396225 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.397186 26324 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:32:01.399550   418 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260924 14:32:01.373893 32700 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:01.385341   300 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.372017   389 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260924 14:32:01.412094   364 consensus_queue.cc:237] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.414002   390 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.423544   355 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.426988   402 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:01.427536   402 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.427886   402 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.428653   385 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.401904   410 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:32:01.431735 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.433691 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.436566 26397 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:32:01.437558 26253 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.403614 32610 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.440673   411 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.440932 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.441666 26252 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:32:01.441412   411 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.442504   300 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.443473 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.444963 32700 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.376078   397 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.450726 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.453092 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.408449   394 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.457998   417 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.458601   417 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.460819   417 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:01.463053 23135 thread.cc:536] Waited for 1000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:01.467762 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:01.468110 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.468782 26330 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.431915   409 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.468672 32610 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.473069   414 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:32:01.473296   417 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.473860   414 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:01.473944   417 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.478574 26330 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:32:01.479650 26397 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:32:01.485493   300 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.426801   355 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:01.486655   413 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260924 14:32:01.475972   390 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.485575   420 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:32:01.455466   412 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:01.488591   402 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.500771   402 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.484524   385 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:01.509964 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53258.6us
I20260924 14:32:01.510942   412 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.511432 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.517057 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:32:01.534658 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 57806.6us
I20260924 14:32:01.538759 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.528335   424 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260924 14:32:01.535549 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57827.1us
I20260924 14:32:01.475102   414 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.544821   398 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.554733 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.555881 26250 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:32:01.557744 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.558058 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.558957 26256 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.559772 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.560860 26252 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.566293 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.567255 26253 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:32:01.569412 26256 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
I20260924 14:32:01.545419   398 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.539558   385 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:32:01.529186   417 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:01.583387 26221 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.547075   424 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:01.584827 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:01.587855 26221 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.589784 26221 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.575358   385 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:01.591310 26221 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.592039 26221 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.596292   415 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.596886   415 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.597057   410 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.599107   415 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.591156   385 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:01.599349   410 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:01.623049 26295 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.604303   398 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:01.626360 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:01.628916 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.607369 26221 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.612394   394 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.614166   411 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.587826   424 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:32:01.584931   409 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:01.632300 26221 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.644552 26221 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.645190 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.646124 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 103337us
W20260924 14:32:01.648065 26221 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.648628 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.649220 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.649847 26406 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.650377 26397 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:32:01.651396   409 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:01.653224 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.655225 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.655958 26402 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 20.
I20260924 14:32:01.659488 26406 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
I20260924 14:32:01.662627 26324 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260924 14:32:01.664011   364 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.664610   364 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.666741   364 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.667117   415 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:32:01.668816   410 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:32:01.669520   410 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:32:01.670600 26295 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.671860 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.676159   410 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.679589 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.680912 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.691607 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.692371 26256 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:01.693045 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.693892 26253 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 19: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.695822 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.696878 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.697619 26330 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 19: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:01.696527 26323 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
W20260924 14:32:01.698347 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:01.698917 26221 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.699412 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:01.700100   385 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:01.702314 26221 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.704516   398 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260924 14:32:01.705647 26295 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.706233 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:01.714967   424 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.719146 26250 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 22: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.719595   424 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.722683   385 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.687469   411 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:32:01.724334   411 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.724711   411 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:01.731149 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.689603   417 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:32:01.703650   394 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:32:01.734416 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.736569 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.738353 26324 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:01.738499   416 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.732851   417 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.740063   416 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:01.741070   416 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.739203   417 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:32:01.737182   300 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:32:01.743916   300 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.744424   300 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:32:01.747885 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.750790 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.752182 26327 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Advancing to term 22
W20260924 14:32:01.751646 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.753007 26221 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.753710 26221 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.754400 26221 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.751036 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:32:01.754851 26221 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.755148 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:32:01.755332 26221 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.755820 26330 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:32:01.756575   429 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.002s
I20260924 14:32:01.760723 26327 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 22.
I20260924 14:32:01.763106 26330 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:32:01.755148 26325 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:32:01.754552   427 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.117s	user 0.000s	sys 0.002s
I20260924 14:32:01.766152   411 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:01.770229 26221 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.821354 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 51217.4us
W20260924 14:32:01.822006 26221 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.772378 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.823498 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51234.9us
I20260924 14:32:01.824512 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.784081 26369 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.786038 26402 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260924 14:32:01.826316 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:01.789161   300 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.793896 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.797931   402 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:32:01.798803 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.839823 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:01.811407   417 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.829803 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.839069 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45265.5us
I20260924 14:32:01.844334 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.845232 32610 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260924 14:32:01.845912 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.846635   418 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:32:01.841104 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.852320 26323 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:01.842021 32641 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:32:01.839824   394 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:01.853137   405 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.853322 32641 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:32:01.853382   394 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.853770   405 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:01.854007   394 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.854404   417 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.856659   424 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:01.857138   424 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.857079 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:01.857666   424 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.858359 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.860273   385 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:32:01.860725   389 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Leader election won for term 22
I20260924 14:32:01.854768   405 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.861299   389 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:01.862221   389 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.863535   432 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:32:01.863991   432 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.864362   432 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:01.867851 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.867898   410 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260924 14:32:01.868736 26249 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:32:01.872260 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.872294 26256 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:32:01.874250 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.874840 26402 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:32:01.883654   423 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:32:01.889541   300 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.849936   409 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:32:01.904062 26256 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260924 14:32:01.868482   411 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.899158 26325 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:32:01.879388 32610 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:32:01.922544 32610 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.923015 32610 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:32:01.915375 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:01.924742 26295 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.926271 26221 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.926689 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 85844.2us
W20260924 14:32:01.927145 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:32:01.930403 26250 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260924 14:32:01.951166 26401 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260924 14:32:01.956528 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.952899 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.957232 26253 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:32:01.958325 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.958624   419 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:32:01.959019 26250 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:01.957835 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:01.966897 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.967823   419 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election won for term 18
W20260924 14:32:01.969823 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:01.977713   419 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:32:01.967865 26295 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.980724   414 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
W20260924 14:32:01.980908 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:01.981389   414 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:01.978775   419 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.982370   414 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.977435   432 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.973618   411 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:01.984776 26076 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 20 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:01.990886 26402 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 21.
I20260924 14:32:01.951038   392 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260924 14:32:01.993477   432 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.994017 26255 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260924 14:32:01.994107   411 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.995572 26400 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260924 14:32:01.996492   411 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.990734 26250 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:32:02.000946   390 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:32:02.002291 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.003480 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:02.008024 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:02.008687 26256 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:02.009546 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:02.010737   409 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:32:02.011267   409 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 21 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:02.011981 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:02.016705 26253 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:02.025307 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:02.017272 26256 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:32:01.939575   423 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260924 14:32:02.029778 26255 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
W20260924 14:32:02.107524 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 76569.8us
W20260924 14:32:02.107916   439 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260924 14:32:02.021855   416 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:32:01.990442   424 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:02.108915   300 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:32:02.151516 26075 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 144) took 3428 ms. Trace:
I20260924 14:32:02.105764   364 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260924 14:32:02.012212   409 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:02.110936   410 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260924 14:32:02.164647 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51023.4us
I20260924 14:32:02.177604 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:02.158726   300 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 19 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:02.152206 26075 rpcz_store.cc:276] 0924 14:31:58.717069 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:58.717319 (+   250us) service_pool.cc:227] Handling call
0924 14:32:02.144845 (+3427526us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:32:00.977064 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:32:00.977713 (+   649us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:32:00.977770 (+    57us) write_op.cc:435] Acquired schema lock
0924 14:32:00.977785 (+    15us) tablet.cc:660] Decoding operations
0924 14:32:00.978913 (+  1128us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:32:00.979016 (+   103us) write_op.cc:641] Partition lock acquired for write op
0924 14:32:00.979054 (+    38us) tablet.cc:683] Acquiring locks for 12 operations
0924 14:32:01.008892 (+ 29838us) tablet.cc:699] Row locks acquired
0924 14:32:01.008942 (+    50us) write_op.cc:260] PREPARE: finished
0924 14:32:01.009162 (+   220us) write_op.cc:270] Start()
0924 14:32:01.009362 (+   200us) write_op.cc:276] Timestamp: P: 1790260321009082 usec, L: 0
0924 14:32:01.009410 (+    48us) op_driver.cc:331] REPLICATION: starting
0924 14:32:01.010175 (+   765us) log.cc:842] Serialized 4845 byte log entry
0924 14:32:01.071123 (+ 60948us) op_driver.cc:444] REPLICATION: finished
0924 14:32:01.235792 (+164669us) write_op.cc:301] APPLY: starting
0924 14:32:01.235953 (+   161us) tablet.cc:1365] starting BulkCheckPresence
0924 14:32:01.317616 (+ 81663us) tablet.cc:1368] finished BulkCheckPresence
0924 14:32:01.317767 (+   151us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:32:01.546063 (+228296us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:32:01.590220 (+ 44157us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0924 14:32:01.606555 (+ 16335us) write_op.cc:312] APPLY: finished
0924 14:32:01.606869 (+   314us) log.cc:842] Serialized 112 byte log entry
0924 14:32:01.607062 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:01.607208 (+   146us) write_op.cc:454] Released schema lock
0924 14:32:01.607272 (+    64us) write_op.cc:341] FINISH: Updating metrics
0924 14:32:01.627912 (+ 20640us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:01.627946 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":25333,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":148937,"apply.run_cpu_time_us":22080,"apply.run_wall_time_us":392654,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54335,"num_ops":12,"prepare.queue_time_us":39737,"prepare.run_cpu_time_us":5172,"prepare.run_wall_time_us":50837,"replication_time_us":61607,"thread_start_us":136104,"threads_started":2}]]}
I20260924 14:32:02.221138   418 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election won for term 16
W20260924 14:32:02.243227   448 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260924 14:32:02.242966   300 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:02.244262 26077 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 144) took 1630 ms. Trace:
I20260924 14:32:02.244692 26077 rpcz_store.cc:276] 0924 14:32:00.514938 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:32:00.515386 (+   448us) service_pool.cc:227] Handling call
0924 14:32:02.127010 (+1611624us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":115670,"reported_tablets":37}
W20260924 14:32:02.246816   443 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260924 14:32:02.247130   446 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
W20260924 14:32:02.243302 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:02.191566   394 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:02.179924 32610 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:02.273703 32610 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.268828   394 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:02.270637   424 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:02.295598 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.297668 26295 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.304244 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.240442   416 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:02.329270   418 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:02.383006   418 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:02.399793 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:02.403151 26324 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 20: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:02.404448 26406 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260924 14:32:02.407279 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:02.415755 26323 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:32:02.420172 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:02.421777 26249 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260924 14:32:02.426008 26255 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 20: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:02.429406   432 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:02.429986   432 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:02.432284   432 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.432987   300 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:32:02.437059   456 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:32:02.440179   456 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:02.440546   456 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:32:02.442579 26369 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.443853 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.443863 26400 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:32:02.391150   416 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:02.448697 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.451459 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:32:02.453089 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:02.449297 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.463510 23135 thread.cc:536] Waited for 2000ms trying to join with diag-logger (tid 27219)
I20260924 14:32:02.464501 26249 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:32:02.480007   455 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:02.480724   455 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:02.483024   455 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.490808 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:02.491966 26400 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:32:02.496596   456 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:02.505894   456 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.509893 26327 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260924 14:32:02.510152   409 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
W20260924 14:32:02.513281 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48305.9us
I20260924 14:32:02.524785 26249 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:32:02.513005 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:02.534961 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:02.535681   456 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:32:02.536377   456 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:02.536882   456 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:02.553115 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:32:02.562103 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:02.574232 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.574781 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.575203 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.575654 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:02.577639 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:02.559788   455 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:32:02.584174 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:02.587594   434 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:32:02.541581   411 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260924 14:32:02.591697 26369 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.595685 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:32:02.580576   412 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:32:02.600718   465 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:32:02.601356   465 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:02.621197   414 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260924 14:32:02.623528 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:02.624164 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.624738 26295 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.625720 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:02.628700   465 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:02.642706 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:02.669487 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:02.670413 26250 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:02.719270   456 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:02.721029 26324 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 18: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:02.730103 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:02.730754 26406 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:02.731269 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:02.744012 26406 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:32:02.750885 26250 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:32:02.762773 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:02.749902 26076 catalog_manager.cc:5495] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 12, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 12 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:02.762745   434 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:32:02.771481 26076 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 20 to 21, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 21 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:02.778179 26323 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260924 14:32:02.784958 26327 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:32:02.789968 26076 catalog_manager.cc:5495] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 12 to 13, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 13 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:02.792265 26076 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 15 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 16 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:02.768452   456 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.773393   394 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:32:02.835887 26076 catalog_manager.cc:5495] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 21 to 22, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 22 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:02.846752 26076 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 14 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 15 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:02.842895   406 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:02.872077   406 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:02.874310   406 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.875733 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:02.876663 26400 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 25.
I20260924 14:32:02.879261 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:32:02.880046 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.880461   416 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260924 14:32:02.880872   416 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:02.881207   416 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Advancing to term 26
W20260924 14:32:02.888998 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:02.850550   394 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:02.858194   390 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260924 14:32:02.866889   385 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260924 14:32:02.854069 26076 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 17 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 18 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:02.901979 26076 catalog_manager.cc:5495] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 14 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 15 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:02.903319 26076 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 13 to 14, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 14 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:02.905452 26076 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 21 to 22, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 22 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:02.907883 26076 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 17 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 18 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:02.909447 26076 catalog_manager.cc:5495] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 12, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 12 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
W20260924 14:32:02.912650 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 59760.2us
I20260924 14:32:02.922034 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:02.930655 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.936167 26249 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260924 14:32:02.923996 26081 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 9) took 19322 ms (client timeout 15000 ms). Trace:
W20260924 14:32:02.938017 26081 rpcz_store.cc:269] 0924 14:31:43.571533 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:43.572537 (+  1004us) service_pool.cc:227] Handling call
0924 14:32:02.888542 (+19316005us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":197489,"reported_tablets":45}
I20260924 14:32:02.942670   394 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:02.945235 26295 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.947247 26067 connection.cc:441] server connection from 127.0.0.1:34948 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 9) could send its response
I20260924 14:32:02.939678   406 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:32:02.950393   406 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:02.951561   406 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:02.962169 26295 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.960335 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:02.978392 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.959376   416 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:02.982180   487 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260924 14:32:02.983846   416 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.992311 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:02.993005 26406 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 25 FOLLOWER]: Advancing to term 26
W20260924 14:32:02.998919   481 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:32:03.007160 26406 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 26.
W20260924 14:32:03.002041 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:03.013666 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:03.025475 26249 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260924 14:32:03.029346   416 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 26 FOLLOWER]: Leader election won for term 26
W20260924 14:32:03.030187 26295 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:03.042912   480 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.190s	user 0.001s	sys 0.002s
I20260924 14:32:03.031145   416 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 26 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:32:03.049203 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:03.075342   416 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:03.082986 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:03.084635 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:03.087980 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 58521.7us
W20260924 14:32:03.085358 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:32:03.119264 26400 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260924 14:32:03.133720 32700 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260924 14:32:03.136433   386 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260924 14:32:03.145520   392 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:32:03.154513 26256 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260924 14:32:03.181744 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:03.192189 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:03.220667   386 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:32:03.241282 26075 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 20 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.243870 26075 catalog_manager.cc:5495] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 20 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.255511 26256 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:32:03.266280 26075 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 20 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.268520 26075 catalog_manager.cc:5495] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 20 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.274857 26075 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 15 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 16 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.303570   398 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
W20260924 14:32:03.347571 26295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111) [suppressed 298 similar messages]
W20260924 14:32:03.359534 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260924 14:32:03.362067 26249 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260924 14:32:03.365201   461 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260924 14:32:03.378485 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:03.396796 26077 catalog_manager.cc:5495] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 20 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.398763 26077 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:03.400676 26077 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 18 to 19, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 19 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
W20260924 14:32:03.464442 23135 thread.cc:536] Waited for 3000ms trying to join with diag-logger (tid 27219)
I20260924 14:32:03.487255 26400 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 26 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260924 14:32:03.490865   461 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
W20260924 14:32:03.497176 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:03.501318 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:03.518544 26397 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260924 14:32:03.539119 26076 catalog_manager.cc:5495] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 12 to 13, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 13 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:32:03.546667   406 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:03.547264   406 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:03.549454   406 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:03.544971   461 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260924 14:32:03.565034 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:03.565819 26250 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
W20260924 14:32:03.568387 26295 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:03.572075 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:03.573122   461 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:32:03.573525   461 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:03.573858   461 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:32:03.604069 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:03.610446 26077 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 15 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 18 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:32:03.630054   461 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:03.645905   461 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:03.662119 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:03.662894 26250 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:03.654438 26076 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 24 to 26, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 26 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:32:03.672518 26076 catalog_manager.cc:5495] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 16 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 18 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:32:03.694213 26295 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:03.703424   394 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:03.688237 26250 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:32:03.704123   394 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:03.705986   394 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:03.711321 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:03.772408 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:03.773126 26250 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:03.774356 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:03.775310   416 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:32:03.775908   416 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:03.776888   416 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:03.786785 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260924 14:32:03.833629 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:03.834873   416 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:03.835280   416 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:03.835727   416 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:03.885890 26075 catalog_manager.cc:5495] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:03.913276   416 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:03.927506   416 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:03.938515 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:03.939975 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:03.940824 26255 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:03.972628 26255 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
W20260924 14:32:04.013576 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:04.016120 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:04.017671   461 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:04.018180   461 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:04.019066   461 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:04.063398 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:32:04.062372 26075 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 18 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 20 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
W20260924 14:32:04.084978 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:04.096690 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.100103 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:32:04.206463 32700 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:04.207029 32700 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:04.209573 32700 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:04.212435 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:04.213238 26325 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 20.
I20260924 14:32:04.215767 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:04.218844   414 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:32:04.219266   414 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:04.219659   414 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:32:04.252046 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:04.257329   414 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:04.266438   414 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:04.273851 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:04.274566 26324 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:32:04.295336 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.299718 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:04.303277 26324 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 21.
W20260924 14:32:04.313498 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:04.314790 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:04.316524   485 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:32:04.317181   485 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 21 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:04.318157   485 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:04.338184 26075 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 19 to 21, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 21 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:04.350034 26255 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:32:04.352231   461 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260924 14:32:04.395020 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.396862 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:04.461097 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.465000 23135 thread.cc:536] Waited for 4000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:04.477735 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.551044 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:04.582283 26255 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260924 14:32:04.586233 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:04.589020 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:04.591241   521 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260924 14:32:04.664917 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.743701 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:04.858108 26325 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260924 14:32:04.878567 32700 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260924 14:32:04.886654 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.886701 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.934119 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:04.938823 26221 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:04.957712 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.961930 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.999856 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.043694 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:05.085246 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:05.127848 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.147665 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.213132 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.251555 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:05.254011 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:05.380529 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.423321 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.465592 23135 thread.cc:536] Waited for 5000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:05.474762 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:05.618351 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.622720 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.660635 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.663894 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.696477 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.752954 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:05.753487 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:05.777654 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.877118 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:05.909227 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:06.118983 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:06.188231 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:06.358372 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:06.466147 23135 thread.cc:536] Waited for 6000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:06.537040 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.539530 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.579761 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.665647 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.679105 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.764724 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:06.843623 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:06.882862 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:06.923712 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.981437 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.985966 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:07.152168 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:07.161672 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.189718 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.274817 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:07.298156 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.308178 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:07.326917 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.380906 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:07.395241 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:07.466741 23135 thread.cc:536] Waited for 7000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:07.593981 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:07.601353 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.608397 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.633653 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.648619 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:07.649770 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.667253 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.667994 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:07.752130 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.779520 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.816569 26221 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:07.930763 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.042743 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.059064 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.089241 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.106432 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.160486 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:08.234573 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.292104 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.359018 26295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111) [suppressed 451 similar messages]
W20260924 14:32:08.450655 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:08.467319 23135 thread.cc:536] Waited for 8000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:08.531231 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:08.557864 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.649145 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:08.800710 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:08.920392 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:08.961392 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:08.998746 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.079344 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 43516.1us
W20260924 14:32:09.122993 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.165695 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.196924 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.258316 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:09.265825 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:09.342201 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.365617 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:09.468688 23135 thread.cc:536] Waited for 9000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:09.533756 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.559929 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:09.702145 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.761616 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:09.784289 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.875020 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:09.938082 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:09.994871 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 56829.5us
W20260924 14:32:09.986171 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:10.017674 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:10.121649 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:10.134660 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.136168 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.191007 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.202419 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.207734 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:10.231009 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:10.241228 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.277853 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.278766 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.306414 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.439465 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.454231 26221 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:10.469246 23135 thread.cc:536] Waited for 10000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:10.535307 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.537303 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.548631 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.622416 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.803813 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46896.4us
W20260924 14:32:10.816978 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.817703 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.818161 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.934212 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:11.000535 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:11.026770 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.027469 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.214046 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:11.270044 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:11.316033 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:11.346325 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:11.469996 23135 thread.cc:536] Waited for 11000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:11.639575 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:11.662381 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:11.668916 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.690331 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:11.744500 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:11.865516 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.895305 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.921384 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.990017 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.089957 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.127990 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:12.156176 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.186362 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.232553 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:12.306815 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:12.382205 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:12.385437 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:12.470602 23135 thread.cc:536] Waited for 12000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:12.527870 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.530433 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.590946 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:12.621915 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.641074 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.721047 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.737201 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.750085 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.779796 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.785885 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44812.1us
W20260924 14:32:12.788272 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:12.890061 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.922580 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.930160 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:12.977823 26221 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:13.011983 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.047322 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.194326 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.216627 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.218266 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.359862 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.366847 26369 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111) [suppressed 449 similar messages]
W20260924 14:32:13.371505 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:13.471166 23135 thread.cc:536] Waited for 13000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:13.540091 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:13.586851 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:13.597900 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:13.628669 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.656071 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:13.806099 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:13.861379 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.076220 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.083030 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.085004 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260924 14:32:14.195957 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.230487 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.362087 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.366910 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.432564 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.461149 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:14.471807 23135 thread.cc:536] Waited for 14000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:14.512750 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.575033 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.695355 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.701464 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.705886 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.889232 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.909308 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260924 14:32:14.994355 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.007460 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.009084 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.046173 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.089067 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.110836 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:15.169303 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:15.223198 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.283408 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.300957 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:15.306639 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.308439 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.320853 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.461051 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 104636us
W20260924 14:32:15.472361 23135 thread.cc:536] Waited for 15000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:15.541515 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.544548 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.618414 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260924 14:32:16.416213   552 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.418824   529 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.418464   552 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:16.419503   529 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:16.421555   552 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:32:16.421720   529 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:16.444757   552 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.449097   552 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:16.451212   552 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:16.473031 23135 thread.cc:536] Waited for 16000ms trying to join with diag-logger (tid 27219)
I20260924 14:32:16.475121 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:32:16.541529 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 1.02258e+06us
W20260924 14:32:16.544014 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260924 14:32:16.544991 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260924 14:32:16.546465 26295 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:16.547555 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260924 14:32:16.548591   535 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:32:16.551819   535 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:16.552901 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:16.554500 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:16.560379 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:16.562561   535 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:32:16.568228   549 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.568873   549 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:16.571384   549 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:16.579276   549 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:32:16.548743 26295 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:16.579874   549 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:16.582037   549 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:16.595162 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 105609us
I20260924 14:32:16.597663   549 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.598448   555 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.599061   555 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:16.600780 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:16.601250   555 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:16.598315   549 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:16.605435   549 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:16.602563 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 94217.8us
I20260924 14:32:16.612133 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:16.617640 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:16.620920 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:32:16.621950   557 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:16.624109 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:16.624972   557 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:32:16.633728 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:16.634629   555 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:32:16.631276 26295 leader_election.cc:334] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:16.637068 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:16.645599 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:32:16.652107 26279 debug-util.cc:398] Leaking SignalData structure 0x7b0800143f40 after lost signal to thread 23138
W20260924 14:32:16.653436 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08001b3b20 after lost signal to thread 26115
W20260924 14:32:16.654861 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08001ae1a0 after lost signal to thread 26282
W20260924 14:32:16.658756 26279 debug-util.cc:398] Leaking SignalData structure 0x7b0800039440 after lost signal to thread 26356
W20260924 14:32:16.660147 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08001ff200 after lost signal to thread 26430
I20260924 14:32:16.669009 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:32:16.672854 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260924 14:32:16.674644 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:16.675246 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:16.676157 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:32:16.681625 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260924 14:32:16.683204 26224 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:16.684299   560 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:32:16.696200 26369 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:16.696887 26369 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:16.701152 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:16.702467   545 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:16.703783 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:16.704640   549 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:16.705372 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:16.706161   549 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:16.706959 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:16.707993   549 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260924 14:32:16.708837 26221 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:16.709681 23140 debug-util.cc:398] Leaking SignalData structure 0x7b080015c8a0 after lost signal to thread 26224
W20260924 14:32:16.709439 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:16.853425 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:16.890489 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:16.958623 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:17.027992 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260924 14:32:17.123510 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:17.154446 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:17.341912 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:17.430331 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:17.473570 23135 thread.cc:536] Waited for 17000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:17.559871 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:17.593525 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:17.611068 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:17.621913 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260924 14:32:17.668937 23135 tablet_server.cc:194] TabletServer@127.22.151.193:38943 shutdown complete.
W20260924 14:32:17.772559 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260924 14:32:17.947808 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.014549 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:18.078135 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 63683.7us
W20260924 14:32:18.074388 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:18.079051 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.053593 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 109982us
W20260924 14:32:18.079041 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.110038 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:18.128418 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:18.129487 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:18.148240 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.190512 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.252377 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:18.330431 23135 tablet_server.cc:178] TabletServer@127.22.151.194:0 shutting down...
W20260924 14:32:18.340801 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.365608 26371 consensus_peers.cc:487] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.376348 26297 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 384 similar messages]
W20260924 14:32:18.380014 26297 consensus_peers.cc:487] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.388661 26371 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.398794 26297 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.404399 26371 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.406152 26297 consensus_peers.cc:487] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.446000 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.446077 26371 consensus_peers.cc:487] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.467770 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:18.470865 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.501717 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.506345 26371 consensus_peers.cc:487] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.540619 26297 consensus_peers.cc:487] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.544828 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:18.559530 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:32:18.563313 23135 tablet_replica.cc:330] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.564400 23135 raft_consensus.cc:2229] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.565737 23135 raft_consensus.cc:2258] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.570778 23135 tablet_replica.cc:330] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:18.579526 26371 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.584306 23135 raft_consensus.cc:2229] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.585678 23135 raft_consensus.cc:2258] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.588786 23135 tablet_replica.cc:330] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.589807 23135 raft_consensus.cc:2229] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.591254 23135 raft_consensus.cc:2258] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:18.595682 26297 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.609375 26371 consensus_peers.cc:487] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.611354 23135 tablet_replica.cc:330] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.612601 23135 raft_consensus.cc:2229] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.613910 23135 raft_consensus.cc:2258] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.617079 23135 tablet_replica.cc:330] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.620443 23135 raft_consensus.cc:2229] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.621763 23135 raft_consensus.cc:2258] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:18.633249 26297 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.634368 26297 consensus_peers.cc:487] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.634572 23135 tablet_replica.cc:330] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.635649 23135 raft_consensus.cc:2229] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.636857 23135 raft_consensus.cc:2258] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.639994 23135 tablet_replica.cc:330] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.640933 23135 raft_consensus.cc:2229] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Raft consensus shutting down.
W20260924 14:32:18.646214 26371 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.652047 26371 consensus_peers.cc:487] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.653678 23135 raft_consensus.cc:2258] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:18.654726 26297 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.657130 23135 tablet_replica.cc:330] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.658162 23135 raft_consensus.cc:2229] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 21 LEADER]: Raft consensus shutting down.
W20260924 14:32:18.659325 26297 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.659369 23135 raft_consensus.cc:2258] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.664417 23135 tablet_replica.cc:330] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:18.665073 26297 consensus_peers.cc:487] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.666733 23135 raft_consensus.cc:2229] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:18.681123 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260924 14:32:18.712698 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:18.667384 23135 raft_consensus.cc:2258] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.720700 23135 tablet_replica.cc:330] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.732659 23135 raft_consensus.cc:2229] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Raft consensus shutting down.
W20260924 14:32:18.737448 26297 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.740141 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:18.747148 23135 raft_consensus.cc:2258] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.750255 23135 tablet_replica.cc:330] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.751144 23135 raft_consensus.cc:2229] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Raft consensus shutting down.
W20260924 14:32:18.754933 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:18.759353 23135 raft_consensus.cc:2258] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.762657 23135 tablet_replica.cc:330] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.763578 23135 raft_consensus.cc:2229] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.764456 23135 raft_consensus.cc:2258] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:18.770335 26371 consensus_peers.cc:487] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.779896 23135 tablet_replica.cc:330] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.781232 23135 raft_consensus.cc:2229] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.782124 23135 raft_consensus.cc:2258] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.785611 23135 tablet_replica.cc:330] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:18.789125 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260924 14:32:18.806779 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:18.801492 23135 raft_consensus.cc:2229] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 LEADER]: Raft consensus shutting down.
W20260924 14:32:18.807562 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:18.808418 23135 raft_consensus.cc:2258] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.811614 23135 tablet_replica.cc:330] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.821934 23135 raft_consensus.cc:2229] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.823223 23135 raft_consensus.cc:2258] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.826202 23135 tablet_replica.cc:330] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:18.828771 26297 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.828991 23135 raft_consensus.cc:2229] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.830346 23135 raft_consensus.cc:2258] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.833050 23135 tablet_replica.cc:330] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.838567 23135 raft_consensus.cc:2229] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.846748 23135 raft_consensus.cc:2258] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.849704 23135 tablet_replica.cc:330] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.850494 23135 raft_consensus.cc:2229] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.851362 23135 raft_consensus.cc:2258] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.854121 23135 tablet_replica.cc:330] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.864084 23135 raft_consensus.cc:2229] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.864755 23135 raft_consensus.cc:2258] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.867586 23135 tablet_replica.cc:330] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.868438 23135 raft_consensus.cc:2229] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.869555 23135 raft_consensus.cc:2258] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.872422 23135 tablet_replica.cc:330] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:18.876981 26371 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.883754 23135 raft_consensus.cc:2229] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.884516 23135 raft_consensus.cc:2258] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:18.885229 26371 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.892935 23135 tablet_replica.cc:330] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.893846 23135 raft_consensus.cc:2229] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.894661 23135 raft_consensus.cc:2258] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.897934 23135 tablet_replica.cc:330] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.898801 23135 raft_consensus.cc:2229] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.900507 23135 raft_consensus.cc:2258] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.903558 23135 tablet_replica.cc:330] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.911612 23135 raft_consensus.cc:2229] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.913079 23135 raft_consensus.cc:2258] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.916152 23135 tablet_replica.cc:330] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.918943 23135 raft_consensus.cc:2229] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.919655 23135 raft_consensus.cc:2258] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:19.016225 26353 debug-util.cc:398] Leaking SignalData structure 0x7b0800261fc0 after lost signal to thread 23138
W20260924 14:32:19.041570 26353 debug-util.cc:398] Leaking SignalData structure 0x7b08000fb000 after lost signal to thread 26115
W20260924 14:32:19.043036 26353 debug-util.cc:398] Leaking SignalData structure 0x7b08001f8d60 after lost signal to thread 26282
W20260924 14:32:19.044479 26353 debug-util.cc:398] Leaking SignalData structure 0x7b080025b6e0 after lost signal to thread 26356
W20260924 14:32:19.045689 26353 debug-util.cc:398] Leaking SignalData structure 0x7b08000d52c0 after lost signal to thread 26430
I20260924 14:32:19.047983 23135 tablet_replica.cc:330] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.048913 23135 raft_consensus.cc:2229] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.049685 23135 raft_consensus.cc:2258] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.053020 23135 tablet_replica.cc:330] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.078039 23135 raft_consensus.cc:2229] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.079231 23135 raft_consensus.cc:2258] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.112622 23135 tablet_replica.cc:330] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.113555 23135 raft_consensus.cc:2229] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Raft consensus shutting down.
I20260924 14:32:19.114854 23135 raft_consensus.cc:2258] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.127461 23135 tablet_replica.cc:330] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.128191 23135 raft_consensus.cc:2229] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.133363 23135 raft_consensus.cc:2258] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.136253 23135 tablet_replica.cc:330] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.136973 23135 raft_consensus.cc:2229] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.137739 23135 raft_consensus.cc:2258] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.146116 23135 tablet_replica.cc:330] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.146978 23135 raft_consensus.cc:2229] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:19.148171 23135 raft_consensus.cc:2258] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.150987 23135 tablet_replica.cc:330] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:19.153993 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260924 14:32:19.160554 23135 raft_consensus.cc:2229] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.161512 23135 raft_consensus.cc:2258] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.164832 23135 tablet_replica.cc:330] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.165659 23135 raft_consensus.cc:2229] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.166491 23135 raft_consensus.cc:2258] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.169714 23135 tablet_replica.cc:330] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.170553 23135 raft_consensus.cc:2229] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Raft consensus shutting down.
I20260924 14:32:19.171872 23135 raft_consensus.cc:2258] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.174804 23135 tablet_replica.cc:330] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.175683 23135 raft_consensus.cc:2229] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.176419 23135 raft_consensus.cc:2258] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.190414 23135 tablet_replica.cc:330] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.195945 23135 raft_consensus.cc:2229] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.196841 23135 raft_consensus.cc:2258] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.208304 23135 tablet_replica.cc:330] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.209239 23135 raft_consensus.cc:2229] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 21 LEADER]: Raft consensus shutting down.
I20260924 14:32:19.216506 23135 raft_consensus.cc:2258] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.220135 23135 tablet_replica.cc:330] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.221069 23135 raft_consensus.cc:2229] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.221843 23135 raft_consensus.cc:2258] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.226089 23135 tablet_replica.cc:330] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.226979 23135 raft_consensus.cc:2229] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.227918 23135 raft_consensus.cc:2258] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.246472 23135 tablet_replica.cc:330] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.247500 23135 raft_consensus.cc:2229] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:19.248291 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:19.248292 23135 raft_consensus.cc:2258] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.252257 23135 tablet_replica.cc:330] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.253113 23135 raft_consensus.cc:2229] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.253758 23135 raft_consensus.cc:2258] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:19.256636 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:19.256806 23135 tablet_replica.cc:330] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.257834 23135 raft_consensus.cc:2229] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.258658 23135 raft_consensus.cc:2258] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.263123 23135 tablet_replica.cc:330] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.263989 23135 raft_consensus.cc:2229] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.264699 23135 raft_consensus.cc:2258] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.268537 23135 tablet_replica.cc:330] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.269280 23135 raft_consensus.cc:2229] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.270128 23135 raft_consensus.cc:2258] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:19.342978 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:19.556044 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260924 14:32:19.652318 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:19.701676 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:19.740130   575 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.740695   575 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:19.742730   575 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:19.744280 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:32:19.749516 26371 leader_election.cc:334] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:19.749939 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:19.751178   575 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:19.764598   575 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.765170   575 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:19.767282   575 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:19.776144 26371 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:19.793493   575 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.794186   575 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:19.800388   567 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.801011   567 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:19.803002   567 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:19.805472   575 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:19.807646 26371 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:19.809374 26297 leader_election.cc:334] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:19.811221 26295 leader_election.cc:334] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:19.811772 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:19.812994   567 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260924 14:32:19.824493 26369 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:19.825232 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:19.827016   575 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:19.828769 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:32:19.831350 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:19.831740 26369 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:19.832561   575 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:32:19.843760   575 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.844285   575 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:19.846359   575 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:19.855724 26371 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:19.858784 26369 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:19.859280 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:19.860817   573 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:32:19.930563   575 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.931156   575 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:19.933431   575 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:19.938386 26371 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:19.939091 26369 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:19.939582 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:19.940346   575 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:32:19.962396   575 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.963023   575 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:19.970981 26371 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:19.978031 26369 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:19.978503 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:19.982013   573 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:19.965386   575 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:20.011761 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:20.014333   567 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.015060   567 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.019575   581 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.020125   581 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.021078   567 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:32:20.021919 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:20.022221   581 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:32:20.022604 26397 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:20.025978 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:32:20.026899 26297 leader_election.cc:334] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.028426   581 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.028973   581 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.033608   581 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:20.034570 26297 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.026952   567 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:20.030687 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:20.037180   567 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.037436 26397 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:32:20.037678   567 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:20.044416 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:32:20.045600   581 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:32:20.045989   581 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.046361   581 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:32:20.051247 26295 leader_election.cc:334] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.059851   575 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:32:20.060038 26297 leader_election.cc:334] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.060504 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.061605   582 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:20.060555   575 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:20.071012 26369 leader_election.cc:334] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.073745   575 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.069621   581 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.090708   581 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:32:20.081470   567 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.093591   567 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:32:20.095129 26297 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.098850 26297 leader_election.cc:334] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.100247 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:20.102516 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:20.104497 26397 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:32:20.104348 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:20.105219 26406 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:20.113507 26406 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
I20260924 14:32:20.132366   581 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.133044   581 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.135094   581 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.142510 26397 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
W20260924 14:32:20.145285 26371 leader_election.cc:334] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.155045 26297 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.158547   575 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.170012   575 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:20.172338   567 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.172945   567 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.174937   567 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.174988   581 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.175559   581 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.175844   573 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.176414   573 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:20.177601   581 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.178326   573 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.169369 26371 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.183593   573 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:32:20.182775 26297 leader_election.cc:334] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.183414 26369 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:20.184227 26297 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.185978 26295 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.187057 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:32:20.188351   567 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:20.189132   567 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:20.189997   567 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.186147   575 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.194136 26295 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.194350 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:32:20.199661   581 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:20.200165   582 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:32:20.200714   582 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:20.201625   582 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:32:20.228495 26369 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.251655   573 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:32:20.251853 26371 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.252394 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.252238   573 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:20.254415   573 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:20.256512 26371 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.256831   582 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.257043 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.257443   582 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.259714   575 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260924 14:32:20.260263 26369 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.261646   582 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:32:20.262639 26295 leader_election.cc:334] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.263033 26295 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.263629   582 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:32:20.263839 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.264269 26295 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.264874   582 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260924 14:32:20.265072 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:20.269641   594 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:32:20.274562 26297 leader_election.cc:334] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.274533 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:20.282199 26298 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:32:20.283609   582 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:20.282130 26075 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 19 to 20, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 20 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
W20260924 14:32:20.285847 26371 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.286320 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.287019   594 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:32:20.286707 26075 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 18 to 19, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 19 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
W20260924 14:32:20.296924 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:20.310659   589 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.311378   589 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.320747   589 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:20.327977 26295 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:20.336241 26297 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.336994 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.338256   589 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:32:20.338925   594 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.339530   594 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:20.342406   594 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.342653 23135 tablet_server.cc:194] TabletServer@127.22.151.194:0 shutdown complete.
W20260924 14:32:20.345252 26371 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.347048 26369 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.347585 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.348781   594 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260924 14:32:20.354873 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:20.476528 23135 tablet_server.cc:178] TabletServer@127.22.151.195:0 shutting down...
W20260924 14:32:20.504868 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:20.510370 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:20.543242   594 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.543887   594 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.546155   594 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:20.555068 26371 leader_election.cc:334] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.560209 26369 leader_election.cc:334] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:32:20.560755 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:20.561640   594 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:20.568717 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:32:20.574366 23135 tablet_replica.cc:330] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.575567 23135 raft_consensus.cc:2229] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.576340   582 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.576905 23135 pending_rounds.cc:62] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Trying to abort 1 pending ops.
I20260924 14:32:20.577315 23135 pending_rounds.cc:68] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Aborting op as it isn't in flight: id { term: 19 index: 12 } timestamp: 7332906353479512064 op_type: NO_OP noop_request { }
I20260924 14:32:20.578966 23135 raft_consensus.cc:2866] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 14:32:20.579419 23135 raft_consensus.cc:2258] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.576905   582 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.582849 23135 tablet_replica.cc:330] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.584384 23135 raft_consensus.cc:2229] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
W20260924 14:32:20.585373 26295 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.585670 23135 raft_consensus.cc:2258] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.589224 23135 tablet_replica.cc:330] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.590278 23135 raft_consensus.cc:2229] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.590919 23135 raft_consensus.cc:2258] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.591787 26297 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.592273 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.593041   589 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:32:20.594137 23135 tablet_replica.cc:330] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.588405   582 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.595619   589 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.596151   589 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.596612 23135 raft_consensus.cc:2229] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.597605 23135 raft_consensus.cc:2258] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.600479 23135 tablet_replica.cc:330] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.602094 23135 raft_consensus.cc:2229] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.602825 23135 raft_consensus.cc:2258] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.603564 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260924 14:32:20.605995 23135 tablet_replica.cc:330] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.606976 23135 raft_consensus.cc:2229] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.607232   589 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.607823 23135 raft_consensus.cc:2258] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.611418 23135 tablet_replica.cc:330] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.612457 23135 raft_consensus.cc:2229] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.613234 23135 raft_consensus.cc:2258] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.616106 23135 tablet_replica.cc:330] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.616992 23135 raft_consensus.cc:2229] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Raft consensus shutting down.
W20260924 14:32:20.618256 26297 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.619275 23135 raft_consensus.cc:2258] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.622704 23135 tablet_replica.cc:330] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.623658 23135 raft_consensus.cc:2229] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.625011 23135 raft_consensus.cc:2258] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.627808 23135 tablet_replica.cc:330] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.628746 23135 raft_consensus.cc:2229] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.629565 23135 pending_rounds.cc:62] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Trying to abort 1 pending ops.
I20260924 14:32:20.629849 23135 pending_rounds.cc:68] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7332906353427582976 op_type: NO_OP noop_request { }
I20260924 14:32:20.630285 23135 raft_consensus.cc:2866] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 14:32:20.630656 23135 raft_consensus.cc:2258] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.631937 26295 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.635607 26295 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.637574   582 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:32:20.639236 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:20.646582 23135 tablet_replica.cc:330] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.647691 23135 raft_consensus.cc:2229] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.648420 23135 raft_consensus.cc:2258] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.651503 23135 tablet_replica.cc:330] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.652309 23135 raft_consensus.cc:2229] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.653594 23135 raft_consensus.cc:2258] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.656509 23135 tablet_replica.cc:330] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
W20260924 14:32:20.656616 26369 consensus_peers.cc:487] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.657428 23135 raft_consensus.cc:2229] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.658169 23135 raft_consensus.cc:2258] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.661293 23135 tablet_replica.cc:330] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.662246 23135 raft_consensus.cc:2229] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.666383 23135 raft_consensus.cc:2258] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.671438 23135 tablet_replica.cc:330] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.674365 23135 raft_consensus.cc:2229] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.675594 23135 raft_consensus.cc:2258] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.678705 23135 tablet_replica.cc:330] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.679689 23135 raft_consensus.cc:2229] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.681979 23135 raft_consensus.cc:2258] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.684840 23135 tablet_replica.cc:330] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.685684 23135 raft_consensus.cc:2229] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.686364 23135 raft_consensus.cc:2258] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.690737 23135 tablet_replica.cc:330] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.691707 23135 raft_consensus.cc:2229] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
W20260924 14:32:20.699717 26369 consensus_peers.cc:487] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.700645 23135 raft_consensus.cc:2258] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.703979 23135 tablet_replica.cc:330] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.704888 23135 raft_consensus.cc:2229] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.706063 23135 raft_consensus.cc:2258] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.709087 23135 tablet_replica.cc:330] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.709970 23135 raft_consensus.cc:2229] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.711236 23135 raft_consensus.cc:2258] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.714181 23135 tablet_replica.cc:330] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.715063 23135 raft_consensus.cc:2229] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.715956 23135 raft_consensus.cc:2258] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.718729 23135 tablet_replica.cc:330] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.719591 23135 raft_consensus.cc:2229] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.720829 23135 raft_consensus.cc:2258] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.724939 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:20.725569 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.764430 23135 tablet_replica.cc:330] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.768420 23135 raft_consensus.cc:2229] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Raft consensus shutting down.
W20260924 14:32:20.780337 26369 consensus_peers.cc:487] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.785007 23135 raft_consensus.cc:2258] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.789222 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.791559 23135 tablet_replica.cc:330] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.793839 23135 raft_consensus.cc:2229] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.797737 23135 raft_consensus.cc:2258] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.800458 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.801757 23135 tablet_replica.cc:330] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.805200 23135 raft_consensus.cc:2229] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.806350 23135 raft_consensus.cc:2258] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.810393 23135 tablet_replica.cc:330] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.811990 23135 raft_consensus.cc:2229] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.813992 23135 raft_consensus.cc:2258] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.819355 23135 tablet_replica.cc:330] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.820537 23135 raft_consensus.cc:2229] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.822606 23135 raft_consensus.cc:2258] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.826314 23135 tablet_replica.cc:330] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.827842 23135 raft_consensus.cc:2229] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.830291 23135 raft_consensus.cc:2258] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.834012 23135 tablet_replica.cc:330] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.834873 23135 raft_consensus.cc:2229] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.838030 23135 raft_consensus.cc:2258] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.843384 23135 tablet_replica.cc:330] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.844344 23135 raft_consensus.cc:2229] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.847525 23135 raft_consensus.cc:2258] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.852530 23135 tablet_replica.cc:330] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
W20260924 14:32:20.853818 26369 consensus_peers.cc:487] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:20.860987 26371 consensus_peers.cc:487] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:32:20.853897 23135 raft_consensus.cc:2229] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.864837 23135 raft_consensus.cc:2258] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.868923 23135 tablet_replica.cc:330] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.869902 23135 raft_consensus.cc:2229] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.871351 23135 raft_consensus.cc:2258] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.875974 23135 tablet_replica.cc:330] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.876915 23135 raft_consensus.cc:2229] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:20.877907 26371 consensus_peers.cc:487] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:32:20.878228 23135 raft_consensus.cc:2258] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.882902 23135 tablet_replica.cc:330] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.883945 23135 raft_consensus.cc:2229] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.886997 23135 raft_consensus.cc:2258] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.891323 23135 tablet_replica.cc:330] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.892376 23135 raft_consensus.cc:2229] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.893333 23135 raft_consensus.cc:2258] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.897696 23135 tablet_replica.cc:330] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.898619 23135 raft_consensus.cc:2229] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.902264 23135 raft_consensus.cc:2258] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.906579 23135 tablet_replica.cc:330] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.907662 23135 raft_consensus.cc:2229] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.909608 23135 raft_consensus.cc:2258] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.914467 23135 tablet_replica.cc:330] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.915637 23135 raft_consensus.cc:2229] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.917023 23135 raft_consensus.cc:2258] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.921238 23135 tablet_replica.cc:330] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.922166 23135 raft_consensus.cc:2229] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.924371 23135 raft_consensus.cc:2258] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.928529 23135 tablet_replica.cc:330] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.929448 23135 raft_consensus.cc:2229] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Raft consensus shutting down.
W20260924 14:32:20.931389 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260924 14:32:20.932300 23135 raft_consensus.cc:2258] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.936990 23135 tablet_replica.cc:330] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.937865 23135 raft_consensus.cc:2229] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.939168 23135 raft_consensus.cc:2258] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.942528 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260924 14:32:20.943146 23135 tablet_replica.cc:330] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.944168 23135 raft_consensus.cc:2229] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 26 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.945822 23135 raft_consensus.cc:2258] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.949870 23135 tablet_replica.cc:330] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.950976 23135 raft_consensus.cc:2229] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.951965 23135 raft_consensus.cc:2258] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.957347 23135 tablet_replica.cc:330] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.958338 23135 raft_consensus.cc:2229] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.960721 23135 raft_consensus.cc:2258] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.970311 26369 consensus_peers.cc:487] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:20.993076 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:21.019672 26371 consensus_peers.cc:487] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.022562 26371 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.036401 26369 consensus_peers.cc:487] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:21.053937 26371 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.097529 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:21.098291 26371 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.110530 26371 consensus_peers.cc:487] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.142534 26371 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.204540 26371 consensus_peers.cc:487] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.357369 26371 consensus_peers.cc:487] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.421296 26371 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.425014 26371 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.491405 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:21.561273 23135 tablet_server.cc:194] TabletServer@127.22.151.195:0 shutdown complete.
I20260924 14:32:21.616537   594 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:32:21.617383   594 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:21.626499   601 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:21.627336   601 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:21.630025   601 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:21.635528 26371 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:21.620981   594 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:21.641641 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Network error: Client connection negotiation failed: client connection to 127.22.151.195:45507: connect: Connection refused (error 111)
W20260924 14:32:21.643658 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.644212 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:21.645145   594 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260924 14:32:21.649324 26369 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.649901 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:21.650810   594 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:21.674921 23135 tablet_server.cc:178] TabletServer@127.22.151.196:0 shutting down...
I20260924 14:32:21.745576   594 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:32:21.746374   594 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:21.748896   594 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:21.756474 26369 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Network error: Client connection negotiation failed: client connection to 127.22.151.195:45507: connect: Connection refused (error 111)
W20260924 14:32:21.757248 26369 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.757684 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:21.758469   594 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:21.774099 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:32:21.775378 23135 tablet_replica.cc:330] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.776630 23135 raft_consensus.cc:2229] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.777767   594 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:32:21.778342 23135 raft_consensus.cc:2258] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.778328   594 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:21.784106 23135 tablet_replica.cc:330] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.785684 23135 raft_consensus.cc:2229] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:21.785715 26371 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:21.787261 23135 raft_consensus.cc:2258] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.791538 23135 tablet_replica.cc:330] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.792837 23135 raft_consensus.cc:2229] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.793655 23135 raft_consensus.cc:2258] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:21.798586 26369 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Network error: Client connection negotiation failed: client connection to 127.22.151.195:45507: connect: Connection refused (error 111)
I20260924 14:32:21.799141 26369 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:21.799974   601 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:21.800832 23135 tablet_replica.cc:330] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.801934 23135 raft_consensus.cc:2229] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.803115 23135 raft_consensus.cc:2258] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.806807   601 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:32:21.806944   594 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:21.807461   601 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:21.809818   601 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:21.811715 23135 tablet_replica.cc:330] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.814378 23135 raft_consensus.cc:2229] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.815920 23135 raft_consensus.cc:2258] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.820498 23135 tablet_replica.cc:330] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.829037 23135 raft_consensus.cc:2229] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.830577 23135 raft_consensus.cc:2258] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.833830 23135 tablet_replica.cc:330] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.834928 23135 raft_consensus.cc:2229] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.835804 23135 raft_consensus.cc:2258] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.835865   594 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:21.836611   594 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:21.838814   594 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:21.839855 23135 tablet_replica.cc:330] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.841127 23135 raft_consensus.cc:2229] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:21.841899 26369 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.842155 23135 raft_consensus.cc:2258] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:21.848136 26371 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:21.848897 23135 tablet_replica.cc:330] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.850034 23135 raft_consensus.cc:2229] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.850915 23135 raft_consensus.cc:2258] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.854542 23135 tablet_replica.cc:330] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.855646 23135 raft_consensus.cc:2229] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:21.855681 26369 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Network error: Client connection negotiation failed: client connection to 127.22.151.195:45507: connect: Connection refused (error 111)
I20260924 14:32:21.856238 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:21.856855 23135 raft_consensus.cc:2258] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:21.856940 26369 raft_consensus.cc:2670] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 14:32:21.860586 23135 tablet_replica.cc:330] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
W20260924 14:32:21.861724 26369 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.861737 23135 raft_consensus.cc:2229] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.862298 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:21.863185   594 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260924 14:32:21.863224 23135 raft_consensus.cc:2258] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.867738 23135 tablet_replica.cc:330] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.868628 23135 raft_consensus.cc:2229] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.869424 23135 raft_consensus.cc:2258] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.872877 23135 tablet_replica.cc:330] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.873719 23135 raft_consensus.cc:2229] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.875252 23135 raft_consensus.cc:2258] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.877749   594 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:21.878410 23135 tablet_replica.cc:330] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.878335   594 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:21.879420 23135 raft_consensus.cc:2229] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.880936 23135 raft_consensus.cc:2258] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.884450 23135 tablet_replica.cc:330] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.885329 23135 raft_consensus.cc:2229] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Raft consensus shutting down.
W20260924 14:32:21.885756 26371 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:21.886669   594 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:21.886706 23135 raft_consensus.cc:2258] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.890213 23135 tablet_replica.cc:330] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.891225 23135 raft_consensus.cc:2229] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.892354 23135 raft_consensus.cc:2258] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:21.892598 26369 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.893301 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:21.894388   594 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:21.895712 23135 tablet_replica.cc:330] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.896713 23135 raft_consensus.cc:2229] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.897446 23135 raft_consensus.cc:2258] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.900774 23135 tablet_replica.cc:330] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.901726 23135 raft_consensus.cc:2229] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.902760 23135 raft_consensus.cc:2258] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.906111 23135 tablet_replica.cc:330] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.907032 23135 raft_consensus.cc:2229] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.908119 23135 raft_consensus.cc:2258] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.911474 23135 tablet_replica.cc:330] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.912482 23135 raft_consensus.cc:2229] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.913568 23135 raft_consensus.cc:2258] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.916882 23135 tablet_replica.cc:330] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.917820 23135 raft_consensus.cc:2229] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.920172 23135 raft_consensus.cc:2258] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.923625 23135 tablet_replica.cc:330] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.924726 23135 raft_consensus.cc:2229] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.926170 23135 raft_consensus.cc:2258] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.931248 23135 tablet_replica.cc:330] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.932232 23135 raft_consensus.cc:2229] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.933825 23135 raft_consensus.cc:2258] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.937429 23135 tablet_replica.cc:330] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.938390 23135 raft_consensus.cc:2229] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.939864 23135 raft_consensus.cc:2258] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.943142 23135 tablet_replica.cc:330] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.944069 23135 raft_consensus.cc:2229] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.945863 23135 raft_consensus.cc:2258] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.949971 23135 tablet_replica.cc:330] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.950860 23135 raft_consensus.cc:2229] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.951583 23135 raft_consensus.cc:2258] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.954906 23135 tablet_replica.cc:330] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.955772 23135 raft_consensus.cc:2229] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.956576 23135 raft_consensus.cc:2258] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.960004 23135 tablet_replica.cc:330] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.960829 23135 raft_consensus.cc:2229] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.961619 23135 raft_consensus.cc:2258] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.964763 23135 tablet_replica.cc:330] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.965588 23135 raft_consensus.cc:2229] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 21 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.966977 23135 raft_consensus.cc:2258] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.970346 23135 tablet_replica.cc:330] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.971266 23135 raft_consensus.cc:2229] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.972332 23135 raft_consensus.cc:2258] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.975617 23135 tablet_replica.cc:330] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.976464 23135 raft_consensus.cc:2229] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.977541 23135 raft_consensus.cc:2258] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.980970 23135 tablet_replica.cc:330] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.981953 23135 raft_consensus.cc:2229] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.983870 23135 raft_consensus.cc:2258] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.987504 23135 tablet_replica.cc:330] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.988330 23135 raft_consensus.cc:2229] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.989741 23135 raft_consensus.cc:2258] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.993489 23135 tablet_replica.cc:330] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.994413 23135 raft_consensus.cc:2229] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.995949 23135 raft_consensus.cc:2258] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.999215 23135 tablet_replica.cc:330] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.000106 23135 raft_consensus.cc:2229] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.001080 23135 raft_consensus.cc:2258] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.004222 23135 tablet_replica.cc:330] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.004868   594 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:22.005108 23135 raft_consensus.cc:2229] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.005508   594 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:22.006090 23135 raft_consensus.cc:2258] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.007861   594 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:22.009518 23135 tablet_replica.cc:330] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.010529 23135 raft_consensus.cc:2229] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.012122 23135 raft_consensus.cc:2258] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:22.014026 26369 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:22.015954 23135 tablet_replica.cc:330] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.016775 23135 raft_consensus.cc:2229] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.017673 23135 raft_consensus.cc:2258] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:22.019191 26371 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:22.019904 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:22.020498 23135 tablet_replica.cc:330] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
W20260924 14:32:22.020627 26371 raft_consensus.cc:2670] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 14:32:22.021274 23135 raft_consensus.cc:2229] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:22.022550 23135 raft_consensus.cc:2258] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.025625 23135 tablet_replica.cc:330] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.026288 23135 raft_consensus.cc:2229] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.027248 23135 raft_consensus.cc:2258] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.030099 23135 tablet_replica.cc:330] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.030804 23135 raft_consensus.cc:2229] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.031461 23135 raft_consensus.cc:2258] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.034339 23135 tablet_replica.cc:330] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.035002 23135 raft_consensus.cc:2229] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:22.036366 23135 raft_consensus.cc:2258] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.039244 23135 tablet_replica.cc:330] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.039966 23135 raft_consensus.cc:2229] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 26 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.040710 23135 raft_consensus.cc:2258] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.043579 23135 tablet_replica.cc:330] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.044209 23135 raft_consensus.cc:2229] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:22.045374 23135 raft_consensus.cc:2258] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:23.246433 23135 tablet_server.cc:194] TabletServer@127.22.151.196:0 shutdown complete.
I20260924 14:32:23.342757 23135 master.cc:537] Master@127.22.151.254:43789 shutting down...
I20260924 14:32:23.387421 23135 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:32:23.388386 23135 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:23.388871 23135 tablet_replica.cc:330] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: stopping tablet replica
I20260924 14:32:23.604394 23135 master.cc:552] Master@127.22.151.254:43789 shutdown complete.
I20260924 14:32:23.644793 23135 test_util.cc:175] -----------------------------------------------
I20260924 14:32:23.644984 23135 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0

[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260924 14:26:59.945029 23135 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.22.151.254:38707
I20260924 14:26:59.946480 23135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 14:26:59.947562 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:26:59.969398 23142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:26:59.969466 23144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:26:59.969400 23141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:26:59.971482 23135 server_base.cc:1034] running on GCE node
I20260924 14:27:01.206472 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:27:01.206707 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:27:01.206912 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260021206894 us; error 0 us; skew 500 ppm
I20260924 14:27:01.207959 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:27:01.213400 23135 webserver.cc:460] Webserver started at http://127.22.151.254:45707/ using document root <none> and password file <none>
I20260924 14:27:01.214280 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:27:01.214506 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:27:01.214982 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:27:01.221426 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/master-0-root/instance:
uuid: "c924d2ec9bb24ae999d74c8a3a3e3bfa"
format_stamp: "Formatted at 2026-09-24 14:27:01 on dist-test-slave-pm5x"
I20260924 14:27:01.229600 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.007s	user 0.004s	sys 0.004s
I20260924 14:27:01.235458 23150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:01.236771 23135 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.002s
I20260924 14:27:01.237195 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/master-0-root
uuid: "c924d2ec9bb24ae999d74c8a3a3e3bfa"
format_stamp: "Formatted at 2026-09-24 14:27:01 on dist-test-slave-pm5x"
I20260924 14:27:01.237586 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:27:01.276131 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:27:01.277732 23135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 14:27:01.278245 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:27:01.353873 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.254:38707
I20260924 14:27:01.353906 23201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.254:38707 every 8 connection(s)
I20260924 14:27:01.362035 23202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:01.385973 23202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa: Bootstrap starting.
I20260924 14:27:01.392481 23202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:01.394405 23202 log.cc:824] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa: Log is configured to *not* fsync() on all Append() calls
I20260924 14:27:01.399267 23202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa: No bootstrap required, opened a new log
I20260924 14:27:01.422382 23202 raft_consensus.cc:348] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c924d2ec9bb24ae999d74c8a3a3e3bfa" member_type: VOTER }
I20260924 14:27:01.422936 23202 raft_consensus.cc:374] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:01.423229 23202 raft_consensus.cc:729] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c924d2ec9bb24ae999d74c8a3a3e3bfa, State: Initialized, Role: FOLLOWER
I20260924 14:27:01.424160 23202 consensus_queue.cc:260] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c924d2ec9bb24ae999d74c8a3a3e3bfa" member_type: VOTER }
I20260924 14:27:01.424662 23202 raft_consensus.cc:388] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 14:27:01.424978 23202 raft_consensus.cc:482] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 14:27:01.425336 23202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:01.429411 23202 raft_consensus.cc:504] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c924d2ec9bb24ae999d74c8a3a3e3bfa" member_type: VOTER }
I20260924 14:27:01.430166 23202 leader_election.cc:302] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c924d2ec9bb24ae999d74c8a3a3e3bfa; no voters: 
I20260924 14:27:01.431974 23202 leader_election.cc:288] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260924 14:27:01.432335 23205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:01.435292 23205 raft_consensus.cc:686] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 1 LEADER]: Becoming Leader. State: Replica: c924d2ec9bb24ae999d74c8a3a3e3bfa, State: Running, Role: LEADER
I20260924 14:27:01.436446 23205 consensus_queue.cc:237] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c924d2ec9bb24ae999d74c8a3a3e3bfa" member_type: VOTER }
I20260924 14:27:01.440395 23202 sys_catalog.cc:564] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [sys.catalog]: configured and running, proceeding with master startup.
I20260924 14:27:01.449622 23207 sys_catalog.cc:455] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c924d2ec9bb24ae999d74c8a3a3e3bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c924d2ec9bb24ae999d74c8a3a3e3bfa" member_type: VOTER } }
I20260924 14:27:01.450614 23207 sys_catalog.cc:458] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [sys.catalog]: This master's current role is: LEADER
I20260924 14:27:01.452339 23206 sys_catalog.cc:455] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader c924d2ec9bb24ae999d74c8a3a3e3bfa. Latest consensus state: current_term: 1 leader_uuid: "c924d2ec9bb24ae999d74c8a3a3e3bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c924d2ec9bb24ae999d74c8a3a3e3bfa" member_type: VOTER } }
I20260924 14:27:01.453104 23206 sys_catalog.cc:458] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [sys.catalog]: This master's current role is: LEADER
I20260924 14:27:01.455999 23215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 14:27:01.469717 23215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 14:27:01.471954 23135 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 14:27:01.488423 23215 catalog_manager.cc:1347] Generated new cluster ID: 35467842181a4adeb67a028e8234c37d
I20260924 14:27:01.488773 23215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 14:27:01.512243 23215 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 14:27:01.514300 23215 catalog_manager.cc:1504] Loading token signing keys...
I20260924 14:27:01.532579 23215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa: Generated new TSK 0
I20260924 14:27:01.533692 23215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 14:27:01.538916 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:27:01.547957 23224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:01.550073 23225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:01.554011 23227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:27:01.555824 23135 server_base.cc:1034] running on GCE node
I20260924 14:27:01.556658 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:27:01.556840 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:27:01.557058 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260021557027 us; error 0 us; skew 500 ppm
I20260924 14:27:01.557720 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:27:01.563057 23135 webserver.cc:460] Webserver started at http://127.22.151.193:46723/ using document root <none> and password file <none>
I20260924 14:27:01.563720 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:27:01.563903 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:27:01.564253 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:27:01.565584 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-0-root/instance:
uuid: "78d920610cdd4a2bbe2dde49988fbb11"
format_stamp: "Formatted at 2026-09-24 14:27:01 on dist-test-slave-pm5x"
I20260924 14:27:01.572220 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.002s
I20260924 14:27:01.577762 23232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:01.579631 23135 fs_manager.cc:725] Time spent opening block manager: real 0.005s	user 0.002s	sys 0.003s
I20260924 14:27:01.579967 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-0-root
uuid: "78d920610cdd4a2bbe2dde49988fbb11"
format_stamp: "Formatted at 2026-09-24 14:27:01 on dist-test-slave-pm5x"
I20260924 14:27:01.580291 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:27:01.622997 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:27:01.624574 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:27:01.627622 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:27:01.632591 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:27:01.632833 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:01.633178 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:27:01.633333 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:01.749713 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.193:35195
I20260924 14:27:01.749850 23294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.193:35195 every 8 connection(s)
I20260924 14:27:01.758041 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:27:01.769649 23299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:01.774626 23300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:27:01.788916 23295 heartbeater.cc:347] Connected to a master server at 127.22.151.254:38707
I20260924 14:27:01.789424 23295 heartbeater.cc:464] Registering TS with master...
I20260924 14:27:01.790584 23295 heartbeater.cc:511] Master 127.22.151.254:38707 requested a full tablet report, sending...
I20260924 14:27:01.795017 23167 ts_manager.cc:194] Registered new tserver with Master: 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195)
I20260924 14:27:01.798820 23167 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47320
W20260924 14:27:01.806922 23302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:27:01.807130 23135 server_base.cc:1034] running on GCE node
I20260924 14:27:01.808295 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:27:01.808557 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:27:01.808758 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260021808743 us; error 0 us; skew 500 ppm
I20260924 14:27:01.809521 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:27:01.812585 23135 webserver.cc:460] Webserver started at http://127.22.151.194:39453/ using document root <none> and password file <none>
I20260924 14:27:01.813206 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:27:01.813431 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:27:01.813742 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:27:01.815152 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-1-root/instance:
uuid: "278ba036c8a04b8ba9f143ffdeef6707"
format_stamp: "Formatted at 2026-09-24 14:27:01 on dist-test-slave-pm5x"
I20260924 14:27:01.821694 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:27:01.826735 23307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:01.827682 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260924 14:27:01.828020 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-1-root
uuid: "278ba036c8a04b8ba9f143ffdeef6707"
format_stamp: "Formatted at 2026-09-24 14:27:01 on dist-test-slave-pm5x"
I20260924 14:27:01.828393 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:27:01.869010 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:27:01.870692 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:27:01.873386 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:27:01.877270 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:27:01.877562 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:01.877864 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:27:01.878077 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:01.970136 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.194:33887
I20260924 14:27:01.970216 23369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.194:33887 every 8 connection(s)
I20260924 14:27:01.976358 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:27:01.986055 23373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:01.987671 23374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:27:01.992209 23370 heartbeater.cc:347] Connected to a master server at 127.22.151.254:38707
I20260924 14:27:01.992727 23370 heartbeater.cc:464] Registering TS with master...
I20260924 14:27:01.993757 23370 heartbeater.cc:511] Master 127.22.151.254:38707 requested a full tablet report, sending...
W20260924 14:27:01.994372 23376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:27:01.995373 23135 server_base.cc:1034] running on GCE node
I20260924 14:27:01.996273 23167 ts_manager.cc:194] Registered new tserver with Master: 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887)
I20260924 14:27:01.996505 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:27:01.996768 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:27:01.996974 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260021996961 us; error 0 us; skew 500 ppm
I20260924 14:27:01.997673 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:27:01.997946 23167 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47332
I20260924 14:27:02.000764 23135 webserver.cc:460] Webserver started at http://127.22.151.195:44389/ using document root <none> and password file <none>
I20260924 14:27:02.001408 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:27:02.001643 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:27:02.001947 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:27:02.003368 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-2-root/instance:
uuid: "2fc7e85a970e4997926a2f2b13bfd562"
format_stamp: "Formatted at 2026-09-24 14:27:01 on dist-test-slave-pm5x"
I20260924 14:27:02.009395 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260924 14:27:02.013578 23381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:02.014432 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260924 14:27:02.014756 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-2-root
uuid: "2fc7e85a970e4997926a2f2b13bfd562"
format_stamp: "Formatted at 2026-09-24 14:27:01 on dist-test-slave-pm5x"
I20260924 14:27:02.015115 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1790260019751168-23135-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:27:02.064745 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:27:02.066030 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:27:02.067512 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:27:02.069621 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:27:02.069789 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:02.070088 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:27:02.070225 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:02.167327 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.195:43155
I20260924 14:27:02.167960 23443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.195:43155 every 8 connection(s)
I20260924 14:27:02.187186 23444 heartbeater.cc:347] Connected to a master server at 127.22.151.254:38707
I20260924 14:27:02.187659 23444 heartbeater.cc:464] Registering TS with master...
I20260924 14:27:02.188572 23444 heartbeater.cc:511] Master 127.22.151.254:38707 requested a full tablet report, sending...
I20260924 14:27:02.190769 23167 ts_manager.cc:194] Registered new tserver with Master: 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:02.191310 23135 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018422285s
I20260924 14:27:02.192948 23167 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47344
I20260924 14:27:02.195319 23135 test_util.cc:268] Using random seed: 1689105502
I20260924 14:27:02.256309 23167 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43208:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260924 14:27:02.260571 23167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 14:27:02.368989 23334 tablet_service.cc:1467] Processing CreateTablet for tablet ca20c5eab35d4b9a8f3945f4e9bf5815 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:27:02.370800 23334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca20c5eab35d4b9a8f3945f4e9bf5815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.371711 23335 tablet_service.cc:1467] Processing CreateTablet for tablet aa058204bc8d43e8893ef87079f0e38a (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:27:02.373045 23335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa058204bc8d43e8893ef87079f0e38a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.380615 23333 tablet_service.cc:1467] Processing CreateTablet for tablet fd806eef81974e49811ad08d74410fdb (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:27:02.382110 23333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd806eef81974e49811ad08d74410fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.388420 23332 tablet_service.cc:1467] Processing CreateTablet for tablet 496d8e256cf2413f8bf270323313d891 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:27:02.389832 23332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 496d8e256cf2413f8bf270323313d891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.400806 23331 tablet_service.cc:1467] Processing CreateTablet for tablet 6160343c1d3e4920801e9a06b7355f1f (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:27:02.402269 23331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6160343c1d3e4920801e9a06b7355f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.403818 23409 tablet_service.cc:1467] Processing CreateTablet for tablet aa058204bc8d43e8893ef87079f0e38a (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:27:02.405174 23409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa058204bc8d43e8893ef87079f0e38a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.408730 23408 tablet_service.cc:1467] Processing CreateTablet for tablet ca20c5eab35d4b9a8f3945f4e9bf5815 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:27:02.410784 23408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca20c5eab35d4b9a8f3945f4e9bf5815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.410483 23330 tablet_service.cc:1467] Processing CreateTablet for tablet ed1c48c6a1254b5685577a524c0ca523 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:27:02.422565 23404 tablet_service.cc:1467] Processing CreateTablet for tablet ed1c48c6a1254b5685577a524c0ca523 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:27:02.430635 23403 tablet_service.cc:1467] Processing CreateTablet for tablet 5b9cfc0424a8429ebd5886060eff5bde (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:27:02.441150 23404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed1c48c6a1254b5685577a524c0ca523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.443414 23330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed1c48c6a1254b5685577a524c0ca523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.452391 23334 tablet_service.cc:1467] Processing CreateTablet for tablet bea8bfb3513f4b5885d3e801189e2848 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:27:02.440649 23329 tablet_service.cc:1467] Processing CreateTablet for tablet 5b9cfc0424a8429ebd5886060eff5bde (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:27:02.443817 23407 tablet_service.cc:1467] Processing CreateTablet for tablet fd806eef81974e49811ad08d74410fdb (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:27:02.432255 23260 tablet_service.cc:1467] Processing CreateTablet for tablet aa058204bc8d43e8893ef87079f0e38a (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:27:02.447580 23406 tablet_service.cc:1467] Processing CreateTablet for tablet 496d8e256cf2413f8bf270323313d891 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:27:02.458966 23403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b9cfc0424a8429ebd5886060eff5bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.443945 23405 tablet_service.cc:1467] Processing CreateTablet for tablet 6160343c1d3e4920801e9a06b7355f1f (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:27:02.447804 23258 tablet_service.cc:1467] Processing CreateTablet for tablet fd806eef81974e49811ad08d74410fdb (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:27:02.472808 23329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b9cfc0424a8429ebd5886060eff5bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.466714 23256 tablet_service.cc:1467] Processing CreateTablet for tablet 6160343c1d3e4920801e9a06b7355f1f (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:27:02.460544 23259 tablet_service.cc:1467] Processing CreateTablet for tablet ca20c5eab35d4b9a8f3945f4e9bf5815 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:27:02.478178 23255 tablet_service.cc:1467] Processing CreateTablet for tablet ed1c48c6a1254b5685577a524c0ca523 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:27:02.476305 23334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bea8bfb3513f4b5885d3e801189e2848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.471599 23257 tablet_service.cc:1467] Processing CreateTablet for tablet 496d8e256cf2413f8bf270323313d891 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:27:02.482511 23406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 496d8e256cf2413f8bf270323313d891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.485841 23402 tablet_service.cc:1467] Processing CreateTablet for tablet bea8bfb3513f4b5885d3e801189e2848 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
W20260924 14:27:02.483072 23214 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:27:02.490152 23254 tablet_service.cc:1467] Processing CreateTablet for tablet 5b9cfc0424a8429ebd5886060eff5bde (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:27:02.478972 23335 tablet_service.cc:1467] Processing CreateTablet for tablet 884a631342f040a2b7babf614155f1a5 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:27:02.485308 23462 tablet_bootstrap.cc:492] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707: Bootstrap starting.
I20260924 14:27:02.494774 23256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6160343c1d3e4920801e9a06b7355f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.499724 23405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6160343c1d3e4920801e9a06b7355f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.497197 23335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 884a631342f040a2b7babf614155f1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.503486 23253 tablet_service.cc:1467] Processing CreateTablet for tablet bea8bfb3513f4b5885d3e801189e2848 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:27:02.516646 23257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 496d8e256cf2413f8bf270323313d891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.521431 23402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bea8bfb3513f4b5885d3e801189e2848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.498090 23401 tablet_service.cc:1467] Processing CreateTablet for tablet 884a631342f040a2b7babf614155f1a5 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:27:02.534888 23255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed1c48c6a1254b5685577a524c0ca523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.541596 23401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 884a631342f040a2b7babf614155f1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.521478 23252 tablet_service.cc:1467] Processing CreateTablet for tablet 884a631342f040a2b7babf614155f1a5 (DEFAULT_TABLE table=test-workload [id=8585656b614846359acae8f7a97faf21]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:27:02.578984 23258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd806eef81974e49811ad08d74410fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.580070 23252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 884a631342f040a2b7babf614155f1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.581066 23253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bea8bfb3513f4b5885d3e801189e2848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.582176 23260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa058204bc8d43e8893ef87079f0e38a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.586144 23259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca20c5eab35d4b9a8f3945f4e9bf5815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.591805 23254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b9cfc0424a8429ebd5886060eff5bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.596701 23407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd806eef81974e49811ad08d74410fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:02.737905 23462 tablet_bootstrap.cc:654] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:02.775820 23464 tablet_bootstrap.cc:492] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562: Bootstrap starting.
I20260924 14:27:02.778151 23463 tablet_bootstrap.cc:492] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11: Bootstrap starting.
I20260924 14:27:02.795941 23462 tablet_bootstrap.cc:492] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707: No bootstrap required, opened a new log
I20260924 14:27:02.796576 23462 ts_tablet_manager.cc:1397] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707: Time spent bootstrapping tablet: real 0.320s	user 0.013s	sys 0.013s
I20260924 14:27:02.797369 23463 tablet_bootstrap.cc:654] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:02.801741 23462 raft_consensus.cc:348] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.802923 23462 raft_consensus.cc:374] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:02.803272 23464 tablet_bootstrap.cc:654] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:02.803233 23462 raft_consensus.cc:729] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Initialized, Role: FOLLOWER
I20260924 14:27:02.803825 23295 heartbeater.cc:503] Master 127.22.151.254:38707 was elected leader, sending a full tablet report...
I20260924 14:27:02.804177 23462 consensus_queue.cc:260] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.815376 23370 heartbeater.cc:503] Master 127.22.151.254:38707 was elected leader, sending a full tablet report...
I20260924 14:27:02.816501 23462 ts_tablet_manager.cc:1428] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707: Time spent starting tablet: real 0.020s	user 0.011s	sys 0.001s
I20260924 14:27:02.817559 23462 tablet_bootstrap.cc:492] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707: Bootstrap starting.
I20260924 14:27:02.818516 23464 tablet_bootstrap.cc:492] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562: No bootstrap required, opened a new log
I20260924 14:27:02.818951 23464 ts_tablet_manager.cc:1397] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562: Time spent bootstrapping tablet: real 0.053s	user 0.004s	sys 0.015s
I20260924 14:27:02.824693 23462 tablet_bootstrap.cc:654] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:02.825052 23464 raft_consensus.cc:348] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.827689 23464 raft_consensus.cc:374] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:02.828003 23464 raft_consensus.cc:729] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Initialized, Role: FOLLOWER
I20260924 14:27:02.828781 23464 consensus_queue.cc:260] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.832085 23463 tablet_bootstrap.cc:492] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11: No bootstrap required, opened a new log
I20260924 14:27:02.832708 23463 ts_tablet_manager.cc:1397] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.009s
I20260924 14:27:02.835927 23462 tablet_bootstrap.cc:492] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707: No bootstrap required, opened a new log
I20260924 14:27:02.836426 23462 ts_tablet_manager.cc:1397] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.009s
I20260924 14:27:02.836714 23463 raft_consensus.cc:348] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.837643 23463 raft_consensus.cc:374] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:02.838022 23463 raft_consensus.cc:729] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Initialized, Role: FOLLOWER
I20260924 14:27:02.838794 23463 consensus_queue.cc:260] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.839316 23462 raft_consensus.cc:348] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.840137 23462 raft_consensus.cc:374] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:02.840544 23462 raft_consensus.cc:729] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Initialized, Role: FOLLOWER
I20260924 14:27:02.841454 23462 consensus_queue.cc:260] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.845808 23462 ts_tablet_manager.cc:1428] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260924 14:27:02.847040 23462 tablet_bootstrap.cc:492] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707: Bootstrap starting.
I20260924 14:27:02.854019 23464 ts_tablet_manager.cc:1428] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562: Time spent starting tablet: real 0.035s	user 0.000s	sys 0.008s
I20260924 14:27:02.855202 23444 heartbeater.cc:503] Master 127.22.151.254:38707 was elected leader, sending a full tablet report...
I20260924 14:27:02.855643 23464 tablet_bootstrap.cc:492] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562: Bootstrap starting.
I20260924 14:27:02.856436 23463 ts_tablet_manager.cc:1428] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11: Time spent starting tablet: real 0.023s	user 0.015s	sys 0.005s
I20260924 14:27:02.864224 23463 tablet_bootstrap.cc:492] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11: Bootstrap starting.
I20260924 14:27:02.864614 23464 tablet_bootstrap.cc:654] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:02.869897 23462 tablet_bootstrap.cc:654] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:02.881884 23463 tablet_bootstrap.cc:654] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:02.882911 23467 raft_consensus.cc:482] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:02.883492 23467 raft_consensus.cc:504] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.887053 23467 leader_election.cc:288] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:02.890213 23462 tablet_bootstrap.cc:492] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707: No bootstrap required, opened a new log
I20260924 14:27:02.895817 23462 ts_tablet_manager.cc:1397] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.005s
I20260924 14:27:02.896907 23464 tablet_bootstrap.cc:492] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562: No bootstrap required, opened a new log
I20260924 14:27:02.897598 23464 ts_tablet_manager.cc:1397] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.009s
I20260924 14:27:02.898926 23463 tablet_bootstrap.cc:492] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11: No bootstrap required, opened a new log
I20260924 14:27:02.898692 23462 raft_consensus.cc:348] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.901028 23462 raft_consensus.cc:374] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:02.900897 23464 raft_consensus.cc:348] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.901418 23462 raft_consensus.cc:729] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Initialized, Role: FOLLOWER
I20260924 14:27:02.901731 23464 raft_consensus.cc:374] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:02.902077 23464 raft_consensus.cc:729] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Initialized, Role: FOLLOWER
I20260924 14:27:02.899480 23463 ts_tablet_manager.cc:1397] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.002s
I20260924 14:27:02.902206 23462 consensus_queue.cc:260] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.905185 23464 consensus_queue.cc:260] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.907416 23464 ts_tablet_manager.cc:1428] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260924 14:27:02.908265 23464 tablet_bootstrap.cc:492] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562: Bootstrap starting.
I20260924 14:27:02.912983 23462 ts_tablet_manager.cc:1428] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260924 14:27:02.913400 23463 raft_consensus.cc:348] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.914300 23463 raft_consensus.cc:374] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:02.914518 23464 tablet_bootstrap.cc:654] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:02.914649 23463 raft_consensus.cc:729] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Initialized, Role: FOLLOWER
I20260924 14:27:02.915674 23462 tablet_bootstrap.cc:492] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707: Bootstrap starting.
W20260924 14:27:02.925077 23445 tablet.cc:2369] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 14:27:02.930202 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11" is_pre_election: true
W20260924 14:27:02.932308 23310 leader_election.cc:341] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:02.935256 23464 tablet_bootstrap.cc:492] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562: No bootstrap required, opened a new log
I20260924 14:27:02.935730 23464 ts_tablet_manager.cc:1397] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562: Time spent bootstrapping tablet: real 0.028s	user 0.002s	sys 0.015s
I20260924 14:27:02.915525 23463 consensus_queue.cc:260] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.940769 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:02.941205 23462 tablet_bootstrap.cc:654] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:02.941617 23419 raft_consensus.cc:2454] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 0.
I20260924 14:27:02.943015 23311 leader_election.cc:302] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562; no voters: 78d920610cdd4a2bbe2dde49988fbb11
I20260924 14:27:02.943892 23463 ts_tablet_manager.cc:1428] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.001s
I20260924 14:27:02.943854 23467 raft_consensus.cc:2781] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:02.944295 23467 raft_consensus.cc:482] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:02.944628 23467 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:02.946552 23464 raft_consensus.cc:348] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.947481 23464 raft_consensus.cc:374] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:02.947826 23464 raft_consensus.cc:729] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Initialized, Role: FOLLOWER
I20260924 14:27:02.948673 23464 consensus_queue.cc:260] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.952884 23467 raft_consensus.cc:504] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.953608 23464 ts_tablet_manager.cc:1428] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.004s
I20260924 14:27:02.954629 23464 tablet_bootstrap.cc:492] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562: Bootstrap starting.
I20260924 14:27:02.959038 23470 raft_consensus.cc:482] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:02.959549 23470 raft_consensus.cc:504] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.960716 23463 tablet_bootstrap.cc:492] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11: Bootstrap starting.
I20260924 14:27:02.962143 23470 leader_election.cc:288] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:02.965708 23467 leader_election.cc:288] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:02.966512 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:02.966903 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:02.967590 23419 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:27:02.967998 23310 leader_election.cc:341] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:02.969470 23463 tablet_bootstrap.cc:654] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:02.976609 23462 tablet_bootstrap.cc:492] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707: No bootstrap required, opened a new log
I20260924 14:27:02.977725 23462 ts_tablet_manager.cc:1397] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707: Time spent bootstrapping tablet: real 0.062s	user 0.019s	sys 0.008s
I20260924 14:27:02.978340 23419 raft_consensus.cc:2454] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 1.
I20260924 14:27:02.979876 23311 leader_election.cc:302] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562; no voters: 78d920610cdd4a2bbe2dde49988fbb11
I20260924 14:27:02.980760 23467 raft_consensus.cc:2781] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:02.982705 23462 raft_consensus.cc:348] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.983338 23471 raft_consensus.cc:482] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:02.983741 23462 raft_consensus.cc:374] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:02.984193 23462 raft_consensus.cc:729] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Initialized, Role: FOLLOWER
I20260924 14:27:02.983843 23471 raft_consensus.cc:504] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.985092 23462 consensus_queue.cc:260] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:02.987094 23471 leader_election.cc:288] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887)
I20260924 14:27:02.996690 23464 tablet_bootstrap.cc:654] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.010341 23463 tablet_bootstrap.cc:492] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11: No bootstrap required, opened a new log
I20260924 14:27:03.010574 23464 tablet_bootstrap.cc:492] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562: No bootstrap required, opened a new log
I20260924 14:27:03.010928 23463 ts_tablet_manager.cc:1397] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.001s
I20260924 14:27:03.011700 23464 ts_tablet_manager.cc:1397] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562: Time spent bootstrapping tablet: real 0.057s	user 0.010s	sys 0.004s
I20260924 14:27:03.013695 23463 raft_consensus.cc:348] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.014602 23463 raft_consensus.cc:374] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.014977 23463 raft_consensus.cc:729] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.015898 23467 raft_consensus.cc:686] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:03.014402 23464 raft_consensus.cc:348] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.017518 23464 raft_consensus.cc:374] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.017946 23464 raft_consensus.cc:729] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.017899 23467 consensus_queue.cc:237] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.019186 23464 consensus_queue.cc:260] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.024379 23462 ts_tablet_manager.cc:1428] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707: Time spent starting tablet: real 0.046s	user 0.011s	sys 0.000s
I20260924 14:27:03.025532 23462 tablet_bootstrap.cc:492] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707: Bootstrap starting.
I20260924 14:27:03.028283 23464 ts_tablet_manager.cc:1428] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.001s
I20260924 14:27:03.027722 23463 consensus_queue.cc:260] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.029690 23464 tablet_bootstrap.cc:492] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562: Bootstrap starting.
I20260924 14:27:03.033957 23470 raft_consensus.cc:482] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.034415 23470 raft_consensus.cc:504] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.038769 23470 leader_election.cc:288] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.042097 23462 tablet_bootstrap.cc:654] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.034816 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884a631342f040a2b7babf614155f1a5" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707" is_pre_election: true
I20260924 14:27:03.048440 23463 ts_tablet_manager.cc:1428] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.004s
I20260924 14:27:03.047421 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496d8e256cf2413f8bf270323313d891" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707" is_pre_election: true
I20260924 14:27:03.050976 23344 raft_consensus.cc:2454] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 0.
W20260924 14:27:03.052560 23234 leader_election.cc:341] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:03.052881 23463 tablet_bootstrap.cc:492] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11: Bootstrap starting.
I20260924 14:27:03.053555 23234 leader_election.cc:302] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:03.064136 23464 tablet_bootstrap.cc:654] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.071377 23463 tablet_bootstrap.cc:654] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.077824 23490 raft_consensus.cc:2781] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:03.078303 23490 raft_consensus.cc:482] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:03.081097 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707" is_pre_election: true
I20260924 14:27:03.081888 23344 raft_consensus.cc:2454] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 0.
I20260924 14:27:03.080255 23490 raft_consensus.cc:3037] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.072516 23165 catalog_manager.cc:5495] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 0 to 1, leader changed from <none> to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 1 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:03.087268 23383 leader_election.cc:302] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562; no voters: 
I20260924 14:27:03.092494 23471 raft_consensus.cc:2781] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:03.092962 23471 raft_consensus.cc:482] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:03.093389 23471 raft_consensus.cc:3037] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.098379 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496d8e256cf2413f8bf270323313d891" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.098098 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884a631342f040a2b7babf614155f1a5" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.099546 23418 raft_consensus.cc:2454] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 0.
W20260924 14:27:03.101938 23236 leader_election.cc:341] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:03.102641 23462 tablet_bootstrap.cc:492] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707: No bootstrap required, opened a new log
I20260924 14:27:03.102366 23236 leader_election.cc:302] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78d920610cdd4a2bbe2dde49988fbb11; no voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:03.103859 23470 raft_consensus.cc:2726] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:03.103583 23462 ts_tablet_manager.cc:1397] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707: Time spent bootstrapping tablet: real 0.078s	user 0.016s	sys 0.016s
I20260924 14:27:03.111269 23464 tablet_bootstrap.cc:492] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562: No bootstrap required, opened a new log
I20260924 14:27:03.112553 23471 raft_consensus.cc:504] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.113222 23464 ts_tablet_manager.cc:1397] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562: Time spent bootstrapping tablet: real 0.084s	user 0.015s	sys 0.003s
I20260924 14:27:03.113329 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11" is_pre_election: true
I20260924 14:27:03.115253 23471 leader_election.cc:288] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 1 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887)
W20260924 14:27:03.115338 23384 leader_election.cc:341] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:03.117163 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:03.117344 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:03.117975 23344 raft_consensus.cc:3037] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.110169 23490 raft_consensus.cc:504] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.116623 23464 raft_consensus.cc:348] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.119809 23464 raft_consensus.cc:374] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260924 14:27:03.120383 23384 leader_election.cc:341] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:03.112442 23462 raft_consensus.cc:348] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.121940 23462 raft_consensus.cc:374] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.120205 23464 raft_consensus.cc:729] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.123679 23462 raft_consensus.cc:729] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.125583 23462 consensus_queue.cc:260] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.126511 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496d8e256cf2413f8bf270323313d891" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:03.127210 23345 raft_consensus.cc:3037] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.127054 23464 consensus_queue.cc:260] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.132423 23462 ts_tablet_manager.cc:1428] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260924 14:27:03.132627 23463 tablet_bootstrap.cc:492] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11: No bootstrap required, opened a new log
I20260924 14:27:03.131831 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496d8e256cf2413f8bf270323313d891" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:03.133183 23463 ts_tablet_manager.cc:1397] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11: Time spent bootstrapping tablet: real 0.081s	user 0.013s	sys 0.002s
I20260924 14:27:03.133383 23462 tablet_bootstrap.cc:492] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707: Bootstrap starting.
I20260924 14:27:03.133283 23418 raft_consensus.cc:3037] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.130100 23490 leader_election.cc:288] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.132627 23344 raft_consensus.cc:2454] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 1.
I20260924 14:27:03.137099 23383 leader_election.cc:302] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562; no voters: 78d920610cdd4a2bbe2dde49988fbb11
I20260924 14:27:03.139792 23471 raft_consensus.cc:2781] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:03.139967 23464 ts_tablet_manager.cc:1428] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260924 14:27:03.140328 23463 raft_consensus.cc:348] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.143097 23464 tablet_bootstrap.cc:492] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562: Bootstrap starting.
I20260924 14:27:03.143241 23463 raft_consensus.cc:374] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.145857 23463 raft_consensus.cc:729] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.146617 23463 consensus_queue.cc:260] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.151648 23463 ts_tablet_manager.cc:1428] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260924 14:27:03.152581 23463 tablet_bootstrap.cc:492] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11: Bootstrap starting.
I20260924 14:27:03.153236 23418 raft_consensus.cc:2454] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 1.
I20260924 14:27:03.153995 23471 raft_consensus.cc:686] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 LEADER]: Becoming Leader. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Running, Role: LEADER
I20260924 14:27:03.155249 23236 leader_election.cc:302] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:03.157272 23490 raft_consensus.cc:2781] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:03.157094 23471 consensus_queue.cc:237] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.157750 23490 raft_consensus.cc:686] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Becoming Leader. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:03.158555 23490 consensus_queue.cc:237] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.164029 23464 tablet_bootstrap.cc:654] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.164556 23470 raft_consensus.cc:482] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.165040 23470 raft_consensus.cc:504] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.165676 23345 raft_consensus.cc:2454] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 1.
I20260924 14:27:03.167052 23470 leader_election.cc:288] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.168416 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9cfc0424a8429ebd5886060eff5bde" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707" is_pre_election: true
I20260924 14:27:03.168648 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9cfc0424a8429ebd5886060eff5bde" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.169059 23345 raft_consensus.cc:2454] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 0.
W20260924 14:27:03.170325 23236 leader_election.cc:341] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:03.171928 23236 leader_election.cc:302] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:03.172717 23470 raft_consensus.cc:2781] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:03.172941 23462 tablet_bootstrap.cc:654] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.173233 23470 raft_consensus.cc:482] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:03.173677 23470 raft_consensus.cc:3037] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.211366 23463 tablet_bootstrap.cc:654] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.221263 23490 raft_consensus.cc:482] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.221944 23490 raft_consensus.cc:504] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.226295 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd806eef81974e49811ad08d74410fdb" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.226922 23490 leader_election.cc:288] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.229022 23418 raft_consensus.cc:2454] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 0.
I20260924 14:27:03.228343 23470 raft_consensus.cc:504] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.232770 23236 leader_election.cc:302] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:03.233769 23470 leader_election.cc:288] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.234330 23464 tablet_bootstrap.cc:492] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562: No bootstrap required, opened a new log
I20260924 14:27:03.234301 23470 raft_consensus.cc:2781] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:03.234818 23464 ts_tablet_manager.cc:1397] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562: Time spent bootstrapping tablet: real 0.092s	user 0.023s	sys 0.015s
I20260924 14:27:03.234735 23470 raft_consensus.cc:482] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:03.234894 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9cfc0424a8429ebd5886060eff5bde" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:03.235112 23470 raft_consensus.cc:3037] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.238613 23463 tablet_bootstrap.cc:492] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11: No bootstrap required, opened a new log
I20260924 14:27:03.238247 23464 raft_consensus.cc:348] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.239167 23463 ts_tablet_manager.cc:1397] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11: Time spent bootstrapping tablet: real 0.087s	user 0.012s	sys 0.005s
I20260924 14:27:03.239316 23464 raft_consensus.cc:374] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.239779 23464 raft_consensus.cc:729] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.240561 23464 consensus_queue.cc:260] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.240891 23462 tablet_bootstrap.cc:492] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707: No bootstrap required, opened a new log
I20260924 14:27:03.242610 23462 ts_tablet_manager.cc:1397] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707: Time spent bootstrapping tablet: real 0.109s	user 0.009s	sys 0.016s
I20260924 14:27:03.242504 23463 raft_consensus.cc:348] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.243131 23418 raft_consensus.cc:3037] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.243697 23464 ts_tablet_manager.cc:1428] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
W20260924 14:27:03.240892 23234 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:33887, user_credentials={real_user=slave}} blocked reactor thread for 70453.2us
I20260924 14:27:03.244925 23464 tablet_bootstrap.cc:492] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562: Bootstrap starting.
I20260924 14:27:03.246318 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd806eef81974e49811ad08d74410fdb" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707" is_pre_election: true
I20260924 14:27:03.246948 23345 raft_consensus.cc:2454] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 0.
I20260924 14:27:03.249891 23418 raft_consensus.cc:2454] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 1.
I20260924 14:27:03.251184 23236 leader_election.cc:302] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:03.252352 23502 raft_consensus.cc:2781] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:03.252920 23502 raft_consensus.cc:686] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Becoming Leader. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:03.254109 23502 consensus_queue.cc:237] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.261909 23462 raft_consensus.cc:348] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.262756 23462 raft_consensus.cc:374] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.263085 23462 raft_consensus.cc:729] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.264473 23464 tablet_bootstrap.cc:654] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.271296 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9cfc0424a8429ebd5886060eff5bde" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:03.273722 23344 raft_consensus.cc:3037] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.262971 23166 catalog_manager.cc:5495] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:03.278625 23462 consensus_queue.cc:260] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.281955 23462 ts_tablet_manager.cc:1428] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260924 14:27:03.283174 23462 tablet_bootstrap.cc:492] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707: Bootstrap starting.
I20260924 14:27:03.285243 23344 raft_consensus.cc:2454] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 1.
I20260924 14:27:03.272280 23470 raft_consensus.cc:504] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.243299 23463 raft_consensus.cc:374] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.286023 23463 raft_consensus.cc:729] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.287014 23463 consensus_queue.cc:260] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.288499 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd806eef81974e49811ad08d74410fdb" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:03.298818 23418 raft_consensus.cc:3037] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.288242 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd806eef81974e49811ad08d74410fdb" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:03.290120 23164 catalog_manager.cc:5495] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:03.299944 23345 raft_consensus.cc:3037] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.300540 23463 ts_tablet_manager.cc:1428] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260924 14:27:03.301539 23463 tablet_bootstrap.cc:492] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11: Bootstrap starting.
I20260924 14:27:03.287528 23470 leader_election.cc:288] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.297187 23462 tablet_bootstrap.cc:654] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.306486 23345 raft_consensus.cc:2454] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 1.
I20260924 14:27:03.307768 23234 leader_election.cc:302] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:03.308380 23463 tablet_bootstrap.cc:654] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.308595 23470 raft_consensus.cc:2781] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:03.309119 23470 raft_consensus.cc:686] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Becoming Leader. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:03.309888 23470 consensus_queue.cc:237] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.312676 23167 catalog_manager.cc:5495] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:03.313867 23464 tablet_bootstrap.cc:492] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562: No bootstrap required, opened a new log
I20260924 14:27:03.325748 23464 ts_tablet_manager.cc:1397] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562: Time spent bootstrapping tablet: real 0.081s	user 0.016s	sys 0.019s
I20260924 14:27:03.333210 23464 raft_consensus.cc:348] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.335318 23418 raft_consensus.cc:2454] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 1.
I20260924 14:27:03.334499 23464 raft_consensus.cc:374] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.339493 23464 raft_consensus.cc:729] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.340193 23464 consensus_queue.cc:260] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.344971 23464 ts_tablet_manager.cc:1428] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.003s
I20260924 14:27:03.345953 23464 tablet_bootstrap.cc:492] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562: Bootstrap starting.
I20260924 14:27:03.350371 23462 tablet_bootstrap.cc:492] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707: No bootstrap required, opened a new log
I20260924 14:27:03.350798 23462 ts_tablet_manager.cc:1397] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.006s
I20260924 14:27:03.353540 23462 raft_consensus.cc:348] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.354346 23462 raft_consensus.cc:374] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.354629 23462 raft_consensus.cc:729] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.355333 23462 consensus_queue.cc:260] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.357996 23462 ts_tablet_manager.cc:1428] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:27:03.358549 23463 tablet_bootstrap.cc:492] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11: No bootstrap required, opened a new log
I20260924 14:27:03.358879 23462 tablet_bootstrap.cc:492] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707: Bootstrap starting.
I20260924 14:27:03.359117 23463 ts_tablet_manager.cc:1397] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.027s
I20260924 14:27:03.357852 23165 catalog_manager.cc:5495] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:03.365182 23463 raft_consensus.cc:348] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.365922 23463 raft_consensus.cc:374] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.366199 23463 raft_consensus.cc:729] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.367031 23463 consensus_queue.cc:260] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.367977 23462 tablet_bootstrap.cc:654] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.369325 23463 ts_tablet_manager.cc:1428] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260924 14:27:03.370217 23463 tablet_bootstrap.cc:492] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11: Bootstrap starting.
I20260924 14:27:03.371405 23502 raft_consensus.cc:482] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.371883 23502 raft_consensus.cc:504] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.373868 23502 leader_election.cc:288] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.374888 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707" is_pre_election: true
I20260924 14:27:03.376413 23467 raft_consensus.cc:482] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.376477 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.377125 23418 raft_consensus.cc:2454] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 0.
I20260924 14:27:03.376930 23467 raft_consensus.cc:504] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.378432 23236 leader_election.cc:302] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:03.378942 23467 leader_election.cc:288] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
W20260924 14:27:03.379617 23234 leader_election.cc:341] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:03.380260 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea8bfb3513f4b5885d3e801189e2848" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.380877 23418 raft_consensus.cc:2454] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 0.
I20260924 14:27:03.381469 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea8bfb3513f4b5885d3e801189e2848" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11" is_pre_election: true
I20260924 14:27:03.382007 23463 tablet_bootstrap.cc:654] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.381866 23311 leader_election.cc:302] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562; no voters: 
I20260924 14:27:03.379834 23502 raft_consensus.cc:2781] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260924 14:27:03.382869 23310 leader_election.cc:341] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:03.382947 23502 raft_consensus.cc:482] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:03.383335 23502 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.383745 23467 raft_consensus.cc:2781] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:03.384105 23467 raft_consensus.cc:482] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:03.384431 23467 raft_consensus.cc:3037] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.396766 23467 raft_consensus.cc:504] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.399801 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea8bfb3513f4b5885d3e801189e2848" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:03.400492 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea8bfb3513f4b5885d3e801189e2848" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:03.401031 23418 raft_consensus.cc:3037] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:27:03.401103 23310 leader_election.cc:341] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:03.403846 23464 tablet_bootstrap.cc:654] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.406368 23462 tablet_bootstrap.cc:492] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707: No bootstrap required, opened a new log
I20260924 14:27:03.398715 23502 raft_consensus.cc:504] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.406966 23462 ts_tablet_manager.cc:1397] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707: Time spent bootstrapping tablet: real 0.048s	user 0.005s	sys 0.011s
I20260924 14:27:03.407732 23418 raft_consensus.cc:2454] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 1.
I20260924 14:27:03.408938 23311 leader_election.cc:302] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562; no voters: 78d920610cdd4a2bbe2dde49988fbb11
I20260924 14:27:03.409240 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:03.409953 23467 leader_election.cc:288] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.410302 23488 raft_consensus.cc:2781] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:03.410583 23462 raft_consensus.cc:348] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.411702 23462 raft_consensus.cc:374] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.412076 23462 raft_consensus.cc:729] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.412839 23462 consensus_queue.cc:260] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
W20260924 14:27:03.413910 23234 leader_election.cc:341] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:03.410770 23488 raft_consensus.cc:686] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:03.416076 23462 ts_tablet_manager.cc:1428] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.004s
I20260924 14:27:03.417205 23462 tablet_bootstrap.cc:492] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707: Bootstrap starting.
I20260924 14:27:03.416695 23488 consensus_queue.cc:237] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.421622 23502 leader_election.cc:288] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.422034 23463 tablet_bootstrap.cc:492] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11: No bootstrap required, opened a new log
I20260924 14:27:03.422480 23463 ts_tablet_manager.cc:1397] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11: Time spent bootstrapping tablet: real 0.052s	user 0.018s	sys 0.026s
I20260924 14:27:03.423077 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:03.423911 23418 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.425036 23463 raft_consensus.cc:348] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.425773 23463 raft_consensus.cc:374] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.426067 23463 raft_consensus.cc:729] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.426717 23463 consensus_queue.cc:260] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.429066 23463 ts_tablet_manager.cc:1428] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260924 14:27:03.429991 23463 tablet_bootstrap.cc:492] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11: Bootstrap starting.
I20260924 14:27:03.430776 23462 tablet_bootstrap.cc:654] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.437493 23418 raft_consensus.cc:2454] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 1.
I20260924 14:27:03.446317 23512 consensus_queue.cc:1035] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:27:03.448582 23167 catalog_manager.cc:5495] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 0 to 1, leader changed from <none> to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 1 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:03.449014 23463 tablet_bootstrap.cc:654] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.448937 23236 leader_election.cc:302] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 278ba036c8a04b8ba9f143ffdeef6707
I20260924 14:27:03.455773 23502 raft_consensus.cc:2781] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:03.456275 23502 raft_consensus.cc:686] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Becoming Leader. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:03.458971 23502 consensus_queue.cc:237] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.469798 23464 tablet_bootstrap.cc:492] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562: No bootstrap required, opened a new log
I20260924 14:27:03.470487 23464 ts_tablet_manager.cc:1397] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562: Time spent bootstrapping tablet: real 0.125s	user 0.014s	sys 0.001s
I20260924 14:27:03.479873 23464 raft_consensus.cc:348] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.480645 23464 raft_consensus.cc:374] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.480918 23464 raft_consensus.cc:729] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.486662 23464 consensus_queue.cc:260] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.488796 23463 tablet_bootstrap.cc:492] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11: No bootstrap required, opened a new log
I20260924 14:27:03.489362 23463 ts_tablet_manager.cc:1397] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.023s
I20260924 14:27:03.492185 23464 ts_tablet_manager.cc:1428] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.001s
I20260924 14:27:03.492192 23463 raft_consensus.cc:348] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.493326 23464 tablet_bootstrap.cc:492] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562: Bootstrap starting.
I20260924 14:27:03.493363 23463 raft_consensus.cc:374] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.493907 23463 raft_consensus.cc:729] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.494666 23463 consensus_queue.cc:260] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.497105 23463 ts_tablet_manager.cc:1428] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:27:03.498047 23463 tablet_bootstrap.cc:492] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11: Bootstrap starting.
I20260924 14:27:03.500001 23464 tablet_bootstrap.cc:654] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.500125 23167 catalog_manager.cc:5495] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
W20260924 14:27:03.501243 23214 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:27:03.501909 23462 tablet_bootstrap.cc:492] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707: No bootstrap required, opened a new log
I20260924 14:27:03.502406 23462 ts_tablet_manager.cc:1397] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707: Time spent bootstrapping tablet: real 0.085s	user 0.030s	sys 0.026s
I20260924 14:27:03.503968 23463 tablet_bootstrap.cc:654] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:03.504848 23462 raft_consensus.cc:348] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.505589 23462 raft_consensus.cc:374] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.505872 23462 raft_consensus.cc:729] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.511490 23462 consensus_queue.cc:260] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.523003 23502 raft_consensus.cc:482] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.528278 23502 raft_consensus.cc:504] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.529035 23464 tablet_bootstrap.cc:492] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562: No bootstrap required, opened a new log
I20260924 14:27:03.523102 23462 ts_tablet_manager.cc:1428] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260924 14:27:03.530011 23464 ts_tablet_manager.cc:1397] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.010s
I20260924 14:27:03.532836 23464 raft_consensus.cc:348] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.534642 23464 raft_consensus.cc:374] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.534921 23464 raft_consensus.cc:729] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.535563 23464 consensus_queue.cc:260] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.537588 23464 ts_tablet_manager.cc:1428] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
W20260924 14:27:03.538528 23310 consensus_peers.cc:487] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 -> Peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Couldn't send request to peer 78d920610cdd4a2bbe2dde49988fbb11. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:27:03.540760 23502 leader_election.cc:288] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.541528 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707" is_pre_election: true
I20260924 14:27:03.542156 23345 raft_consensus.cc:2454] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 0.
I20260924 14:27:03.542320 23463 tablet_bootstrap.cc:492] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11: No bootstrap required, opened a new log
I20260924 14:27:03.542377 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.542909 23463 ts_tablet_manager.cc:1397] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11: Time spent bootstrapping tablet: real 0.045s	user 0.014s	sys 0.008s
I20260924 14:27:03.543325 23234 leader_election.cc:302] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:03.544193 23502 raft_consensus.cc:2781] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:03.544535 23502 raft_consensus.cc:482] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:03.544912 23502 raft_consensus.cc:3037] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.546523 23463 raft_consensus.cc:348] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.543166 23419 raft_consensus.cc:2454] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 0.
I20260924 14:27:03.547292 23463 raft_consensus.cc:374] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:03.547736 23463 raft_consensus.cc:729] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Initialized, Role: FOLLOWER
I20260924 14:27:03.548408 23463 consensus_queue.cc:260] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.554934 23463 ts_tablet_manager.cc:1428] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260924 14:27:03.558988 23502 raft_consensus.cc:504] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.561683 23502 leader_election.cc:288] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.561765 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:03.561930 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:03.562464 23345 raft_consensus.cc:3037] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.562546 23419 raft_consensus.cc:3037] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.567015 23345 raft_consensus.cc:2454] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 1.
I20260924 14:27:03.568115 23234 leader_election.cc:302] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:03.568802 23502 raft_consensus.cc:2781] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:03.569109 23419 raft_consensus.cc:2454] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 1.
I20260924 14:27:03.569232 23502 raft_consensus.cc:686] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Becoming Leader. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:03.570194 23502 consensus_queue.cc:237] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.578320 23164 catalog_manager.cc:5495] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:03.585515 23502 consensus_queue.cc:1035] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:03.592506 23470 raft_consensus.cc:482] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.593040 23470 raft_consensus.cc:504] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.594995 23470 leader_election.cc:288] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.597205 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.597538 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707" is_pre_election: true
I20260924 14:27:03.598650 23344 raft_consensus.cc:2379] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 78d920610cdd4a2bbe2dde49988fbb11 in current term 1: Already voted for candidate 2fc7e85a970e4997926a2f2b13bfd562 in this term.
I20260924 14:27:03.600392 23234 leader_election.cc:302] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78d920610cdd4a2bbe2dde49988fbb11; no voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:03.602970 23502 raft_consensus.cc:3037] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.608307 23470 consensus_queue.cc:1035] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:03.609643 23502 raft_consensus.cc:2726] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:03.647042 23467 raft_consensus.cc:482] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.647622 23467 raft_consensus.cc:504] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.649232 23467 leader_election.cc:288] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.650257 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884a631342f040a2b7babf614155f1a5" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11" is_pre_election: true
I20260924 14:27:03.650374 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884a631342f040a2b7babf614155f1a5" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.650818 23270 raft_consensus.cc:2454] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 0.
I20260924 14:27:03.651055 23418 raft_consensus.cc:2454] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 0.
I20260924 14:27:03.651902 23310 leader_election.cc:302] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:03.652693 23467 raft_consensus.cc:2781] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:03.653062 23467 raft_consensus.cc:482] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:03.653414 23467 raft_consensus.cc:3037] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.660305 23467 raft_consensus.cc:504] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.662281 23467 leader_election.cc:288] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.663214 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884a631342f040a2b7babf614155f1a5" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:03.664031 23270 raft_consensus.cc:3037] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.663825 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884a631342f040a2b7babf614155f1a5" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:03.664731 23418 raft_consensus.cc:3037] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.668846 23270 raft_consensus.cc:2454] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 1.
I20260924 14:27:03.669842 23310 leader_election.cc:302] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:03.670547 23467 raft_consensus.cc:2781] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:03.670985 23467 raft_consensus.cc:686] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:03.671187 23418 raft_consensus.cc:2454] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 1.
I20260924 14:27:03.671818 23467 consensus_queue.cc:237] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.680416 23471 consensus_queue.cc:1035] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:27:03.680341 23165 catalog_manager.cc:5495] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 0 to 1, leader changed from <none> to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 1 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:03.690531 23490 raft_consensus.cc:482] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.691043 23490 raft_consensus.cc:504] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.694243 23490 leader_election.cc:288] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.695011 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea8bfb3513f4b5885d3e801189e2848" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.695909 23418 raft_consensus.cc:2379] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 78d920610cdd4a2bbe2dde49988fbb11 in current term 1: Already voted for candidate 278ba036c8a04b8ba9f143ffdeef6707 in this term.
I20260924 14:27:03.699842 23490 raft_consensus.cc:482] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.700332 23490 raft_consensus.cc:504] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.701450 23471 consensus_queue.cc:1035] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:03.702251 23490 leader_election.cc:288] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.702462 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea8bfb3513f4b5885d3e801189e2848" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707" is_pre_election: true
I20260924 14:27:03.703791 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707" is_pre_election: true
I20260924 14:27:03.704254 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.705466 23234 leader_election.cc:302] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78d920610cdd4a2bbe2dde49988fbb11; no voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:03.706781 23490 raft_consensus.cc:3037] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.712458 23236 leader_election.cc:302] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78d920610cdd4a2bbe2dde49988fbb11; no voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:03.713778 23470 raft_consensus.cc:2726] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:03.715859 23490 raft_consensus.cc:2726] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:03.734332 23490 consensus_queue.cc:1035] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:03.737735 23490 consensus_queue.cc:1035] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:03.755722 23470 consensus_queue.cc:1035] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:03.766374 23490 consensus_queue.cc:1035] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:03.812343 23467 raft_consensus.cc:482] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:03.812817 23467 raft_consensus.cc:504] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.814365 23467 leader_election.cc:288] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.815429 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562" is_pre_election: true
I20260924 14:27:03.815485 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11" is_pre_election: true
I20260924 14:27:03.816252 23418 raft_consensus.cc:2379] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 278ba036c8a04b8ba9f143ffdeef6707 in current term 1: Already voted for candidate 78d920610cdd4a2bbe2dde49988fbb11 in this term.
I20260924 14:27:03.817502 23311 leader_election.cc:302] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707; no voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11
I20260924 14:27:03.818244 23467 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.822573 23467 raft_consensus.cc:2726] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:03.891093 23345 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
mode: GRACEFUL
new_leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
 from {username='slave'} at 127.0.0.1:55162
I20260924 14:27:03.891878 23345 raft_consensus.cc:595] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Received request to transfer leadership to TS 78d920610cdd4a2bbe2dde49988fbb11
I20260924 14:27:03.907111 23418 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815"
dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
mode: GRACEFUL
new_leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
 from {username='slave'} at 127.0.0.1:44146
I20260924 14:27:03.907706 23418 raft_consensus.cc:595] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 LEADER]: Received request to transfer leadership to TS 78d920610cdd4a2bbe2dde49988fbb11
I20260924 14:27:03.917589 23467 consensus_queue.cc:1035] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:03.934823 23512 consensus_queue.cc:1035] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:03.936504 23270 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
mode: GRACEFUL
new_leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55456
I20260924 14:27:03.937191 23270 raft_consensus.cc:595] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Received request to transfer leadership to TS 278ba036c8a04b8ba9f143ffdeef6707
I20260924 14:27:03.936446 23490 consensus_queue.cc:1035] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:27:03.944435 23270 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
mode: GRACEFUL
new_leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55456
I20260924 14:27:03.945075 23270 raft_consensus.cc:595] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Received request to transfer leadership to TS 278ba036c8a04b8ba9f143ffdeef6707
I20260924 14:27:03.953825 23270 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5b9cfc0424a8429ebd5886060eff5bde"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
mode: GRACEFUL
new_leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55456
I20260924 14:27:03.954470 23270 raft_consensus.cc:595] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Received request to transfer leadership to TS 278ba036c8a04b8ba9f143ffdeef6707
I20260924 14:27:03.965736 23490 consensus_queue.cc:1035] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:03.966167 23270 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:03.981580 23344 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "884a631342f040a2b7babf614155f1a5"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
mode: GRACEFUL
new_leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
 from {username='slave'} at 127.0.0.1:55162
I20260924 14:27:03.982175 23344 raft_consensus.cc:595] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Received request to transfer leadership to TS 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:03.984189 23467 raft_consensus.cc:979] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707: : Instructing follower 78d920610cdd4a2bbe2dde49988fbb11 to start an election
I20260924 14:27:03.984668 23488 raft_consensus.cc:1067] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Signalling peer 78d920610cdd4a2bbe2dde49988fbb11 to start an election
I20260924 14:27:03.987075 23270 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
 from {username='slave'} at 127.0.0.1:55444
I20260924 14:27:03.988690 23270 raft_consensus.cc:482] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:03.989128 23270 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:03.990387 23490 raft_consensus.cc:979] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11: : Instructing follower 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:03.990794 23490 raft_consensus.cc:1067] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Signalling peer 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:03.994074 23345 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55138
I20260924 14:27:03.994710 23345 raft_consensus.cc:482] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:03.995127 23345 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:03.995934 23270 raft_consensus.cc:504] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:03.998044 23270 leader_election.cc:288] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 2 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:03.999089 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:03.999569 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:03.999753 23418 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.000167 23344 raft_consensus.cc:3032] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:04.000490 23344 raft_consensus.cc:729] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:04.001225 23344 consensus_queue.cc:260] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.002562 23344 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.005838 23345 raft_consensus.cc:504] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.006124 23418 raft_consensus.cc:2454] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 2.
I20260924 14:27:04.008620 23345 leader_election.cc:288] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 2 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:04.008622 23236 leader_election.cc:302] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:04.010298 23502 raft_consensus.cc:2781] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:04.010771 23502 raft_consensus.cc:686] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Becoming Leader. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:04.010614 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:04.011541 23270 raft_consensus.cc:3032] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:04.011881 23270 raft_consensus.cc:729] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:04.011682 23502 consensus_queue.cc:237] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.012701 23270 consensus_queue.cc:260] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.013942 23270 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.014092 23344 raft_consensus.cc:2454] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 2.
I20260924 14:27:04.010771 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:04.016657 23418 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.020709 23270 raft_consensus.cc:2454] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 2.
I20260924 14:27:04.022149 23310 leader_election.cc:302] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:04.021454 23164 catalog_manager.cc:5495] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 reported cstate change: term changed from 1 to 2, leader changed from 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194) to 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:04.023030 23512 raft_consensus.cc:2781] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:04.023530 23512 raft_consensus.cc:686] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:04.023751 23418 raft_consensus.cc:2454] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 2.
I20260924 14:27:04.024343 23512 consensus_queue.cc:237] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.035007 23164 catalog_manager.cc:5495] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 1 to 2, leader changed from 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193) to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 2 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:04.133404 23502 consensus_queue.cc:1035] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:04.139760 23490 raft_consensus.cc:979] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11: : Instructing follower 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:04.140265 23490 raft_consensus.cc:1067] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Signalling peer 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:04.143787 23345 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55138
I20260924 14:27:04.144349 23345 raft_consensus.cc:482] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:04.144737 23345 raft_consensus.cc:3037] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.150121 23502 consensus_queue.cc:1035] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:04.150420 23345 raft_consensus.cc:504] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.151434 23344 raft_consensus.cc:1226] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Rejecting Update request from peer 78d920610cdd4a2bbe2dde49988fbb11 for earlier term 1. Current term is 2. Ops: []
I20260924 14:27:04.153959 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:04.154924 23502 consensus_queue.cc:1046] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:04.155252 23345 leader_election.cc:288] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 2 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:04.154596 23270 raft_consensus.cc:3032] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:04.156038 23270 raft_consensus.cc:729] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:04.156258 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:04.157284 23270 consensus_queue.cc:260] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.158255 23270 raft_consensus.cc:3037] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.162637 23270 raft_consensus.cc:2454] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 2.
W20260924 14:27:04.163048 23490 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260924 14:27:04.163750 23310 leader_election.cc:302] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:04.164901 23512 raft_consensus.cc:2781] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:04.165360 23512 raft_consensus.cc:686] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:04.166182 23512 consensus_queue.cc:237] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.175244 23502 raft_consensus.cc:979] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11: : Instructing follower 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:04.175468 23512 consensus_queue.cc:1035] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:04.175719 23502 raft_consensus.cc:1067] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Signalling peer 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:04.176433 23167 catalog_manager.cc:5495] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 1 to 2, leader changed from 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193) to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 2 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:04.177384 23345 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5b9cfc0424a8429ebd5886060eff5bde"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55138
I20260924 14:27:04.177948 23345 raft_consensus.cc:482] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:04.178275 23345 raft_consensus.cc:3037] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.199472 23345 raft_consensus.cc:504] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.201443 23345 leader_election.cc:288] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 2 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:04.204020 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9cfc0424a8429ebd5886060eff5bde" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:04.204653 23418 raft_consensus.cc:3037] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.205858 23467 consensus_queue.cc:1035] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:04.206173 23269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9cfc0424a8429ebd5886060eff5bde" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:04.206800 23269 raft_consensus.cc:3032] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:04.207726 23269 raft_consensus.cc:729] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:04.208473 23269 consensus_queue.cc:260] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.209697 23269 raft_consensus.cc:3037] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.212497 23418 raft_consensus.cc:2454] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 2.
I20260924 14:27:04.213804 23311 leader_election.cc:302] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562; no voters: 
I20260924 14:27:04.214586 23467 raft_consensus.cc:2781] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:04.215080 23467 raft_consensus.cc:686] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:04.216166 23467 consensus_queue.cc:237] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.224329 23269 raft_consensus.cc:2454] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 2.
I20260924 14:27:04.226889 23164 catalog_manager.cc:5495] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 1 to 2, leader changed from 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193) to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 2 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:04.228287 23488 raft_consensus.cc:979] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707: : Instructing follower 2fc7e85a970e4997926a2f2b13bfd562 to start an election
I20260924 14:27:04.228749 23488 raft_consensus.cc:1067] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Signalling peer 2fc7e85a970e4997926a2f2b13bfd562 to start an election
I20260924 14:27:04.230650 23419 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "884a631342f040a2b7babf614155f1a5"
dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
 from {username='slave'} at 127.0.0.1:44128
I20260924 14:27:04.231164 23419 raft_consensus.cc:482] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:04.231642 23419 raft_consensus.cc:3037] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.237680 23419 raft_consensus.cc:504] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.240291 23269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884a631342f040a2b7babf614155f1a5" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:04.240515 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884a631342f040a2b7babf614155f1a5" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:04.241199 23345 raft_consensus.cc:3032] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:04.241580 23269 raft_consensus.cc:3037] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.241611 23345 raft_consensus.cc:729] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:04.242616 23345 consensus_queue.cc:260] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.243738 23345 raft_consensus.cc:3037] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.241132 23419 leader_election.cc:288] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 2 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887)
I20260924 14:27:04.248687 23269 raft_consensus.cc:2454] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 2.
I20260924 14:27:04.249929 23345 raft_consensus.cc:2454] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 2.
I20260924 14:27:04.249902 23384 leader_election.cc:302] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:04.250851 23529 raft_consensus.cc:2781] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:04.251308 23529 raft_consensus.cc:686] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 LEADER]: Becoming Leader. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Running, Role: LEADER
I20260924 14:27:04.252138 23529 consensus_queue.cc:237] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.260354 23164 catalog_manager.cc:5495] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 reported cstate change: term changed from 1 to 2, leader changed from 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194) to 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:04.285648 23528 raft_consensus.cc:979] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562: : Instructing follower 78d920610cdd4a2bbe2dde49988fbb11 to start an election
I20260924 14:27:04.286131 23549 raft_consensus.cc:1067] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 LEADER]: Signalling peer 78d920610cdd4a2bbe2dde49988fbb11 to start an election
I20260924 14:27:04.287849 23269 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
 from {username='slave'} at 127.0.0.1:55446
I20260924 14:27:04.288431 23269 raft_consensus.cc:482] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:04.288807 23269 raft_consensus.cc:3037] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.293697 23269 raft_consensus.cc:504] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.295193 23269 leader_election.cc:288] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 2 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:04.296231 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:04.296125 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:04.296898 23419 raft_consensus.cc:3032] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:04.296988 23345 raft_consensus.cc:3037] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.297269 23419 raft_consensus.cc:729] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Running, Role: LEADER
I20260924 14:27:04.298131 23419 consensus_queue.cc:260] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.299245 23419 raft_consensus.cc:3037] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.303784 23419 raft_consensus.cc:2454] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 2.
I20260924 14:27:04.303907 23345 raft_consensus.cc:2454] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 2.
I20260924 14:27:04.304944 23236 leader_election.cc:302] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:04.305996 23547 raft_consensus.cc:2781] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:04.306425 23547 raft_consensus.cc:686] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Becoming Leader. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:04.307247 23547 consensus_queue.cc:237] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:04.315025 23164 catalog_manager.cc:5495] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 reported cstate change: term changed from 1 to 2, leader changed from 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195) to 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:04.402539 23419 raft_consensus.cc:1261] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Refusing update from remote peer 78d920610cdd4a2bbe2dde49988fbb11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.403960 23547 consensus_queue.cc:1035] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:04.413174 23345 raft_consensus.cc:1261] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Refusing update from remote peer 78d920610cdd4a2bbe2dde49988fbb11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.414739 23502 consensus_queue.cc:1035] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:04.460469 23419 raft_consensus.cc:1261] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.462119 23467 consensus_queue.cc:1035] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:04.472141 23269 raft_consensus.cc:1261] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.474151 23512 consensus_queue.cc:1035] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:04.604614 23269 raft_consensus.cc:1261] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.605918 23467 consensus_queue.cc:1035] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:04.614377 23419 raft_consensus.cc:3037] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:04.621738 23419 raft_consensus.cc:1261] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.623137 23512 consensus_queue.cc:1035] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:27:04.669977 23269 raft_consensus.cc:1261] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Refusing update from remote peer 2fc7e85a970e4997926a2f2b13bfd562: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.671267 23549 consensus_queue.cc:1035] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:04.678483 23345 raft_consensus.cc:1261] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Refusing update from remote peer 2fc7e85a970e4997926a2f2b13bfd562: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.680946 23529 consensus_queue.cc:1035] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:04.725365 23345 raft_consensus.cc:1261] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Refusing update from remote peer 78d920610cdd4a2bbe2dde49988fbb11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.726547 23490 consensus_queue.cc:1035] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:04.735316 23419 raft_consensus.cc:1261] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Refusing update from remote peer 78d920610cdd4a2bbe2dde49988fbb11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.737001 23547 consensus_queue.cc:1035] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:04.754667 23419 raft_consensus.cc:1261] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.755869 23512 consensus_queue.cc:1035] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:04.764321 23269 raft_consensus.cc:1261] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:04.765689 23488 consensus_queue.cc:1035] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:07.011754 23135 auto_leader_rebalancer-test.cc:269] The leader distribution is 
I20260924 14:27:07.012563 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:07.035681 23270 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
mode: GRACEFUL
new_leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
 from {username='slave'} at 127.0.0.1:55464
I20260924 14:27:07.036285 23270 raft_consensus.cc:595] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Received request to transfer leadership to TS 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:07.048311 23344 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
mode: GRACEFUL
new_leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
 from {username='slave'} at 127.0.0.1:55164
I20260924 14:27:07.048815 23344 raft_consensus.cc:595] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Received request to transfer leadership to TS 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:07.049831 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260924 14:27:07.050369 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:07.097693 23560 raft_consensus.cc:979] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707: : Instructing follower 2fc7e85a970e4997926a2f2b13bfd562 to start an election
I20260924 14:27:07.098227 23467 raft_consensus.cc:1067] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Signalling peer 2fc7e85a970e4997926a2f2b13bfd562 to start an election
I20260924 14:27:07.099845 23418 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523"
dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
 from {username='slave'} at 127.0.0.1:44128
I20260924 14:27:07.100334 23418 raft_consensus.cc:482] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:07.100631 23418 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:07.105255 23418 raft_consensus.cc:504] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:07.107015 23418 leader_election.cc:288] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 3 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887)
I20260924 14:27:07.108192 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:07.108224 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:07.108795 23270 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:07.108834 23344 raft_consensus.cc:3032] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:07.109208 23344 raft_consensus.cc:729] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:07.109826 23344 consensus_queue.cc:260] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:07.110756 23344 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:07.113749 23270 raft_consensus.cc:2454] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 3.
I20260924 14:27:07.115003 23384 leader_election.cc:302] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:07.115770 23554 raft_consensus.cc:2781] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:07.116647 23344 raft_consensus.cc:2454] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 3.
I20260924 14:27:07.118618 23554 raft_consensus.cc:686] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Becoming Leader. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Running, Role: LEADER
I20260924 14:27:07.119614 23554 consensus_queue.cc:237] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:07.126294 23167 catalog_manager.cc:5495] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 reported cstate change: term changed from 2 to 3, leader changed from 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194) to 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:07.363487 23555 raft_consensus.cc:979] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11: : Instructing follower 2fc7e85a970e4997926a2f2b13bfd562 to start an election
I20260924 14:27:07.363960 23568 raft_consensus.cc:1067] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Signalling peer 2fc7e85a970e4997926a2f2b13bfd562 to start an election
I20260924 14:27:07.365443 23418 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a"
dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
 from {username='slave'} at 127.0.0.1:44134
I20260924 14:27:07.365877 23418 raft_consensus.cc:482] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:07.366103 23418 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:07.370694 23418 raft_consensus.cc:504] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:07.372133 23418 leader_election.cc:288] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 3 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195)
I20260924 14:27:07.373149 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:07.373348 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:07.373776 23344 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:07.373870 23270 raft_consensus.cc:3032] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:07.374191 23270 raft_consensus.cc:729] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:07.374953 23270 consensus_queue.cc:260] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:07.376225 23270 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:07.380421 23344 raft_consensus.cc:2454] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 3.
I20260924 14:27:07.381582 23383 leader_election.cc:302] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562; no voters: 
I20260924 14:27:07.382362 23554 raft_consensus.cc:2781] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:07.382776 23270 raft_consensus.cc:2454] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 3.
I20260924 14:27:07.382828 23554 raft_consensus.cc:686] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Becoming Leader. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Running, Role: LEADER
I20260924 14:27:07.383775 23554 consensus_queue.cc:237] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:07.393064 23167 catalog_manager.cc:5495] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 reported cstate change: term changed from 2 to 3, leader changed from 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193) to 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:07.553328 23270 raft_consensus.cc:1261] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Refusing update from remote peer 2fc7e85a970e4997926a2f2b13bfd562: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:07.554666 23554 consensus_queue.cc:1035] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:07.563804 23344 raft_consensus.cc:1261] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Refusing update from remote peer 2fc7e85a970e4997926a2f2b13bfd562: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:07.566838 23554 consensus_queue.cc:1035] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:07.860463 23345 raft_consensus.cc:1261] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Refusing update from remote peer 2fc7e85a970e4997926a2f2b13bfd562: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:07.861722 23554 consensus_queue.cc:1035] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:07.870244 23270 raft_consensus.cc:1261] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Refusing update from remote peer 2fc7e85a970e4997926a2f2b13bfd562: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:07.871476 23562 consensus_queue.cc:1035] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:08.051226 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:08.056730 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:08.057118 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:09.058027 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:09.063395 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:09.063830 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:10.065311 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:10.071548 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:10.072026 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:11.073020 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:11.077644 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:11.078097 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:12.078887 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:12.084738 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:12.085220 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:13.086136 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:13.091653 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:13.092041 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:14.093124 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:14.099416 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:14.100008 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:15.100917 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:15.105772 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:15.106241 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:16.107086 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:16.111763 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:16.112217 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:17.140350 23135 auto_leader_rebalancer-test.cc:288] The leader distribution is 
I20260924 14:27:17.145032 23419 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a"
dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
mode: GRACEFUL
new_leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:44146
I20260924 14:27:17.145794 23419 raft_consensus.cc:595] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Received request to transfer leadership to TS 278ba036c8a04b8ba9f143ffdeef6707
I20260924 14:27:17.151153 23269 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
mode: GRACEFUL
new_leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55456
I20260924 14:27:17.151717 23269 raft_consensus.cc:595] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Received request to transfer leadership to TS 278ba036c8a04b8ba9f143ffdeef6707
I20260924 14:27:17.156888 23269 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fd806eef81974e49811ad08d74410fdb"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
mode: GRACEFUL
new_leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55456
I20260924 14:27:17.157445 23269 raft_consensus.cc:595] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Received request to transfer leadership to TS 278ba036c8a04b8ba9f143ffdeef6707
I20260924 14:27:17.162151 23270 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "496d8e256cf2413f8bf270323313d891"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
mode: GRACEFUL
new_leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55456
I20260924 14:27:17.162711 23270 raft_consensus.cc:595] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Received request to transfer leadership to TS 278ba036c8a04b8ba9f143ffdeef6707
I20260924 14:27:17.169000 23419 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523"
dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
mode: GRACEFUL
new_leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:44146
I20260924 14:27:17.169467 23419 raft_consensus.cc:595] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Received request to transfer leadership to TS 278ba036c8a04b8ba9f143ffdeef6707
I20260924 14:27:17.181663 23583 raft_consensus.cc:979] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11: : Instructing follower 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:17.182227 23587 raft_consensus.cc:1067] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Signalling peer 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:17.183665 23345 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fd806eef81974e49811ad08d74410fdb"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55138
I20260924 14:27:17.184095 23345 raft_consensus.cc:482] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:17.184388 23345 raft_consensus.cc:3037] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:17.189224 23345 raft_consensus.cc:504] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.191087 23345 leader_election.cc:288] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 2 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:17.191960 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd806eef81974e49811ad08d74410fdb" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:17.192274 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd806eef81974e49811ad08d74410fdb" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:17.192718 23270 raft_consensus.cc:3032] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:17.192845 23419 raft_consensus.cc:3037] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:17.193054 23270 raft_consensus.cc:729] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:17.193747 23270 consensus_queue.cc:260] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.195070 23270 raft_consensus.cc:3037] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:17.199744 23419 raft_consensus.cc:2454] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 2.
I20260924 14:27:17.201222 23311 leader_election.cc:302] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562; no voters: 
I20260924 14:27:17.201354 23270 raft_consensus.cc:2454] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 2.
I20260924 14:27:17.202096 23584 raft_consensus.cc:2781] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:17.202957 23584 raft_consensus.cc:686] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:17.203787 23584 consensus_queue.cc:237] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.210335 23167 catalog_manager.cc:5495] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 1 to 2, leader changed from 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193) to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 2 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:17.404305 23554 raft_consensus.cc:979] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562: : Instructing follower 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:17.404706 23554 raft_consensus.cc:1067] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Signalling peer 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:17.406294 23345 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55146
I20260924 14:27:17.406782 23345 raft_consensus.cc:482] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:17.407069 23345 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:17.410646 23583 raft_consensus.cc:979] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11: : Instructing follower 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:17.411074 23583 raft_consensus.cc:1067] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Signalling peer 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:17.412592 23344 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55138
I20260924 14:27:17.413060 23344 raft_consensus.cc:482] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:17.412804 23345 raft_consensus.cc:504] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.413414 23344 raft_consensus.cc:3037] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:17.414951 23345 leader_election.cc:288] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 4 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:17.415331 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:17.416217 23270 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:17.415654 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:17.416980 23419 raft_consensus.cc:3032] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:27:17.417270 23419 raft_consensus.cc:729] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Running, Role: LEADER
I20260924 14:27:17.418012 23419 consensus_queue.cc:260] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.419294 23419 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:17.419108 23344 raft_consensus.cc:504] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.421248 23344 leader_election.cc:288] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 3 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:17.421727 23269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:17.422434 23269 raft_consensus.cc:3032] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:17.422700 23269 raft_consensus.cc:729] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:17.422888 23270 raft_consensus.cc:2454] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 4.
I20260924 14:27:17.423282 23269 consensus_queue.cc:260] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.424098 23310 leader_election.cc:302] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:17.424566 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca20c5eab35d4b9a8f3945f4e9bf5815" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:17.424926 23584 raft_consensus.cc:2781] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:27:17.424731 23269 raft_consensus.cc:3037] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:17.425154 23418 raft_consensus.cc:3037] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:17.425426 23584 raft_consensus.cc:686] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:17.426303 23419 raft_consensus.cc:2454] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 4.
I20260924 14:27:17.426324 23584 consensus_queue.cc:237] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.431757 23418 raft_consensus.cc:2454] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 3.
I20260924 14:27:17.431757 23269 raft_consensus.cc:2454] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 3.
I20260924 14:27:17.433106 23311 leader_election.cc:302] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 2fc7e85a970e4997926a2f2b13bfd562; no voters: 
I20260924 14:27:17.434160 23584 raft_consensus.cc:2781] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:17.434558 23584 raft_consensus.cc:686] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:17.435263 23584 consensus_queue.cc:237] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.435205 23167 catalog_manager.cc:5495] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 3 to 4, leader changed from 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195) to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 4 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:17.446956 23167 catalog_manager.cc:5495] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 2 to 3, leader changed from 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193) to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 3 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:17.466259 23587 raft_consensus.cc:979] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11: : Instructing follower 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:17.466748 23587 raft_consensus.cc:1067] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Signalling peer 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:17.468727 23344 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "496d8e256cf2413f8bf270323313d891"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55138
I20260924 14:27:17.469249 23344 raft_consensus.cc:482] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:17.469738 23344 raft_consensus.cc:3037] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:17.476459 23344 raft_consensus.cc:504] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.478266 23344 leader_election.cc:288] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 2 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:17.479339 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496d8e256cf2413f8bf270323313d891" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:17.479339 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496d8e256cf2413f8bf270323313d891" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:17.479915 23594 raft_consensus.cc:979] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562: : Instructing follower 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:17.480024 23418 raft_consensus.cc:3037] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:17.480105 23270 raft_consensus.cc:3032] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:17.480943 23270 raft_consensus.cc:729] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:17.480952 23594 raft_consensus.cc:1067] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Signalling peer 278ba036c8a04b8ba9f143ffdeef6707 to start an election
I20260924 14:27:17.481716 23270 consensus_queue.cc:260] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.482918 23270 raft_consensus.cc:3037] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:17.482889 23344 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
 from {username='slave'} at 127.0.0.1:55146
I20260924 14:27:17.483590 23344 raft_consensus.cc:482] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:17.483960 23344 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:17.487522 23270 raft_consensus.cc:2454] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 2.
I20260924 14:27:17.487401 23418 raft_consensus.cc:2454] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 2.
I20260924 14:27:17.488598 23310 leader_election.cc:302] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:17.489264 23584 raft_consensus.cc:2781] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:17.489766 23584 raft_consensus.cc:686] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:17.490063 23344 raft_consensus.cc:504] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.490590 23584 consensus_queue.cc:237] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.491679 23344 leader_election.cc:288] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 4 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:17.492822 23418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:17.492890 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "278ba036c8a04b8ba9f143ffdeef6707" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:17.493451 23418 raft_consensus.cc:3032] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:27:17.493505 23270 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:17.493767 23418 raft_consensus.cc:729] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Running, Role: LEADER
I20260924 14:27:17.494783 23418 consensus_queue.cc:260] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.496191 23418 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:17.498613 23167 catalog_manager.cc:5495] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 1 to 2, leader changed from 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193) to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 2 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:17.501343 23270 raft_consensus.cc:2454] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 4.
I20260924 14:27:17.502485 23310 leader_election.cc:302] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278ba036c8a04b8ba9f143ffdeef6707, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:17.502878 23418 raft_consensus.cc:2454] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278ba036c8a04b8ba9f143ffdeef6707 in term 4.
I20260924 14:27:17.503283 23584 raft_consensus.cc:2781] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:27:17.503927 23584 raft_consensus.cc:686] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 LEADER]: Becoming Leader. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:17.504804 23584 consensus_queue.cc:237] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:17.512137 23167 catalog_manager.cc:5495] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 reported cstate change: term changed from 3 to 4, leader changed from 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195) to 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194). New cstate: current_term: 4 leader_uuid: "278ba036c8a04b8ba9f143ffdeef6707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:17.671314 23270 raft_consensus.cc:1261] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:17.672711 23584 consensus_queue.cc:1035] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:27:17.683573 23418 raft_consensus.cc:1261] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:17.684934 23584 consensus_queue.cc:1035] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:17.927302 23270 raft_consensus.cc:1261] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 4 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:27:17.928802 23584 consensus_queue.cc:1035] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:27:17.938043 23418 raft_consensus.cc:1261] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 4 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:27:17.939491 23584 consensus_queue.cc:1035] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:27:17.947146 23419 raft_consensus.cc:1261] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 4 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:27:17.948328 23584 consensus_queue.cc:1035] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:27:17.957155 23270 raft_consensus.cc:1261] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 4 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:27:17.958937 23588 consensus_queue.cc:1035] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:27:17.994676 23270 raft_consensus.cc:1261] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:17.995872 23588 consensus_queue.cc:1035] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:18.004347 23419 raft_consensus.cc:1261] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:18.006031 23584 consensus_queue.cc:1035] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:18.068563 23419 raft_consensus.cc:1261] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:18.069715 23584 consensus_queue.cc:1035] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:18.079491 23270 raft_consensus.cc:1261] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Refusing update from remote peer 278ba036c8a04b8ba9f143ffdeef6707: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:18.081063 23584 consensus_queue.cc:1035] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:20.214500 23135 auto_leader_rebalancer-test.cc:303] The leader distribution is 
I20260924 14:27:20.215667 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:20.222749 23344 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "496d8e256cf2413f8bf270323313d891"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
mode: GRACEFUL
new_leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
 from {username='slave'} at 127.0.0.1:55164
I20260924 14:27:20.223523 23344 raft_consensus.cc:595] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Received request to transfer leadership to TS 78d920610cdd4a2bbe2dde49988fbb11
I20260924 14:27:20.226253 23344 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
mode: GRACEFUL
new_leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
 from {username='slave'} at 127.0.0.1:55164
I20260924 14:27:20.226730 23344 raft_consensus.cc:595] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Received request to transfer leadership to TS 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:20.228932 23344 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
mode: GRACEFUL
new_leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
 from {username='slave'} at 127.0.0.1:55164
I20260924 14:27:20.229485 23344 raft_consensus.cc:595] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 LEADER]: Received request to transfer leadership to TS 78d920610cdd4a2bbe2dde49988fbb11
I20260924 14:27:20.231958 23344 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bea8bfb3513f4b5885d3e801189e2848"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
mode: GRACEFUL
new_leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
 from {username='slave'} at 127.0.0.1:55164
I20260924 14:27:20.232443 23344 raft_consensus.cc:595] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Received request to transfer leadership to TS 78d920610cdd4a2bbe2dde49988fbb11
I20260924 14:27:20.234792 23344 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523"
dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
mode: GRACEFUL
new_leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
 from {username='slave'} at 127.0.0.1:55164
I20260924 14:27:20.235299 23344 raft_consensus.cc:595] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 LEADER]: Received request to transfer leadership to TS 2fc7e85a970e4997926a2f2b13bfd562
I20260924 14:27:20.236238 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:27:20.236708 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:20.252655 23615 raft_consensus.cc:979] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707: : Instructing follower 2fc7e85a970e4997926a2f2b13bfd562 to start an election
I20260924 14:27:20.253211 23613 raft_consensus.cc:1067] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 LEADER]: Signalling peer 2fc7e85a970e4997926a2f2b13bfd562 to start an election
I20260924 14:27:20.254998 23419 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523"
dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
 from {username='slave'} at 127.0.0.1:44128
I20260924 14:27:20.255579 23419 raft_consensus.cc:482] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:20.255879 23419 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:27:20.270042 23419 raft_consensus.cc:504] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.272908 23419 leader_election.cc:288] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 5 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887)
I20260924 14:27:20.274369 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:20.274726 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1c48c6a1254b5685577a524c0ca523" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:20.275228 23270 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:27:20.275352 23344 raft_consensus.cc:3032] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:27:20.275774 23344 raft_consensus.cc:729] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:20.276510 23344 consensus_queue.cc:260] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.277796 23344 raft_consensus.cc:3037] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:27:20.278334 23615 raft_consensus.cc:979] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707: : Instructing follower 78d920610cdd4a2bbe2dde49988fbb11 to start an election
I20260924 14:27:20.278795 23613 raft_consensus.cc:1067] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Signalling peer 78d920610cdd4a2bbe2dde49988fbb11 to start an election
I20260924 14:27:20.280591 23269 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bea8bfb3513f4b5885d3e801189e2848"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
 from {username='slave'} at 127.0.0.1:55444
I20260924 14:27:20.281118 23269 raft_consensus.cc:482] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:20.281472 23269 raft_consensus.cc:3037] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:20.307936 23270 raft_consensus.cc:2454] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 5.
I20260924 14:27:20.307942 23344 raft_consensus.cc:2454] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 5.
I20260924 14:27:20.308048 23269 raft_consensus.cc:504] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.310052 23384 leader_election.cc:302] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:20.311241 23269 leader_election.cc:288] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 2 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:20.311390 23554 raft_consensus.cc:2781] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:27:20.312321 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea8bfb3513f4b5885d3e801189e2848" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:20.312716 23554 raft_consensus.cc:686] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 5 LEADER]: Becoming Leader. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Running, Role: LEADER
I20260924 14:27:20.313042 23419 raft_consensus.cc:3037] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:20.312321 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea8bfb3513f4b5885d3e801189e2848" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:20.313751 23344 raft_consensus.cc:3032] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:20.313663 23554 consensus_queue.cc:237] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.314082 23344 raft_consensus.cc:729] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:20.314883 23344 consensus_queue.cc:260] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.316274 23344 raft_consensus.cc:3037] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:20.324424 23167 catalog_manager.cc:5495] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 reported cstate change: term changed from 4 to 5, leader changed from 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194) to 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195). New cstate: current_term: 5 leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:20.407040 23419 raft_consensus.cc:2454] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 2.
I20260924 14:27:20.408886 23344 raft_consensus.cc:2454] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 2.
I20260924 14:27:20.409137 23236 leader_election.cc:302] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:20.411170 23622 raft_consensus.cc:2781] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:20.412660 23622 raft_consensus.cc:686] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Becoming Leader. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:20.413581 23622 consensus_queue.cc:237] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.421631 23167 catalog_manager.cc:5495] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 reported cstate change: term changed from 1 to 2, leader changed from 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194) to 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:20.508306 23615 raft_consensus.cc:979] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707: : Instructing follower 2fc7e85a970e4997926a2f2b13bfd562 to start an election
I20260924 14:27:20.508744 23613 raft_consensus.cc:1067] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Signalling peer 2fc7e85a970e4997926a2f2b13bfd562 to start an election
I20260924 14:27:20.510083 23419 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f"
dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
 from {username='slave'} at 127.0.0.1:44128
I20260924 14:27:20.510512 23419 raft_consensus.cc:482] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:20.510768 23419 raft_consensus.cc:3037] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:20.604357 23419 raft_consensus.cc:504] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.606024 23419 leader_election.cc:288] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 3 election: Requested vote from peers 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195), 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887)
I20260924 14:27:20.607079 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:20.606984 23269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6160343c1d3e4920801e9a06b7355f1f" candidate_uuid: "2fc7e85a970e4997926a2f2b13bfd562" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
I20260924 14:27:20.607761 23344 raft_consensus.cc:3032] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:20.607838 23269 raft_consensus.cc:3037] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:20.608135 23344 raft_consensus.cc:729] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:20.608834 23344 consensus_queue.cc:260] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.609867 23344 raft_consensus.cc:3037] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:20.663920 23615 raft_consensus.cc:979] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707: : Instructing follower 78d920610cdd4a2bbe2dde49988fbb11 to start an election
I20260924 14:27:20.664405 23613 raft_consensus.cc:1067] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 LEADER]: Signalling peer 78d920610cdd4a2bbe2dde49988fbb11 to start an election
I20260924 14:27:20.665891 23270 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
 from {username='slave'} at 127.0.0.1:55444
I20260924 14:27:20.666327 23270 raft_consensus.cc:482] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:20.666610 23270 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:27:20.698974 23268 raft_consensus.cc:1261] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 5 FOLLOWER]: Refusing update from remote peer 2fc7e85a970e4997926a2f2b13bfd562: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:27:20.700224 23554 consensus_queue.cc:1035] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:27:20.703313 23269 raft_consensus.cc:2454] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 3.
I20260924 14:27:20.704826 23384 leader_election.cc:302] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:20.703365 23344 raft_consensus.cc:2454] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fc7e85a970e4997926a2f2b13bfd562 in term 3.
I20260924 14:27:20.706720 23554 raft_consensus.cc:2781] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:20.707198 23554 raft_consensus.cc:686] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Becoming Leader. State: Replica: 2fc7e85a970e4997926a2f2b13bfd562, State: Running, Role: LEADER
I20260924 14:27:20.707993 23554 consensus_queue.cc:237] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.717826 23166 catalog_manager.cc:5495] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 reported cstate change: term changed from 2 to 3, leader changed from 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194) to 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "2fc7e85a970e4997926a2f2b13bfd562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:20.718983 23344 raft_consensus.cc:1261] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 5 FOLLOWER]: Refusing update from remote peer 2fc7e85a970e4997926a2f2b13bfd562: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:27:20.722043 23627 consensus_queue.cc:1035] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260924 14:27:20.804340 23270 raft_consensus.cc:504] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.806007 23270 leader_election.cc:288] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 5 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:20.806895 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:20.807133 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa058204bc8d43e8893ef87079f0e38a" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:20.807832 23344 raft_consensus.cc:3032] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:27:20.807945 23419 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:27:20.808245 23344 raft_consensus.cc:729] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:20.808980 23344 consensus_queue.cc:260] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.809958 23344 raft_consensus.cc:3037] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:27:20.811007 23615 raft_consensus.cc:979] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707: : Instructing follower 78d920610cdd4a2bbe2dde49988fbb11 to start an election
I20260924 14:27:20.811467 23613 raft_consensus.cc:1067] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Signalling peer 78d920610cdd4a2bbe2dde49988fbb11 to start an election
I20260924 14:27:20.812971 23270 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "496d8e256cf2413f8bf270323313d891"
dest_uuid: "78d920610cdd4a2bbe2dde49988fbb11"
 from {username='slave'} at 127.0.0.1:55444
I20260924 14:27:20.813474 23270 raft_consensus.cc:482] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:20.813791 23270 raft_consensus.cc:3037] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:20.872509 23418 raft_consensus.cc:1261] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Refusing update from remote peer 78d920610cdd4a2bbe2dde49988fbb11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:20.873894 23622 consensus_queue.cc:1035] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:20.885404 23345 raft_consensus.cc:1261] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Refusing update from remote peer 78d920610cdd4a2bbe2dde49988fbb11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:20.887109 23622 consensus_queue.cc:1035] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:20.904560 23419 raft_consensus.cc:2454] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 5.
I20260924 14:27:20.904911 23270 raft_consensus.cc:504] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.904539 23344 raft_consensus.cc:2454] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 5.
I20260924 14:27:20.906114 23236 leader_election.cc:302] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:20.906951 23622 raft_consensus.cc:2781] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:27:20.907169 23270 leader_election.cc:288] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 3 election: Requested vote from peers 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194:33887), 2fc7e85a970e4997926a2f2b13bfd562 (127.22.151.195:43155)
I20260924 14:27:20.907958 23622 raft_consensus.cc:686] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 5 LEADER]: Becoming Leader. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:20.908394 23419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496d8e256cf2413f8bf270323313d891" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2fc7e85a970e4997926a2f2b13bfd562"
I20260924 14:27:20.909126 23419 raft_consensus.cc:3037] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:20.908839 23622 consensus_queue.cc:237] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.910143 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496d8e256cf2413f8bf270323313d891" candidate_uuid: "78d920610cdd4a2bbe2dde49988fbb11" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "278ba036c8a04b8ba9f143ffdeef6707"
I20260924 14:27:20.910621 23345 raft_consensus.cc:3032] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:20.910864 23345 raft_consensus.cc:729] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 278ba036c8a04b8ba9f143ffdeef6707, State: Running, Role: LEADER
I20260924 14:27:20.911744 23345 consensus_queue.cc:260] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:20.913280 23345 raft_consensus.cc:3037] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:20.918488 23166 catalog_manager.cc:5495] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 reported cstate change: term changed from 4 to 5, leader changed from 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194) to 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:21.004616 23419 raft_consensus.cc:2454] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 3.
I20260924 14:27:21.004630 23345 raft_consensus.cc:2454] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d920610cdd4a2bbe2dde49988fbb11 in term 3.
I20260924 14:27:21.005896 23236 leader_election.cc:302] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fc7e85a970e4997926a2f2b13bfd562, 78d920610cdd4a2bbe2dde49988fbb11; no voters: 
I20260924 14:27:21.006811 23622 raft_consensus.cc:2781] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:21.007239 23622 raft_consensus.cc:686] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 LEADER]: Becoming Leader. State: Replica: 78d920610cdd4a2bbe2dde49988fbb11, State: Running, Role: LEADER
I20260924 14:27:21.008329 23622 consensus_queue.cc:237] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } }
I20260924 14:27:21.014822 23166 catalog_manager.cc:5495] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 reported cstate change: term changed from 2 to 3, leader changed from 278ba036c8a04b8ba9f143ffdeef6707 (127.22.151.194) to 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "78d920610cdd4a2bbe2dde49988fbb11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:21.172264 23345 raft_consensus.cc:1261] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Refusing update from remote peer 2fc7e85a970e4997926a2f2b13bfd562: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:21.173635 23616 consensus_queue.cc:1035] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:21.181509 23270 raft_consensus.cc:1261] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Refusing update from remote peer 2fc7e85a970e4997926a2f2b13bfd562: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:21.183672 23627 consensus_queue.cc:1035] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d920610cdd4a2bbe2dde49988fbb11" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:21.237552 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:21.242712 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:21.243127 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:21.408303 23345 raft_consensus.cc:1261] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 5 FOLLOWER]: Refusing update from remote peer 78d920610cdd4a2bbe2dde49988fbb11: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:27:21.409667 23622 consensus_queue.cc:1035] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:27:21.419899 23419 raft_consensus.cc:1261] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 5 FOLLOWER]: Refusing update from remote peer 78d920610cdd4a2bbe2dde49988fbb11: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:27:21.421746 23640 consensus_queue.cc:1035] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:27:21.433507 23344 raft_consensus.cc:1261] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Refusing update from remote peer 78d920610cdd4a2bbe2dde49988fbb11: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:21.434880 23640 consensus_queue.cc:1035] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278ba036c8a04b8ba9f143ffdeef6707" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:21.443205 23419 raft_consensus.cc:1261] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Refusing update from remote peer 78d920610cdd4a2bbe2dde49988fbb11: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:21.445573 23622 consensus_queue.cc:1035] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fc7e85a970e4997926a2f2b13bfd562" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:22.243968 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:22.249440 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:22.249893 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:23.391366 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:23.396207 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:23.396679 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260924 14:27:24.164440 23440 debug-util.cc:398] Leaking SignalData structure 0x7b0800174060 after lost signal to thread 23138
W20260924 14:27:24.165236 23440 debug-util.cc:398] Leaking SignalData structure 0x7b0800174740 after lost signal to thread 23201
W20260924 14:27:24.166011 23440 debug-util.cc:398] Leaking SignalData structure 0x7b0800097840 after lost signal to thread 23294
W20260924 14:27:24.166735 23440 debug-util.cc:398] Leaking SignalData structure 0x7b08000d57c0 after lost signal to thread 23369
W20260924 14:27:24.167429 23440 debug-util.cc:398] Leaking SignalData structure 0x7b0800097f80 after lost signal to thread 23443
I20260924 14:27:24.397694 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:24.405839 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:24.406381 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:25.407542 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:25.415633 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:25.416208 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:26.417076 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:26.423173 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:26.423662 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:27.424479 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:27.429291 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:27.429708 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:28.430493 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:28.435484 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:28.435926 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:29.436661 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:29.441318 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:29.441715 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:30.475950 23135 auto_leader_rebalancer-test.cc:314] The leader distribution is 
I20260924 14:27:30.476727 23135 tablet_server.cc:178] TabletServer@127.22.151.193:0 shutting down...
W20260924 14:27:30.495355 23384 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:27:30.505048 23384 consensus_peers.cc:487] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 -> Peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Couldn't send request to peer 78d920610cdd4a2bbe2dde49988fbb11. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:27:30.515589 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:27:30.516337 23135 tablet_replica.cc:330] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11: stopping tablet replica
I20260924 14:27:30.517305 23135 raft_consensus.cc:2229] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.518025 23135 raft_consensus.cc:2258] T ed1c48c6a1254b5685577a524c0ca523 P 78d920610cdd4a2bbe2dde49988fbb11 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.520448 23135 tablet_replica.cc:330] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11: stopping tablet replica
I20260924 14:27:30.520967 23135 raft_consensus.cc:2229] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.521415 23135 raft_consensus.cc:2258] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.523180 23135 tablet_replica.cc:330] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11: stopping tablet replica
I20260924 14:27:30.523701 23135 raft_consensus.cc:2229] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.524119 23135 raft_consensus.cc:2258] T 5b9cfc0424a8429ebd5886060eff5bde P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.525730 23135 tablet_replica.cc:330] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11: stopping tablet replica
I20260924 14:27:30.526230 23135 raft_consensus.cc:2229] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 5 LEADER]: Raft consensus shutting down.
I20260924 14:27:30.527055 23135 raft_consensus.cc:2258] T aa058204bc8d43e8893ef87079f0e38a P 78d920610cdd4a2bbe2dde49988fbb11 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.528772 23135 tablet_replica.cc:330] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11: stopping tablet replica
I20260924 14:27:30.529247 23135 raft_consensus.cc:2229] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.529672 23135 raft_consensus.cc:2258] T fd806eef81974e49811ad08d74410fdb P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.531220 23135 tablet_replica.cc:330] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11: stopping tablet replica
I20260924 14:27:30.531749 23135 raft_consensus.cc:2229] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.532272 23135 raft_consensus.cc:2258] T 6160343c1d3e4920801e9a06b7355f1f P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.533771 23135 tablet_replica.cc:330] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11: stopping tablet replica
I20260924 14:27:30.534227 23135 raft_consensus.cc:2229] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:27:30.534988 23135 raft_consensus.cc:2258] T bea8bfb3513f4b5885d3e801189e2848 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.536541 23135 tablet_replica.cc:330] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11: stopping tablet replica
I20260924 14:27:30.536984 23135 raft_consensus.cc:2229] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.537406 23135 raft_consensus.cc:2258] T 884a631342f040a2b7babf614155f1a5 P 78d920610cdd4a2bbe2dde49988fbb11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.539000 23135 tablet_replica.cc:330] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11: stopping tablet replica
I20260924 14:27:30.539433 23135 raft_consensus.cc:2229] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:27:30.540288 23135 raft_consensus.cc:2258] T 496d8e256cf2413f8bf270323313d891 P 78d920610cdd4a2bbe2dde49988fbb11 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260924 14:27:30.626413 23384 consensus_peers.cc:487] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 -> Peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Couldn't send request to peer 78d920610cdd4a2bbe2dde49988fbb11. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:35195: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:27:30.637660 23135 tablet_server.cc:194] TabletServer@127.22.151.193:0 shutdown complete.
I20260924 14:27:30.656701 23135 tablet_server.cc:178] TabletServer@127.22.151.194:0 shutting down...
I20260924 14:27:30.684887 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:27:30.685564 23135 tablet_replica.cc:330] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707: stopping tablet replica
I20260924 14:27:30.686108 23135 raft_consensus.cc:2229] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.686587 23135 raft_consensus.cc:2258] T ed1c48c6a1254b5685577a524c0ca523 P 278ba036c8a04b8ba9f143ffdeef6707 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.688313 23135 tablet_replica.cc:330] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707: stopping tablet replica
I20260924 14:27:30.688786 23135 raft_consensus.cc:2229] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:27:30.689519 23135 raft_consensus.cc:2258] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.691150 23135 tablet_replica.cc:330] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707: stopping tablet replica
I20260924 14:27:30.691695 23135 raft_consensus.cc:2229] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:27:30.692384 23135 raft_consensus.cc:2258] T 5b9cfc0424a8429ebd5886060eff5bde P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.694202 23135 tablet_replica.cc:330] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707: stopping tablet replica
I20260924 14:27:30.694787 23135 raft_consensus.cc:2229] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.695338 23135 raft_consensus.cc:2258] T 884a631342f040a2b7babf614155f1a5 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.698114 23135 tablet_replica.cc:330] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707: stopping tablet replica
I20260924 14:27:30.698810 23135 raft_consensus.cc:2229] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.699368 23135 raft_consensus.cc:2258] T bea8bfb3513f4b5885d3e801189e2848 P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:27:30.699337 23384 consensus_peers.cc:487] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 -> Peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Couldn't send request to peer 78d920610cdd4a2bbe2dde49988fbb11. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:35195: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:27:30.701757 23135 tablet_replica.cc:330] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707: stopping tablet replica
W20260924 14:27:30.701823 23310 consensus_peers.cc:487] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 -> Peer 78d920610cdd4a2bbe2dde49988fbb11 (127.22.151.193:35195): Couldn't send request to peer 78d920610cdd4a2bbe2dde49988fbb11. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:35195: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:27:30.702473 23135 raft_consensus.cc:2229] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.702889 23135 raft_consensus.cc:2258] T 6160343c1d3e4920801e9a06b7355f1f P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.704510 23135 tablet_replica.cc:330] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707: stopping tablet replica
I20260924 14:27:30.704979 23135 raft_consensus.cc:2229] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.705379 23135 raft_consensus.cc:2258] T 496d8e256cf2413f8bf270323313d891 P 278ba036c8a04b8ba9f143ffdeef6707 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.706883 23135 tablet_replica.cc:330] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707: stopping tablet replica
I20260924 14:27:30.707346 23135 raft_consensus.cc:2229] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:27:30.708120 23135 raft_consensus.cc:2258] T fd806eef81974e49811ad08d74410fdb P 278ba036c8a04b8ba9f143ffdeef6707 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.709733 23135 tablet_replica.cc:330] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707: stopping tablet replica
I20260924 14:27:30.710251 23135 raft_consensus.cc:2229] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.710705 23135 raft_consensus.cc:2258] T aa058204bc8d43e8893ef87079f0e38a P 278ba036c8a04b8ba9f143ffdeef6707 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.740480 23135 tablet_server.cc:194] TabletServer@127.22.151.194:0 shutdown complete.
I20260924 14:27:30.760893 23135 tablet_server.cc:178] TabletServer@127.22.151.195:0 shutting down...
I20260924 14:27:30.792510 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:27:30.793210 23135 tablet_replica.cc:330] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562: stopping tablet replica
I20260924 14:27:30.793864 23135 raft_consensus.cc:2229] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 5 LEADER]: Raft consensus shutting down.
I20260924 14:27:30.794688 23135 raft_consensus.cc:2258] T ed1c48c6a1254b5685577a524c0ca523 P 2fc7e85a970e4997926a2f2b13bfd562 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.796495 23135 tablet_replica.cc:330] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562: stopping tablet replica
I20260924 14:27:30.797075 23135 raft_consensus.cc:2229] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:27:30.797881 23135 raft_consensus.cc:2258] T 884a631342f040a2b7babf614155f1a5 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.799407 23135 tablet_replica.cc:330] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562: stopping tablet replica
I20260924 14:27:30.799952 23135 raft_consensus.cc:2229] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:27:30.800680 23135 raft_consensus.cc:2258] T 6160343c1d3e4920801e9a06b7355f1f P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.802233 23135 tablet_replica.cc:330] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562: stopping tablet replica
I20260924 14:27:30.802717 23135 raft_consensus.cc:2229] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.803126 23135 raft_consensus.cc:2258] T bea8bfb3513f4b5885d3e801189e2848 P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.804692 23135 tablet_replica.cc:330] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562: stopping tablet replica
I20260924 14:27:30.805184 23135 raft_consensus.cc:2229] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.805622 23135 raft_consensus.cc:2258] T 496d8e256cf2413f8bf270323313d891 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.807085 23135 tablet_replica.cc:330] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562: stopping tablet replica
I20260924 14:27:30.807602 23135 raft_consensus.cc:2229] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.808066 23135 raft_consensus.cc:2258] T 5b9cfc0424a8429ebd5886060eff5bde P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.809504 23135 tablet_replica.cc:330] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562: stopping tablet replica
I20260924 14:27:30.809973 23135 raft_consensus.cc:2229] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.810407 23135 raft_consensus.cc:2258] T ca20c5eab35d4b9a8f3945f4e9bf5815 P 2fc7e85a970e4997926a2f2b13bfd562 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.811978 23135 tablet_replica.cc:330] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562: stopping tablet replica
I20260924 14:27:30.812407 23135 raft_consensus.cc:2229] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.812790 23135 raft_consensus.cc:2258] T fd806eef81974e49811ad08d74410fdb P 2fc7e85a970e4997926a2f2b13bfd562 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.814289 23135 tablet_replica.cc:330] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562: stopping tablet replica
I20260924 14:27:30.814698 23135 raft_consensus.cc:2229] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:30.815105 23135 raft_consensus.cc:2258] T aa058204bc8d43e8893ef87079f0e38a P 2fc7e85a970e4997926a2f2b13bfd562 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:30.858251 23135 tablet_server.cc:194] TabletServer@127.22.151.195:0 shutdown complete.
I20260924 14:27:30.875577 23135 master.cc:537] Master@127.22.151.254:38707 shutting down...
I20260924 14:27:31.005210 23135 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:27:31.005849 23135 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:31.006201 23135 tablet_replica.cc:330] T 00000000000000000000000000000000 P c924d2ec9bb24ae999d74c8a3a3e3bfa: stopping tablet replica
I20260924 14:27:31.025048 23135 master.cc:552] Master@127.22.151.254:38707 shutdown complete.
278ba036c8a04b8ba9f143ffdeef6707  4
2fc7e85a970e4997926a2f2b13bfd562  1
78d920610cdd4a2bbe2dde49988fbb11  4
278ba036c8a04b8ba9f143ffdeef6707  3
2fc7e85a970e4997926a2f2b13bfd562  3
78d920610cdd4a2bbe2dde49988fbb11  3
278ba036c8a04b8ba9f143ffdeef6707  8
2fc7e85a970e4997926a2f2b13bfd562  1
78d920610cdd4a2bbe2dde49988fbb11  0
278ba036c8a04b8ba9f143ffdeef6707  3
2fc7e85a970e4997926a2f2b13bfd562  3
78d920610cdd4a2bbe2dde49988fbb11  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31126 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260924 14:27:31.065801 23135 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.22.151.254:46333
I20260924 14:27:31.066874 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:27:31.072546 23655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:31.073117 23656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:31.073851 23658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:27:31.074155 23135 server_base.cc:1034] running on GCE node
I20260924 14:27:31.075286 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:27:31.075810 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:27:31.076023 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260051075984 us; error 0 us; skew 500 ppm
I20260924 14:27:31.076570 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:27:31.079167 23135 webserver.cc:460] Webserver started at http://127.22.151.254:34941/ using document root <none> and password file <none>
I20260924 14:27:31.079694 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:27:31.079883 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:27:31.080189 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:27:31.081388 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/master-0-root/instance:
uuid: "bfaa0f81c05c413a87b0d9a75b8c7191"
format_stamp: "Formatted at 2026-09-24 14:27:31 on dist-test-slave-pm5x"
I20260924 14:27:31.085966 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260924 14:27:31.089359 23663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:31.090281 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260924 14:27:31.090533 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/master-0-root
uuid: "bfaa0f81c05c413a87b0d9a75b8c7191"
format_stamp: "Formatted at 2026-09-24 14:27:31 on dist-test-slave-pm5x"
I20260924 14:27:31.090773 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:27:31.117691 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:27:31.118939 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:27:31.156867 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.254:46333
I20260924 14:27:31.156962 23714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.254:46333 every 8 connection(s)
I20260924 14:27:31.160636 23715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.169858 23715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191: Bootstrap starting.
I20260924 14:27:31.174373 23715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:31.178550 23715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191: No bootstrap required, opened a new log
I20260924 14:27:31.180509 23715 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa0f81c05c413a87b0d9a75b8c7191" member_type: VOTER }
I20260924 14:27:31.180994 23715 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:31.181245 23715 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfaa0f81c05c413a87b0d9a75b8c7191, State: Initialized, Role: FOLLOWER
I20260924 14:27:31.181783 23715 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa0f81c05c413a87b0d9a75b8c7191" member_type: VOTER }
I20260924 14:27:31.182220 23715 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 14:27:31.182440 23715 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 14:27:31.182758 23715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:31.186510 23715 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa0f81c05c413a87b0d9a75b8c7191" member_type: VOTER }
I20260924 14:27:31.187039 23715 leader_election.cc:302] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfaa0f81c05c413a87b0d9a75b8c7191; no voters: 
I20260924 14:27:31.188144 23715 leader_election.cc:288] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260924 14:27:31.188599 23718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:31.189878 23718 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 1 LEADER]: Becoming Leader. State: Replica: bfaa0f81c05c413a87b0d9a75b8c7191, State: Running, Role: LEADER
I20260924 14:27:31.190564 23718 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa0f81c05c413a87b0d9a75b8c7191" member_type: VOTER }
I20260924 14:27:31.191149 23715 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 14:27:31.196657 23719 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfaa0f81c05c413a87b0d9a75b8c7191" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa0f81c05c413a87b0d9a75b8c7191" member_type: VOTER } }
I20260924 14:27:31.196797 23720 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfaa0f81c05c413a87b0d9a75b8c7191. Latest consensus state: current_term: 1 leader_uuid: "bfaa0f81c05c413a87b0d9a75b8c7191" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaa0f81c05c413a87b0d9a75b8c7191" member_type: VOTER } }
I20260924 14:27:31.197598 23720 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [sys.catalog]: This master's current role is: LEADER
I20260924 14:27:31.197561 23719 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [sys.catalog]: This master's current role is: LEADER
I20260924 14:27:31.201007 23726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 14:27:31.206704 23726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 14:27:31.209596 23135 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 14:27:31.216141 23726 catalog_manager.cc:1347] Generated new cluster ID: fecbe0f9afbe41f8b0d2d4201198a249
I20260924 14:27:31.216470 23726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 14:27:31.250653 23726 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 14:27:31.252280 23726 catalog_manager.cc:1504] Loading token signing keys...
I20260924 14:27:31.269919 23726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191: Generated new TSK 0
I20260924 14:27:31.270594 23726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 14:27:31.276135 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:27:31.282404 23736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:31.283573 23737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:31.285127 23739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:27:31.286314 23135 server_base.cc:1034] running on GCE node
I20260924 14:27:31.287197 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:27:31.287411 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:27:31.287626 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260051287608 us; error 0 us; skew 500 ppm
I20260924 14:27:31.288149 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:27:31.290499 23135 webserver.cc:460] Webserver started at http://127.22.151.193:34545/ using document root <none> and password file <none>
I20260924 14:27:31.290998 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:27:31.291252 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:27:31.291591 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:27:31.293006 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-0-root/instance:
uuid: "5b18601c6e1f45358f9a1ad392cff957"
format_stamp: "Formatted at 2026-09-24 14:27:31 on dist-test-slave-pm5x"
I20260924 14:27:31.297523 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260924 14:27:31.300721 23744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:31.301563 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260924 14:27:31.301872 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-0-root
uuid: "5b18601c6e1f45358f9a1ad392cff957"
format_stamp: "Formatted at 2026-09-24 14:27:31 on dist-test-slave-pm5x"
I20260924 14:27:31.302284 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:27:31.319593 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:27:31.320990 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:27:31.322491 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:27:31.324820 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:27:31.324998 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:31.325273 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:27:31.325448 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:31.366611 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.193:40675
I20260924 14:27:31.366691 23806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.193:40675 every 8 connection(s)
I20260924 14:27:31.371588 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:27:31.379386 23811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:31.381749 23812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:31.382284 23814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:27:31.382542 23135 server_base.cc:1034] running on GCE node
I20260924 14:27:31.383610 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:27:31.383821 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:27:31.383983 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260051383968 us; error 0 us; skew 500 ppm
I20260924 14:27:31.384481 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:27:31.387327 23135 webserver.cc:460] Webserver started at http://127.22.151.194:40739/ using document root <none> and password file <none>
I20260924 14:27:31.387938 23807 heartbeater.cc:347] Connected to a master server at 127.22.151.254:46333
I20260924 14:27:31.388027 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:27:31.388360 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:27:31.388407 23807 heartbeater.cc:464] Registering TS with master...
I20260924 14:27:31.388749 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:27:31.389338 23807 heartbeater.cc:511] Master 127.22.151.254:46333 requested a full tablet report, sending...
I20260924 14:27:31.390260 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-1-root/instance:
uuid: "7744d9495c594ba59d47a45af215ff8f"
format_stamp: "Formatted at 2026-09-24 14:27:31 on dist-test-slave-pm5x"
I20260924 14:27:31.391755 23680 ts_manager.cc:194] Registered new tserver with Master: 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675)
I20260924 14:27:31.393993 23680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39484
I20260924 14:27:31.396088 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260924 14:27:31.400185 23819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:31.401077 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260924 14:27:31.401396 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-1-root
uuid: "7744d9495c594ba59d47a45af215ff8f"
format_stamp: "Formatted at 2026-09-24 14:27:31 on dist-test-slave-pm5x"
I20260924 14:27:31.401688 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:27:31.413300 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:27:31.414507 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:27:31.416198 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:27:31.418433 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:27:31.418663 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:31.418908 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:27:31.419071 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:31.460506 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.194:46295
I20260924 14:27:31.460614 23881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.194:46295 every 8 connection(s)
I20260924 14:27:31.465212 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:27:31.472980 23885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:27:31.473847 23886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:27:31.476094 23135 server_base.cc:1034] running on GCE node
W20260924 14:27:31.476963 23888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:27:31.478154 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:27:31.478456 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:27:31.478652 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260051478639 us; error 0 us; skew 500 ppm
I20260924 14:27:31.479229 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:27:31.480489 23882 heartbeater.cc:347] Connected to a master server at 127.22.151.254:46333
I20260924 14:27:31.480933 23882 heartbeater.cc:464] Registering TS with master...
I20260924 14:27:31.481714 23882 heartbeater.cc:511] Master 127.22.151.254:46333 requested a full tablet report, sending...
I20260924 14:27:31.482210 23135 webserver.cc:460] Webserver started at http://127.22.151.195:39009/ using document root <none> and password file <none>
I20260924 14:27:31.482846 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:27:31.483036 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:27:31.483357 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:27:31.484122 23680 ts_manager.cc:194] Registered new tserver with Master: 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:31.484742 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-2-root/instance:
uuid: "bc8c63a642ab43efa3b4829ea4294d37"
format_stamp: "Formatted at 2026-09-24 14:27:31 on dist-test-slave-pm5x"
I20260924 14:27:31.485731 23680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39492
I20260924 14:27:31.490003 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260924 14:27:31.493650 23893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:31.494391 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260924 14:27:31.494676 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-2-root
uuid: "bc8c63a642ab43efa3b4829ea4294d37"
format_stamp: "Formatted at 2026-09-24 14:27:31 on dist-test-slave-pm5x"
I20260924 14:27:31.494949 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1790260019751168-23135-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:27:31.525230 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:27:31.526443 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:27:31.527971 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:27:31.530222 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:27:31.530404 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:31.530675 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:27:31.530807 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:27:31.570765 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.195:41467
I20260924 14:27:31.570876 23955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.195:41467 every 8 connection(s)
I20260924 14:27:31.586447 23956 heartbeater.cc:347] Connected to a master server at 127.22.151.254:46333
I20260924 14:27:31.586849 23956 heartbeater.cc:464] Registering TS with master...
I20260924 14:27:31.587635 23956 heartbeater.cc:511] Master 127.22.151.254:46333 requested a full tablet report, sending...
I20260924 14:27:31.589656 23680 ts_manager.cc:194] Registered new tserver with Master: bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:31.589748 23135 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016059385s
I20260924 14:27:31.591548 23680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39494
I20260924 14:27:31.592118 23135 test_util.cc:268] Using random seed: 1718502302
I20260924 14:27:31.619776 23680 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39510:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260924 14:27:31.622746 23680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 14:27:31.692868 23847 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4565b47ba44cc09bc0850742b28c57 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260924 14:27:31.695581 23847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4565b47ba44cc09bc0850742b28c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.701154 23846 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc8a59ba9b94ffb9cb345e3eef41700 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260924 14:27:31.702119 23845 tablet_service.cc:1467] Processing CreateTablet for tablet 811a523968ff4d059b0f66e8ee15b059 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260924 14:27:31.702689 23846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc8a59ba9b94ffb9cb345e3eef41700. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.703513 23845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811a523968ff4d059b0f66e8ee15b059. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.709342 23844 tablet_service.cc:1467] Processing CreateTablet for tablet 458a9bce2fb24e0d81141d72eef8e669 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260924 14:27:31.711175 23844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458a9bce2fb24e0d81141d72eef8e669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.710975 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 6528eea31cb945f89f6bbb6507d2b57e (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260924 14:27:31.712517 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6528eea31cb945f89f6bbb6507d2b57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.712445 23841 tablet_service.cc:1467] Processing CreateTablet for tablet f05549cf4bfc4950856a68b77823b204 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260924 14:27:31.713755 23841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05549cf4bfc4950856a68b77823b204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.713580 23839 tablet_service.cc:1467] Processing CreateTablet for tablet aac851188d724766ac3d80b1a1a6b98b (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260924 14:27:31.714840 23839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac851188d724766ac3d80b1a1a6b98b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.717533 23838 tablet_service.cc:1467] Processing CreateTablet for tablet 1ca1ed23241b46cb8c65b57f65f85588 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260924 14:27:31.718806 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ca1ed23241b46cb8c65b57f65f85588. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.720577 23840 tablet_service.cc:1467] Processing CreateTablet for tablet cb58b7233f5a44daaf2209d0394cd248 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260924 14:27:31.721853 23840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb58b7233f5a44daaf2209d0394cd248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.724745 23842 tablet_service.cc:1467] Processing CreateTablet for tablet 141131b2a0824d3cb3c82671609b0e19 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260924 14:27:31.726461 23842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 141131b2a0824d3cb3c82671609b0e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.751042 23970 tablet_bootstrap.cc:492] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f: Bootstrap starting.
I20260924 14:27:31.756422 23921 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4565b47ba44cc09bc0850742b28c57 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260924 14:27:31.757822 23921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4565b47ba44cc09bc0850742b28c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.760684 23915 tablet_service.cc:1467] Processing CreateTablet for tablet f05549cf4bfc4950856a68b77823b204 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260924 14:27:31.760692 23920 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc8a59ba9b94ffb9cb345e3eef41700 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260924 14:27:31.762156 23915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05549cf4bfc4950856a68b77823b204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.762853 23920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc8a59ba9b94ffb9cb345e3eef41700. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.764328 23916 tablet_service.cc:1467] Processing CreateTablet for tablet 141131b2a0824d3cb3c82671609b0e19 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260924 14:27:31.765576 23916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 141131b2a0824d3cb3c82671609b0e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.768719 23917 tablet_service.cc:1467] Processing CreateTablet for tablet 6528eea31cb945f89f6bbb6507d2b57e (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260924 14:27:31.770009 23917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6528eea31cb945f89f6bbb6507d2b57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.772607 23918 tablet_service.cc:1467] Processing CreateTablet for tablet 458a9bce2fb24e0d81141d72eef8e669 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260924 14:27:31.774112 23918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458a9bce2fb24e0d81141d72eef8e669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.777154 23919 tablet_service.cc:1467] Processing CreateTablet for tablet 811a523968ff4d059b0f66e8ee15b059 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260924 14:27:31.778963 23919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811a523968ff4d059b0f66e8ee15b059. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.783308 23772 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4565b47ba44cc09bc0850742b28c57 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260924 14:27:31.783926 23770 tablet_service.cc:1467] Processing CreateTablet for tablet 811a523968ff4d059b0f66e8ee15b059 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260924 14:27:31.784633 23772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4565b47ba44cc09bc0850742b28c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.785240 23770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811a523968ff4d059b0f66e8ee15b059. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.788646 23771 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc8a59ba9b94ffb9cb345e3eef41700 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260924 14:27:31.789247 23769 tablet_service.cc:1467] Processing CreateTablet for tablet 458a9bce2fb24e0d81141d72eef8e669 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260924 14:27:31.790019 23771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc8a59ba9b94ffb9cb345e3eef41700. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.791522 23768 tablet_service.cc:1467] Processing CreateTablet for tablet 6528eea31cb945f89f6bbb6507d2b57e (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260924 14:27:31.792797 23768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6528eea31cb945f89f6bbb6507d2b57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.792623 23914 tablet_service.cc:1467] Processing CreateTablet for tablet cb58b7233f5a44daaf2209d0394cd248 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260924 14:27:31.793880 23914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb58b7233f5a44daaf2209d0394cd248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.797070 23913 tablet_service.cc:1467] Processing CreateTablet for tablet 1ca1ed23241b46cb8c65b57f65f85588 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260924 14:27:31.798347 23913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ca1ed23241b46cb8c65b57f65f85588. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.802237 23769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458a9bce2fb24e0d81141d72eef8e669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.804503 23766 tablet_service.cc:1467] Processing CreateTablet for tablet f05549cf4bfc4950856a68b77823b204 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260924 14:27:31.806631 23766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05549cf4bfc4950856a68b77823b204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.812496 23767 tablet_service.cc:1467] Processing CreateTablet for tablet 141131b2a0824d3cb3c82671609b0e19 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260924 14:27:31.813992 23767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 141131b2a0824d3cb3c82671609b0e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.819870 23971 tablet_bootstrap.cc:492] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37: Bootstrap starting.
I20260924 14:27:31.819573 23763 tablet_service.cc:1467] Processing CreateTablet for tablet 1ca1ed23241b46cb8c65b57f65f85588 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260924 14:27:31.821018 23763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ca1ed23241b46cb8c65b57f65f85588. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.817385 23765 tablet_service.cc:1467] Processing CreateTablet for tablet cb58b7233f5a44daaf2209d0394cd248 (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260924 14:27:31.824849 23765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb58b7233f5a44daaf2209d0394cd248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.824966 23971 tablet_bootstrap.cc:654] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:31.819600 23764 tablet_service.cc:1467] Processing CreateTablet for tablet aac851188d724766ac3d80b1a1a6b98b (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260924 14:27:31.832322 23764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac851188d724766ac3d80b1a1a6b98b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.852032 23920 tablet_service.cc:1467] Processing CreateTablet for tablet aac851188d724766ac3d80b1a1a6b98b (DEFAULT_TABLE table=test-workload [id=2b3c6a0b3c8e48a188f57e637133881a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260924 14:27:31.853494 23920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac851188d724766ac3d80b1a1a6b98b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:27:31.893879 23970 tablet_bootstrap.cc:654] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:31.941758 23973 tablet_bootstrap.cc:492] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957: Bootstrap starting.
I20260924 14:27:31.945752 23970 tablet_bootstrap.cc:492] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f: No bootstrap required, opened a new log
I20260924 14:27:31.946553 23970 ts_tablet_manager.cc:1397] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f: Time spent bootstrapping tablet: real 0.196s	user 0.010s	sys 0.004s
I20260924 14:27:31.948529 23973 tablet_bootstrap.cc:654] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:31.951210 23971 tablet_bootstrap.cc:492] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37: No bootstrap required, opened a new log
I20260924 14:27:31.951061 23970 raft_consensus.cc:348] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:31.951733 23971 ts_tablet_manager.cc:1397] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37: Time spent bootstrapping tablet: real 0.132s	user 0.030s	sys 0.082s
I20260924 14:27:31.951891 23970 raft_consensus.cc:374] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:31.952206 23970 raft_consensus.cc:729] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Initialized, Role: FOLLOWER
I20260924 14:27:31.953076 23970 consensus_queue.cc:260] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:31.954350 23973 tablet_bootstrap.cc:492] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957: No bootstrap required, opened a new log
I20260924 14:27:31.954895 23973 ts_tablet_manager.cc:1397] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957: Time spent bootstrapping tablet: real 0.013s	user 0.003s	sys 0.008s
I20260924 14:27:31.955062 23971 raft_consensus.cc:348] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:31.955919 23970 ts_tablet_manager.cc:1428] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260924 14:27:31.955963 23971 raft_consensus.cc:374] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:31.955858 23882 heartbeater.cc:503] Master 127.22.151.254:46333 was elected leader, sending a full tablet report...
I20260924 14:27:31.956367 23971 raft_consensus.cc:729] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Initialized, Role: FOLLOWER
I20260924 14:27:31.957073 23970 tablet_bootstrap.cc:492] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f: Bootstrap starting.
I20260924 14:27:31.957331 23971 consensus_queue.cc:260] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:31.957729 23973 raft_consensus.cc:348] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:31.958487 23973 raft_consensus.cc:374] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:31.958760 23973 raft_consensus.cc:729] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Initialized, Role: FOLLOWER
I20260924 14:27:31.962553 23956 heartbeater.cc:503] Master 127.22.151.254:46333 was elected leader, sending a full tablet report...
I20260924 14:27:31.963142 23971 ts_tablet_manager.cc:1428] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.002s
I20260924 14:27:31.963267 23970 tablet_bootstrap.cc:654] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:31.964447 23971 tablet_bootstrap.cc:492] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37: Bootstrap starting.
I20260924 14:27:31.970824 23971 tablet_bootstrap.cc:654] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:31.959558 23973 consensus_queue.cc:260] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:31.974664 23807 heartbeater.cc:503] Master 127.22.151.254:46333 was elected leader, sending a full tablet report...
I20260924 14:27:31.983016 23970 tablet_bootstrap.cc:492] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f: No bootstrap required, opened a new log
I20260924 14:27:31.983255 23971 tablet_bootstrap.cc:492] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37: No bootstrap required, opened a new log
I20260924 14:27:31.983561 23970 ts_tablet_manager.cc:1397] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.011s
I20260924 14:27:31.983763 23971 ts_tablet_manager.cc:1397] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.006s
I20260924 14:27:31.983976 23973 ts_tablet_manager.cc:1428] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260924 14:27:31.985697 23973 tablet_bootstrap.cc:492] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957: Bootstrap starting.
I20260924 14:27:31.986605 23970 raft_consensus.cc:348] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:31.986958 23979 raft_consensus.cc:482] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:31.988231 23971 raft_consensus.cc:348] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:31.988029 23979 raft_consensus.cc:504] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:31.989048 23971 raft_consensus.cc:374] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:31.989374 23971 raft_consensus.cc:729] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Initialized, Role: FOLLOWER
I20260924 14:27:31.989456 23970 raft_consensus.cc:374] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:31.990700 23970 raft_consensus.cc:729] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Initialized, Role: FOLLOWER
I20260924 14:27:31.992965 23970 consensus_queue.cc:260] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:31.996246 23979 leader_election.cc:288] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:31.996948 23970 ts_tablet_manager.cc:1428] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260924 14:27:31.997839 23970 tablet_bootstrap.cc:492] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f: Bootstrap starting.
I20260924 14:27:32.003799 23970 tablet_bootstrap.cc:654] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:31.990139 23971 consensus_queue.cc:260] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.022619 23973 tablet_bootstrap.cc:654] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.022766 23971 ts_tablet_manager.cc:1428] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.000s
I20260924 14:27:32.023790 23971 tablet_bootstrap.cc:492] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37: Bootstrap starting.
I20260924 14:27:32.038276 23973 tablet_bootstrap.cc:492] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957: No bootstrap required, opened a new log
I20260924 14:27:32.038892 23973 ts_tablet_manager.cc:1397] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.000s
I20260924 14:27:32.041026 23971 tablet_bootstrap.cc:654] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.041556 23973 raft_consensus.cc:348] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.042331 23973 raft_consensus.cc:374] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.042639 23973 raft_consensus.cc:729] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.043499 23973 consensus_queue.cc:260] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.045933 23973 ts_tablet_manager.cc:1428] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:27:32.046705 23970 tablet_bootstrap.cc:492] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f: No bootstrap required, opened a new log
I20260924 14:27:32.047276 23970 ts_tablet_manager.cc:1397] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.025s
I20260924 14:27:32.050436 23970 raft_consensus.cc:348] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.051359 23970 raft_consensus.cc:374] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.051756 23970 raft_consensus.cc:729] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.052600 23970 consensus_queue.cc:260] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.046797 23973 tablet_bootstrap.cc:492] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957: Bootstrap starting.
I20260924 14:27:32.060606 23970 ts_tablet_manager.cc:1428] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260924 14:27:32.061615 23973 tablet_bootstrap.cc:654] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.069432 23970 tablet_bootstrap.cc:492] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f: Bootstrap starting.
I20260924 14:27:32.078013 23970 tablet_bootstrap.cc:654] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.081569 23973 tablet_bootstrap.cc:492] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957: No bootstrap required, opened a new log
I20260924 14:27:32.082115 23973 ts_tablet_manager.cc:1397] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.001s
I20260924 14:27:32.083489 23976 raft_consensus.cc:482] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.084177 23976 raft_consensus.cc:504] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.085722 23973 raft_consensus.cc:348] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.086748 23973 raft_consensus.cc:374] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.087136 23973 raft_consensus.cc:729] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.090104 23976 leader_election.cc:288] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.088093 23973 consensus_queue.cc:260] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.099753 23979 raft_consensus.cc:482] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.100445 23979 raft_consensus.cc:504] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.106181 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.106611 23856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.109524 23973 ts_tablet_manager.cc:1428] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.004s
I20260924 14:27:32.110785 23973 tablet_bootstrap.cc:492] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957: Bootstrap starting.
W20260924 14:27:32.112135 23747 leader_election.cc:341] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.112268 23979 leader_election.cc:288] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
W20260924 14:27:32.112877 23747 leader_election.cc:341] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.118983 23973 tablet_bootstrap.cc:654] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.132674 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.134306 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
W20260924 14:27:32.136217 23745 leader_election.cc:341] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.136698 23745 leader_election.cc:302] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957; no voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:32.138069 23973 tablet_bootstrap.cc:492] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957: No bootstrap required, opened a new log
W20260924 14:27:32.138125 23745 leader_election.cc:341] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.138847 23973 ts_tablet_manager.cc:1397] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.005s
I20260924 14:27:32.138772 23745 leader_election.cc:302] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957; no voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:32.139752 23989 raft_consensus.cc:2726] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:32.140363 23989 raft_consensus.cc:2726] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:32.142462 23976 raft_consensus.cc:482] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.142526 23973 raft_consensus.cc:348] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.142961 23976 raft_consensus.cc:504] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.143466 23971 tablet_bootstrap.cc:492] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37: No bootstrap required, opened a new log
I20260924 14:27:32.143605 23973 raft_consensus.cc:374] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.143929 23971 ts_tablet_manager.cc:1397] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37: Time spent bootstrapping tablet: real 0.120s	user 0.054s	sys 0.037s
I20260924 14:27:32.144053 23973 raft_consensus.cc:729] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.144981 23976 leader_election.cc:288] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.146759 23971 raft_consensus.cc:348] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.147706 23971 raft_consensus.cc:374] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.148025 23971 raft_consensus.cc:729] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.144990 23973 consensus_queue.cc:260] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.152153 23971 consensus_queue.cc:260] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.154729 23971 ts_tablet_manager.cc:1428] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260924 14:27:32.155954 23971 tablet_bootstrap.cc:492] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37: Bootstrap starting.
I20260924 14:27:32.154743 23973 ts_tablet_manager.cc:1428] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260924 14:27:32.162894 23973 tablet_bootstrap.cc:492] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957: Bootstrap starting.
I20260924 14:27:32.171473 23970 tablet_bootstrap.cc:492] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f: No bootstrap required, opened a new log
I20260924 14:27:32.171301 23971 tablet_bootstrap.cc:654] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.171797 23977 raft_consensus.cc:482] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.172322 23970 ts_tablet_manager.cc:1397] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f: Time spent bootstrapping tablet: real 0.111s	user 0.030s	sys 0.040s
I20260924 14:27:32.172614 23977 raft_consensus.cc:504] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.174649 23973 tablet_bootstrap.cc:654] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.175642 23970 raft_consensus.cc:348] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.176563 23970 raft_consensus.cc:374] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.176935 23970 raft_consensus.cc:729] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.177747 23970 consensus_queue.cc:260] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.179275 23971 tablet_bootstrap.cc:492] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37: No bootstrap required, opened a new log
I20260924 14:27:32.180032 23971 ts_tablet_manager.cc:1397] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.004s
I20260924 14:27:32.180357 23970 ts_tablet_manager.cc:1428] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260924 14:27:32.181636 23977 leader_election.cc:288] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:32.182700 23970 tablet_bootstrap.cc:492] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f: Bootstrap starting.
I20260924 14:27:32.182611 23971 raft_consensus.cc:348] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.183382 23971 raft_consensus.cc:374] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.183692 23971 raft_consensus.cc:729] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.184372 23971 consensus_queue.cc:260] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.186600 23971 ts_tablet_manager.cc:1428] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260924 14:27:32.187520 23971 tablet_bootstrap.cc:492] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37: Bootstrap starting.
I20260924 14:27:32.194643 23971 tablet_bootstrap.cc:654] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.206871 23973 tablet_bootstrap.cc:492] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957: No bootstrap required, opened a new log
I20260924 14:27:32.207356 23973 ts_tablet_manager.cc:1397] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957: Time spent bootstrapping tablet: real 0.045s	user 0.006s	sys 0.003s
I20260924 14:27:32.208171 23970 tablet_bootstrap.cc:654] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.209851 23781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6528eea31cb945f89f6bbb6507d2b57e" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.210039 23973 raft_consensus.cc:348] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:32.207875 23782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.210752 23973 raft_consensus.cc:374] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.211061 23973 raft_consensus.cc:729] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Initialized, Role: FOLLOWER
W20260924 14:27:32.211193 23728 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260924 14:27:32.211858 23820 leader_election.cc:341] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.211802 23973 consensus_queue.cc:260] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
W20260924 14:27:32.212991 23820 leader_election.cc:341] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.216197 23973 ts_tablet_manager.cc:1428] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260924 14:27:32.217178 23973 tablet_bootstrap.cc:492] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957: Bootstrap starting.
I20260924 14:27:32.223047 23973 tablet_bootstrap.cc:654] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.230593 23782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
W20260924 14:27:32.233075 23894 leader_election.cc:341] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.233484 23979 raft_consensus.cc:482] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.234001 23979 raft_consensus.cc:504] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.235356 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.236034 23979 leader_election.cc:288] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.236492 23856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.237540 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.237927 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6528eea31cb945f89f6bbb6507d2b57e" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.238152 23857 raft_consensus.cc:2454] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 0.
W20260924 14:27:32.239032 23820 leader_election.cc:341] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.239472 23820 leader_election.cc:302] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f; no voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:32.239987 23747 leader_election.cc:302] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: 
I20260924 14:27:32.240561 23976 raft_consensus.cc:2726] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:32.240988 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
W20260924 14:27:32.241492 23820 leader_election.cc:341] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.241892 23820 leader_election.cc:302] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f; no voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37
W20260924 14:27:32.242246 23745 leader_election.cc:341] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.242884 23976 raft_consensus.cc:2726] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:32.243044 23979 raft_consensus.cc:2781] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:32.243386 23979 raft_consensus.cc:482] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:32.243738 23979 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:27:32.243959 23896 leader_election.cc:341] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.244375 23896 leader_election.cc:302] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc8c63a642ab43efa3b4829ea4294d37; no voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:32.245136 23977 raft_consensus.cc:2726] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:32.250280 23979 raft_consensus.cc:504] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.251714 23970 tablet_bootstrap.cc:492] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f: No bootstrap required, opened a new log
I20260924 14:27:32.252224 23970 ts_tablet_manager.cc:1397] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f: Time spent bootstrapping tablet: real 0.070s	user 0.024s	sys 0.020s
I20260924 14:27:32.253333 23971 tablet_bootstrap.cc:492] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37: No bootstrap required, opened a new log
I20260924 14:27:32.253193 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:32.253857 23857 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.254057 23971 ts_tablet_manager.cc:1397] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.007s
I20260924 14:27:32.255118 23970 raft_consensus.cc:348] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.255954 23970 raft_consensus.cc:374] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.256263 23970 raft_consensus.cc:729] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.257036 23970 consensus_queue.cc:260] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.259729 23971 raft_consensus.cc:348] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.260602 23971 raft_consensus.cc:374] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.260962 23971 raft_consensus.cc:729] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.261551 23857 raft_consensus.cc:2454] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 1.
I20260924 14:27:32.261834 23971 consensus_queue.cc:260] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.262758 23747 leader_election.cc:302] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: 
I20260924 14:27:32.263672 23989 raft_consensus.cc:2781] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:32.265265 23971 ts_tablet_manager.cc:1428] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260924 14:27:32.265817 23989 raft_consensus.cc:686] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 1 LEADER]: Becoming Leader. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:32.266578 23971 tablet_bootstrap.cc:492] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37: Bootstrap starting.
I20260924 14:27:32.266672 23989 consensus_queue.cc:237] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.268281 23970 ts_tablet_manager.cc:1428] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260924 14:27:32.272001 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:32.273416 23971 tablet_bootstrap.cc:654] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37: Neither blocks nor log segments found. Creating new log.
W20260924 14:27:32.273319 23745 leader_election.cc:341] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.274032 23979 leader_election.cc:288] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.274197 23970 tablet_bootstrap.cc:492] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f: Bootstrap starting.
I20260924 14:27:32.277633 23973 tablet_bootstrap.cc:492] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957: No bootstrap required, opened a new log
I20260924 14:27:32.278267 23973 ts_tablet_manager.cc:1397] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957: Time spent bootstrapping tablet: real 0.061s	user 0.008s	sys 0.002s
I20260924 14:27:32.281805 23970 tablet_bootstrap.cc:654] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.282035 23973 raft_consensus.cc:348] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.283052 23973 raft_consensus.cc:374] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.283404 23973 raft_consensus.cc:729] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.284412 23973 consensus_queue.cc:260] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.287828 23973 ts_tablet_manager.cc:1428] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260924 14:27:32.289041 23973 tablet_bootstrap.cc:492] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957: Bootstrap starting.
I20260924 14:27:32.290854 23679 catalog_manager.cc:5495] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "5b18601c6e1f45358f9a1ad392cff957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:32.298419 23973 tablet_bootstrap.cc:654] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.298318 23977 raft_consensus.cc:482] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.299139 23977 raft_consensus.cc:504] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:32.302357 23970 tablet_bootstrap.cc:492] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f: No bootstrap required, opened a new log
I20260924 14:27:32.302484 23971 tablet_bootstrap.cc:492] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37: No bootstrap required, opened a new log
I20260924 14:27:32.303071 23970 ts_tablet_manager.cc:1397] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.002s
I20260924 14:27:32.303071 23971 ts_tablet_manager.cc:1397] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37: Time spent bootstrapping tablet: real 0.037s	user 0.027s	sys 0.008s
I20260924 14:27:32.304431 23977 leader_election.cc:288] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675)
I20260924 14:27:32.303577 23782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.304651 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.305449 23857 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 0.
I20260924 14:27:32.305433 23782 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 0.
I20260924 14:27:32.306622 23896 leader_election.cc:302] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:32.307344 23977 raft_consensus.cc:2781] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:32.307895 23977 raft_consensus.cc:482] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:32.308343 23977 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.309321 23971 raft_consensus.cc:348] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.310086 23971 raft_consensus.cc:374] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.310433 23971 raft_consensus.cc:729] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.310102 23970 raft_consensus.cc:348] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.311165 23970 raft_consensus.cc:374] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.311609 23970 raft_consensus.cc:729] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.311241 23971 consensus_queue.cc:260] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.312500 23970 consensus_queue.cc:260] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.307557 23973 tablet_bootstrap.cc:492] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957: No bootstrap required, opened a new log
I20260924 14:27:32.316835 23973 ts_tablet_manager.cc:1397] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.001s
I20260924 14:27:32.316515 23977 raft_consensus.cc:504] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:32.319103 23970 ts_tablet_manager.cc:1428] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260924 14:27:32.319650 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:32.320164 23970 tablet_bootstrap.cc:492] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f: Bootstrap starting.
I20260924 14:27:32.320263 23857 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.327253 23970 tablet_bootstrap.cc:654] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.328325 23857 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 1.
I20260924 14:27:32.330745 23782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:32.331806 23782 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.341118 23973 raft_consensus.cc:348] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.342245 23973 raft_consensus.cc:374] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.342679 23973 raft_consensus.cc:729] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.343907 23896 leader_election.cc:302] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:32.342337 23977 leader_election.cc:288] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675)
I20260924 14:27:32.348078 23977 raft_consensus.cc:2781] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:32.348855 23977 raft_consensus.cc:686] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Becoming Leader. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:32.352415 23977 consensus_queue.cc:237] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:32.350966 23782 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 1.
I20260924 14:27:32.357386 23973 consensus_queue.cc:260] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.358417 23970 tablet_bootstrap.cc:492] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f: No bootstrap required, opened a new log
I20260924 14:27:32.360957 23970 ts_tablet_manager.cc:1397] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.020s
I20260924 14:27:32.364336 23970 raft_consensus.cc:348] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.365255 23973 ts_tablet_manager.cc:1428] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260924 14:27:32.365300 23970 raft_consensus.cc:374] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.367136 23970 raft_consensus.cc:729] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.367719 23973 tablet_bootstrap.cc:492] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957: Bootstrap starting.
I20260924 14:27:32.370047 23971 ts_tablet_manager.cc:1428] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37: Time spent starting tablet: real 0.066s	user 0.026s	sys 0.035s
I20260924 14:27:32.371871 23971 tablet_bootstrap.cc:492] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37: Bootstrap starting.
I20260924 14:27:32.373845 23989 raft_consensus.cc:482] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.374308 23989 raft_consensus.cc:504] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.376471 23989 leader_election.cc:288] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.378227 23971 tablet_bootstrap.cc:654] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.379319 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6528eea31cb945f89f6bbb6507d2b57e" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.379920 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6528eea31cb945f89f6bbb6507d2b57e" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.380525 23857 raft_consensus.cc:2454] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 0.
W20260924 14:27:32.380589 23745 leader_election.cc:341] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.380637 23970 consensus_queue.cc:260] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.381635 23976 raft_consensus.cc:482] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.378141 23973 tablet_bootstrap.cc:654] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.381997 23747 leader_election.cc:302] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:32.386308 23989 raft_consensus.cc:2781] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:32.382193 23976 raft_consensus.cc:504] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.386746 23989 raft_consensus.cc:482] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:32.387131 23989 raft_consensus.cc:3037] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.388702 23976 leader_election.cc:288] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.389231 23977 raft_consensus.cc:482] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.389690 23977 raft_consensus.cc:504] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.393328 23782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.394117 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.394243 23782 raft_consensus.cc:2454] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 0.
I20260924 14:27:32.394793 23930 raft_consensus.cc:2454] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 0.
I20260924 14:27:32.394675 23989 raft_consensus.cc:504] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.393368 23781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.395699 23894 leader_election.cc:302] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:32.396080 23820 leader_election.cc:302] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:32.396606 24006 raft_consensus.cc:2781] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:32.390574 23679 catalog_manager.cc:5495] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:32.396983 24006 raft_consensus.cc:482] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:32.397490 24006 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.397778 23977 leader_election.cc:288] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
W20260924 14:27:32.397066 23820 leader_election.cc:341] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.399592 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.400259 23857 raft_consensus.cc:2454] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 0.
I20260924 14:27:32.402343 23976 raft_consensus.cc:2781] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:32.402791 23976 raft_consensus.cc:482] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:32.403391 23976 raft_consensus.cc:3037] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.410740 23979 raft_consensus.cc:482] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.411312 23979 raft_consensus.cc:504] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.415911 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811a523968ff4d059b0f66e8ee15b059" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.416688 23930 raft_consensus.cc:2454] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 0.
I20260924 14:27:32.418011 23745 leader_election.cc:302] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:32.411626 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6528eea31cb945f89f6bbb6507d2b57e" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:32.422113 23999 raft_consensus.cc:2781] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:32.423153 23976 raft_consensus.cc:504] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.423681 23857 raft_consensus.cc:3037] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.415189 23979 leader_election.cc:288] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.423501 23856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811a523968ff4d059b0f66e8ee15b059" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.424342 24006 raft_consensus.cc:504] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.426386 24006 leader_election.cc:288] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:32.427150 23989 leader_election.cc:288] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
W20260924 14:27:32.428143 23747 leader_election.cc:341] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.429198 23781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:32.429582 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6528eea31cb945f89f6bbb6507d2b57e" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:32.429972 23781 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.430709 23971 tablet_bootstrap.cc:492] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37: No bootstrap required, opened a new log
W20260924 14:27:32.430869 23745 leader_election.cc:341] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.423573 23999 raft_consensus.cc:482] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:32.432209 23999 raft_consensus.cc:3037] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.431061 23857 raft_consensus.cc:2454] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 1.
I20260924 14:27:32.435357 23747 leader_election.cc:302] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:32.436460 23989 raft_consensus.cc:2781] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:32.437038 23989 raft_consensus.cc:686] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 1 LEADER]: Becoming Leader. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:32.437876 23989 consensus_queue.cc:237] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.439754 23856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:32.440368 23856 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.441699 23971 ts_tablet_manager.cc:1397] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.017s
I20260924 14:27:32.444747 24012 raft_consensus.cc:482] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.445006 24006 raft_consensus.cc:482] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.445232 24012 raft_consensus.cc:504] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.445458 24006 raft_consensus.cc:504] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.446905 23782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:32.447435 24006 leader_election.cc:288] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:32.448729 23782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca1ed23241b46cb8c65b57f65f85588" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.448974 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb58b7233f5a44daaf2209d0394cd248" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.449683 23780 raft_consensus.cc:2454] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 0.
I20260924 14:27:32.449339 23999 raft_consensus.cc:504] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
W20260924 14:27:32.450093 23894 leader_election.cc:341] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.447336 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:32.452646 23930 raft_consensus.cc:3037] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.452546 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca1ed23241b46cb8c65b57f65f85588" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.453189 23970 ts_tablet_manager.cc:1428] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f: Time spent starting tablet: real 0.092s	user 0.023s	sys 0.015s
I20260924 14:27:32.447302 24012 leader_election.cc:288] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.454347 23970 tablet_bootstrap.cc:492] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f: Bootstrap starting.
W20260924 14:27:32.455717 23896 leader_election.cc:341] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.455777 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb58b7233f5a44daaf2209d0394cd248" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.456266 23896 leader_election.cc:302] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc8c63a642ab43efa3b4829ea4294d37; no voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:32.457011 24006 raft_consensus.cc:2726] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:32.457264 23971 raft_consensus.cc:348] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
W20260924 14:27:32.457664 23820 leader_election.cc:341] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.458389 23820 leader_election.cc:302] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: 
W20260924 14:27:32.459292 23820 leader_election.cc:341] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.460139 24012 raft_consensus.cc:2781] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:32.460503 24012 raft_consensus.cc:482] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:32.460862 24012 raft_consensus.cc:3037] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.461416 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811a523968ff4d059b0f66e8ee15b059" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:32.463228 23970 tablet_bootstrap.cc:654] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f: Neither blocks nor log segments found. Creating new log.
W20260924 14:27:32.463503 23747 leader_election.cc:341] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.468631 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811a523968ff4d059b0f66e8ee15b059" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:32.469259 23931 raft_consensus.cc:3037] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.470039 24012 raft_consensus.cc:504] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.474720 23781 raft_consensus.cc:2454] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 1.
I20260924 14:27:32.475531 23999 leader_election.cc:288] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.476207 23894 leader_election.cc:302] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:32.477100 24006 raft_consensus.cc:2781] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:32.477625 24006 raft_consensus.cc:686] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Becoming Leader. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:32.478526 24006 consensus_queue.cc:237] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.479305 23971 raft_consensus.cc:374] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.479393 23970 tablet_bootstrap.cc:492] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f: No bootstrap required, opened a new log
I20260924 14:27:32.479694 23971 raft_consensus.cc:729] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.480126 23970 ts_tablet_manager.cc:1397] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.001s
I20260924 14:27:32.480403 23971 consensus_queue.cc:260] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.482887 23971 ts_tablet_manager.cc:1428] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37: Time spent starting tablet: real 0.039s	user 0.000s	sys 0.004s
I20260924 14:27:32.483349 23930 raft_consensus.cc:2454] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 1.
I20260924 14:27:32.483673 23970 raft_consensus.cc:348] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.484359 23820 leader_election.cc:302] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:32.484606 23970 raft_consensus.cc:374] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.485518 23971 tablet_bootstrap.cc:492] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37: Bootstrap starting.
I20260924 14:27:32.484959 23781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb58b7233f5a44daaf2209d0394cd248" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:32.485324 24013 raft_consensus.cc:2781] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:32.487501 24012 leader_election.cc:288] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.488245 23929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb58b7233f5a44daaf2209d0394cd248" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:32.489395 23781 raft_consensus.cc:3037] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:27:32.489459 23820 leader_election.cc:341] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.490778 23931 raft_consensus.cc:2454] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 1.
I20260924 14:27:32.485749 23970 raft_consensus.cc:729] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.493388 23977 raft_consensus.cc:482] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.493609 23971 tablet_bootstrap.cc:654] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.493883 23977 raft_consensus.cc:504] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.494505 23970 consensus_queue.cc:260] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.498881 23977 leader_election.cc:288] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:32.500779 23970 ts_tablet_manager.cc:1428] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260924 14:27:32.500710 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.503787 23857 raft_consensus.cc:2454] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 0.
I20260924 14:27:32.505551 23896 leader_election.cc:302] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:32.504249 23970 tablet_bootstrap.cc:492] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f: Bootstrap starting.
I20260924 14:27:32.506634 23977 raft_consensus.cc:2781] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:32.507215 23977 raft_consensus.cc:482] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:32.507714 23977 raft_consensus.cc:3037] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.506798 23679 catalog_manager.cc:5495] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "5b18601c6e1f45358f9a1ad392cff957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:32.511171 23781 raft_consensus.cc:2454] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 1.
I20260924 14:27:32.493072 23973 tablet_bootstrap.cc:492] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957: No bootstrap required, opened a new log
I20260924 14:27:32.517087 23820 leader_election.cc:302] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:32.517822 23970 tablet_bootstrap.cc:654] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.503875 23745 leader_election.cc:302] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:32.500924 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.519326 23973 ts_tablet_manager.cc:1397] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957: Time spent bootstrapping tablet: real 0.153s	user 0.011s	sys 0.006s
W20260924 14:27:32.520927 23894 leader_election.cc:341] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.501708 23976 leader_election.cc:288] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.502980 23856 raft_consensus.cc:2454] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 1.
I20260924 14:27:32.515551 24013 raft_consensus.cc:686] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Becoming Leader. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:32.522591 23973 raft_consensus.cc:348] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.523640 23973 raft_consensus.cc:374] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.524065 23973 raft_consensus.cc:729] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.523509 24013 consensus_queue.cc:237] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.526197 23973 consensus_queue.cc:260] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.522634 23976 raft_consensus.cc:2781] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:32.530488 23999 raft_consensus.cc:2781] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:32.538412 23976 raft_consensus.cc:686] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Becoming Leader. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:32.538478 23999 raft_consensus.cc:686] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 LEADER]: Becoming Leader. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:32.539601 23999 consensus_queue.cc:237] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.539690 23976 consensus_queue.cc:237] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.541699 23678 catalog_manager.cc:5495] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7744d9495c594ba59d47a45af215ff8f (127.22.151.194). New cstate: current_term: 1 leader_uuid: "7744d9495c594ba59d47a45af215ff8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:32.532495 23973 ts_tablet_manager.cc:1428] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260924 14:27:32.548483 23973 tablet_bootstrap.cc:492] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957: Bootstrap starting.
I20260924 14:27:32.535703 23971 tablet_bootstrap.cc:492] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37: No bootstrap required, opened a new log
I20260924 14:27:32.529215 23977 raft_consensus.cc:504] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.558451 23970 tablet_bootstrap.cc:492] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f: No bootstrap required, opened a new log
I20260924 14:27:32.558912 23970 ts_tablet_manager.cc:1397] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f: Time spent bootstrapping tablet: real 0.055s	user 0.021s	sys 0.023s
I20260924 14:27:32.558758 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:32.555928 23971 ts_tablet_manager.cc:1397] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37: Time spent bootstrapping tablet: real 0.071s	user 0.024s	sys 0.024s
I20260924 14:27:32.562768 23970 raft_consensus.cc:348] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.563728 23970 raft_consensus.cc:374] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.564061 23970 raft_consensus.cc:729] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.564615 23971 raft_consensus.cc:348] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.565418 23971 raft_consensus.cc:374] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.565802 23971 raft_consensus.cc:729] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.565574 23679 catalog_manager.cc:5495] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:32.566473 23971 consensus_queue.cc:260] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.567634 23970 consensus_queue.cc:260] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.568819 23971 ts_tablet_manager.cc:1428] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260924 14:27:32.569826 23971 tablet_bootstrap.cc:492] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37: Bootstrap starting.
I20260924 14:27:32.570719 23970 ts_tablet_manager.cc:1428] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260924 14:27:32.572098 23970 tablet_bootstrap.cc:492] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f: Bootstrap starting.
I20260924 14:27:32.575796 23971 tablet_bootstrap.cc:654] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37: Neither blocks nor log segments found. Creating new log.
W20260924 14:27:32.576148 23894 leader_election.cc:341] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:27:32.574970 23678 catalog_manager.cc:5495] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "5b18601c6e1f45358f9a1ad392cff957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:32.579257 23970 tablet_bootstrap.cc:654] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.579607 23977 leader_election.cc:288] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:32.580964 23856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:32.581605 23856 raft_consensus.cc:3037] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.583081 23973 tablet_bootstrap.cc:654] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.591251 23970 tablet_bootstrap.cc:492] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f: No bootstrap required, opened a new log
I20260924 14:27:32.591796 23970 ts_tablet_manager.cc:1397] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.003s
I20260924 14:27:32.592224 23856 raft_consensus.cc:2454] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 1.
I20260924 14:27:32.593413 23896 leader_election.cc:302] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:32.594614 23977 raft_consensus.cc:2781] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:32.594411 23970 raft_consensus.cc:348] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.595110 23970 raft_consensus.cc:374] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.595238 23977 raft_consensus.cc:686] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Becoming Leader. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:32.595377 23970 raft_consensus.cc:729] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.596194 23970 consensus_queue.cc:260] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.596288 23977 consensus_queue.cc:237] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.598309 23970 ts_tablet_manager.cc:1428] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:27:32.601894 23971 tablet_bootstrap.cc:492] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37: No bootstrap required, opened a new log
I20260924 14:27:32.605340 23971 ts_tablet_manager.cc:1397] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.000s
I20260924 14:27:32.604521 23677 catalog_manager.cc:5495] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7744d9495c594ba59d47a45af215ff8f (127.22.151.194). New cstate: current_term: 1 leader_uuid: "7744d9495c594ba59d47a45af215ff8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:32.607971 23971 raft_consensus.cc:348] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.608860 23971 raft_consensus.cc:374] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.609174 23971 raft_consensus.cc:729] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.609805 23971 consensus_queue.cc:260] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.611860 23971 ts_tablet_manager.cc:1428] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260924 14:27:32.612696 23971 tablet_bootstrap.cc:492] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37: Bootstrap starting.
I20260924 14:27:32.616532 23973 tablet_bootstrap.cc:492] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957: No bootstrap required, opened a new log
I20260924 14:27:32.616983 23973 ts_tablet_manager.cc:1397] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957: Time spent bootstrapping tablet: real 0.069s	user 0.009s	sys 0.000s
I20260924 14:27:32.617439 23971 tablet_bootstrap.cc:654] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.619400 23973 raft_consensus.cc:348] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.620085 23973 raft_consensus.cc:374] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.620364 23973 raft_consensus.cc:729] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.620954 23973 consensus_queue.cc:260] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.620117 23678 catalog_manager.cc:5495] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:32.622466 23973 ts_tablet_manager.cc:1428] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260924 14:27:32.623728 23973 tablet_bootstrap.cc:492] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957: Bootstrap starting.
I20260924 14:27:32.628911 23973 tablet_bootstrap.cc:654] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957: Neither blocks nor log segments found. Creating new log.
I20260924 14:27:32.629527 23971 tablet_bootstrap.cc:492] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37: No bootstrap required, opened a new log
I20260924 14:27:32.629938 23971 ts_tablet_manager.cc:1397] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37: Time spent bootstrapping tablet: real 0.017s	user 0.003s	sys 0.011s
I20260924 14:27:32.632483 23971 raft_consensus.cc:348] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.633172 23971 raft_consensus.cc:374] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.633481 23971 raft_consensus.cc:729] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.634197 23971 consensus_queue.cc:260] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.634943 23973 tablet_bootstrap.cc:492] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957: No bootstrap required, opened a new log
I20260924 14:27:32.635416 23973 ts_tablet_manager.cc:1397] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260924 14:27:32.636001 23976 raft_consensus.cc:482] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.636140 23971 ts_tablet_manager.cc:1428] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260924 14:27:32.636965 23976 raft_consensus.cc:504] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.638955 24018 raft_consensus.cc:482] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.639400 24018 raft_consensus.cc:504] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.641050 23973 raft_consensus.cc:348] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.641796 23973 raft_consensus.cc:374] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:27:32.642132 23973 raft_consensus.cc:729] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Initialized, Role: FOLLOWER
I20260924 14:27:32.641415 24018 leader_election.cc:288] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.642906 23973 consensus_queue.cc:260] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.643523 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.643767 23976 leader_election.cc:288] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.644223 23931 raft_consensus.cc:2454] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 0.
I20260924 14:27:32.645557 23973 ts_tablet_manager.cc:1428] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260924 14:27:32.645421 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.645442 23820 leader_election.cc:302] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:32.645921 23781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca1ed23241b46cb8c65b57f65f85588" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.646235 23780 raft_consensus.cc:2454] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 0.
I20260924 14:27:32.646522 23976 raft_consensus.cc:2781] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:32.646539 23781 raft_consensus.cc:2454] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 0.
I20260924 14:27:32.646858 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca1ed23241b46cb8c65b57f65f85588" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.647018 23976 raft_consensus.cc:482] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:32.647384 23976 raft_consensus.cc:3037] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.647495 23931 raft_consensus.cc:2454] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 0.
I20260924 14:27:32.647934 23820 leader_election.cc:302] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: 
I20260924 14:27:32.648733 24018 raft_consensus.cc:2781] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:27:32.649106 24018 raft_consensus.cc:482] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:32.649475 24018 raft_consensus.cc:3037] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.656126 24018 raft_consensus.cc:504] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.657126 23976 raft_consensus.cc:504] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.658100 24018 leader_election.cc:288] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.658895 23781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca1ed23241b46cb8c65b57f65f85588" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:32.659000 23976 leader_election.cc:288] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.659046 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca1ed23241b46cb8c65b57f65f85588" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:32.659500 23781 raft_consensus.cc:3037] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.659711 23931 raft_consensus.cc:3037] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.660526 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:32.661065 23930 raft_consensus.cc:3037] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.663962 23781 raft_consensus.cc:2454] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 1.
I20260924 14:27:32.663926 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:32.664602 23780 raft_consensus.cc:3037] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.665998 23931 raft_consensus.cc:2454] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 1.
I20260924 14:27:32.667135 23930 raft_consensus.cc:2454] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 1.
I20260924 14:27:32.667057 23820 leader_election.cc:302] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: 
I20260924 14:27:32.667975 23976 raft_consensus.cc:2781] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:32.668417 23976 raft_consensus.cc:686] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Becoming Leader. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:32.668632 23820 leader_election.cc:302] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:32.669348 24013 raft_consensus.cc:2781] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:27:32.669500 23999 raft_consensus.cc:482] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.669227 23976 consensus_queue.cc:237] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.669874 24013 raft_consensus.cc:686] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Becoming Leader. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:32.669967 23999 raft_consensus.cc:504] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.670612 24013 consensus_queue.cc:237] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.671932 23999 leader_election.cc:288] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.674431 23856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.674913 23780 raft_consensus.cc:2454] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 1.
I20260924 14:27:32.675673 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.676276 23930 raft_consensus.cc:2379] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b18601c6e1f45358f9a1ad392cff957 in current term 1: Already voted for candidate 7744d9495c594ba59d47a45af215ff8f in this term.
I20260924 14:27:32.677186 23745 leader_election.cc:302] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957; no voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:32.677995 23999 raft_consensus.cc:3037] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.678678 23679 catalog_manager.cc:5495] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7744d9495c594ba59d47a45af215ff8f (127.22.151.194). New cstate: current_term: 1 leader_uuid: "7744d9495c594ba59d47a45af215ff8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:32.683792 23999 raft_consensus.cc:2726] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:32.691648 23679 catalog_manager.cc:5495] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7744d9495c594ba59d47a45af215ff8f (127.22.151.194). New cstate: current_term: 1 leader_uuid: "7744d9495c594ba59d47a45af215ff8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:32.718876 23930 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.723974 23977 raft_consensus.cc:482] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.724503 23977 raft_consensus.cc:504] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.724910 23999 consensus_queue.cc:1035] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:32.726840 23977 leader_election.cc:288] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:32.727806 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.727916 23856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.728569 23856 raft_consensus.cc:2454] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 1.
I20260924 14:27:32.729635 23896 leader_election.cc:302] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:32.730527 23977 raft_consensus.cc:2781] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:27:32.730877 23977 raft_consensus.cc:482] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5b18601c6e1f45358f9a1ad392cff957)
I20260924 14:27:32.731257 23977 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:32.738762 23999 consensus_queue.cc:1035] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:32.739339 23977 raft_consensus.cc:504] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.740476 23930 raft_consensus.cc:1226] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Rejecting Update request from peer 5b18601c6e1f45358f9a1ad392cff957 for earlier term 1. Current term is 2. Ops: []
I20260924 14:27:32.741771 23977 leader_election.cc:288] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 2 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:32.743299 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:32.745530 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:32.745857 23989 consensus_queue.cc:1046] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:32.747046 23999 raft_consensus.cc:3032] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:32.747030 23894 leader_election.cc:302] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc8c63a642ab43efa3b4829ea4294d37; no voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:32.747625 23999 raft_consensus.cc:729] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:32.748627 23977 raft_consensus.cc:2726] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:27:32.748509 23999 consensus_queue.cc:260] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.749697 23999 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:32.778355 23999 raft_consensus.cc:482] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.778789 23999 raft_consensus.cc:504] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.780313 23999 leader_election.cc:288] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.781201 23856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.781344 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.781883 23856 raft_consensus.cc:2379] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b18601c6e1f45358f9a1ad392cff957 in current term 1: Already voted for candidate bc8c63a642ab43efa3b4829ea4294d37 in this term.
I20260924 14:27:32.782743 23747 leader_election.cc:302] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957; no voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:32.783397 23999 raft_consensus.cc:3037] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.787878 23999 raft_consensus.cc:2726] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:32.821296 23977 raft_consensus.cc:482] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.821794 23977 raft_consensus.cc:504] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.823758 23977 leader_election.cc:288] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:32.824314 24013 raft_consensus.cc:482] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:32.824522 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6528eea31cb945f89f6bbb6507d2b57e" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.824795 24013 raft_consensus.cc:504] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.825627 23856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6528eea31cb945f89f6bbb6507d2b57e" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:32.826519 23856 raft_consensus.cc:2379] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc8c63a642ab43efa3b4829ea4294d37 in current term 1: Already voted for candidate 5b18601c6e1f45358f9a1ad392cff957 in this term.
I20260924 14:27:32.826742 24013 leader_election.cc:288] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.827759 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811a523968ff4d059b0f66e8ee15b059" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:32.827795 23896 leader_election.cc:302] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc8c63a642ab43efa3b4829ea4294d37; no voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:32.828308 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811a523968ff4d059b0f66e8ee15b059" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:32.828663 23930 raft_consensus.cc:2379] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7744d9495c594ba59d47a45af215ff8f in current term 1: Already voted for candidate 5b18601c6e1f45358f9a1ad392cff957 in this term.
I20260924 14:27:32.829211 23977 raft_consensus.cc:3037] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.830082 23820 leader_election.cc:302] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f; no voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:32.831209 24013 raft_consensus.cc:3037] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.836717 23977 raft_consensus.cc:2726] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:32.838318 24013 raft_consensus.cc:2726] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:27:32.848624 23930 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57"
dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
mode: GRACEFUL
new_leader_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:58530
I20260924 14:27:32.849323 23930 raft_consensus.cc:595] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Received request to transfer leadership to TS 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:32.860733 23977 consensus_queue.cc:1035] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:32.869566 23857 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
mode: GRACEFUL
new_leader_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:41104
I20260924 14:27:32.870180 23857 raft_consensus.cc:595] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Received request to transfer leadership to TS 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:32.882141 24006 consensus_queue.cc:1035] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:32.882493 23856 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
mode: GRACEFUL
new_leader_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:41104
I20260924 14:27:32.883111 23856 raft_consensus.cc:595] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Received request to transfer leadership to TS 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:32.891568 24008 raft_consensus.cc:979] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37: : Instructing follower 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:32.891757 23930 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19"
dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
mode: GRACEFUL
new_leader_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:58530
I20260924 14:27:32.892089 24006 raft_consensus.cc:1067] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Signalling peer 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:32.892364 23930 raft_consensus.cc:595] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Received request to transfer leadership to TS 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:32.893972 23780 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:55028
I20260924 14:27:32.894583 23780 raft_consensus.cc:482] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:32.894961 23780 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:32.897795 23930 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204"
dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
mode: GRACEFUL
new_leader_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:58530
I20260924 14:27:32.898319 23930 raft_consensus.cc:595] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Received request to transfer leadership to TS 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:32.901618 23780 raft_consensus.cc:504] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:32.904255 23780 leader_election.cc:288] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 2 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.904908 23856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:32.905778 23856 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:32.906450 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:32.907022 23930 raft_consensus.cc:3032] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:32.907333 23930 raft_consensus.cc:729] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:32.908099 23930 consensus_queue.cc:260] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:32.909799 23930 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:32.912031 24006 consensus_queue.cc:1035] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:32.914462 23856 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 2.
I20260924 14:27:32.915863 23747 leader_election.cc:302] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: 
I20260924 14:27:32.917161 23999 raft_consensus.cc:2781] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:32.917652 23999 raft_consensus.cc:686] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Becoming Leader. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:32.918566 23999 consensus_queue.cc:237] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:32.925457 23989 consensus_queue.cc:1035] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:32.926220 24008 raft_consensus.cc:979] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37: : Instructing follower 7744d9495c594ba59d47a45af215ff8f to start an election
I20260924 14:27:32.926648 24008 raft_consensus.cc:1067] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Signalling peer 7744d9495c594ba59d47a45af215ff8f to start an election
I20260924 14:27:32.929394 23855 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:52536
I20260924 14:27:32.929977 23855 raft_consensus.cc:482] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:32.930330 23855 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:32.933869 24008 consensus_queue.cc:1035] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:32.934883 23679 catalog_manager.cc:5495] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 reported cstate change: term changed from 1 to 2, leader changed from bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195) to 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "5b18601c6e1f45358f9a1ad392cff957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:32.946048 23930 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 2.
I20260924 14:27:32.945976 23999 consensus_queue.cc:1035] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:27:32.948417 23931 raft_consensus.cc:3037] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:27:32.952581 24013 consensus_queue.cc:1035] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:32.947074 23855 raft_consensus.cc:504] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.958189 23857 raft_consensus.cc:1226] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Rejecting Update request from peer bc8c63a642ab43efa3b4829ea4294d37 for earlier term 1. Current term is 2. Ops: []
I20260924 14:27:32.960608 24013 consensus_queue.cc:1035] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:27:32.965277 23977 consensus_queue.cc:1046] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:27:32.968834 23977 raft_consensus.cc:3032] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:32.969307 23977 raft_consensus.cc:729] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:32.970033 23979 consensus_queue.cc:1035] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:27:32.970331 23977 consensus_queue.cc:260] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:32.972119 23977 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:32.968923 23781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:32.977537 23855 leader_election.cc:288] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 2 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:32.987272 23927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:32.999635 23979 consensus_queue.cc:1035] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:27:33.000087 24008 consensus_queue.cc:1035] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:27:33.002499 24013 consensus_queue.cc:1035] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:33.009086 23779 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
mode: GRACEFUL
new_leader_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:53458
I20260924 14:27:33.009967 23779 raft_consensus.cc:595] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Received request to transfer leadership to TS 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:33.010406 23779 raft_consensus.cc:603] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:27:33.016944 23857 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1ca1ed23241b46cb8c65b57f65f85588"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
mode: GRACEFUL
new_leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
 from {username='slave'} at 127.0.0.1:41104
I20260924 14:27:33.017546 23857 raft_consensus.cc:595] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Received request to transfer leadership to TS bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:33.030448 23927 raft_consensus.cc:2454] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 2.
I20260924 14:27:33.041587 23820 leader_election.cc:302] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:33.042361 23976 raft_consensus.cc:2781] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:33.046077 23976 raft_consensus.cc:686] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Becoming Leader. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:33.048617 23976 consensus_queue.cc:237] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.058434 24013 consensus_queue.cc:1035] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:33.076427 24008 consensus_queue.cc:1035] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:27:33.089908 24054 consensus_queue.cc:1035] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260924 14:27:33.105299 23976 consensus_queue.cc:1035] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:33.114064 23679 catalog_manager.cc:5495] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f reported cstate change: term changed from 1 to 2, leader changed from bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195) to 7744d9495c594ba59d47a45af215ff8f (127.22.151.194). New cstate: current_term: 2 leader_uuid: "7744d9495c594ba59d47a45af215ff8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:33.121943 24006 raft_consensus.cc:979] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37: : Instructing follower 7744d9495c594ba59d47a45af215ff8f to start an election
I20260924 14:27:33.122488 24056 raft_consensus.cc:1067] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Signalling peer 7744d9495c594ba59d47a45af215ff8f to start an election
I20260924 14:27:33.124302 23855 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:52536
I20260924 14:27:33.124904 23855 raft_consensus.cc:482] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:33.125332 23855 raft_consensus.cc:3037] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.138125 23976 raft_consensus.cc:979] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f: : Instructing follower 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:33.138625 23976 raft_consensus.cc:1067] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Signalling peer 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:33.138760 23855 raft_consensus.cc:504] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.140544 24054 raft_consensus.cc:979] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f: : Instructing follower bc8c63a642ab43efa3b4829ea4294d37 to start an election
I20260924 14:27:33.141455 23780 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:55012
I20260924 14:27:33.143399 23855 leader_election.cc:288] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 2 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:33.143682 23780 raft_consensus.cc:482] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:33.144121 23780 raft_consensus.cc:3037] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.143422 24054 raft_consensus.cc:1067] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Signalling peer bc8c63a642ab43efa3b4829ea4294d37 to start an election
I20260924 14:27:33.147058 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:33.147764 23930 raft_consensus.cc:3032] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:33.148102 23930 raft_consensus.cc:729] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:33.148810 23782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:33.148945 23930 consensus_queue.cc:260] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.150454 23930 raft_consensus.cc:3037] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.152144 23931 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1ca1ed23241b46cb8c65b57f65f85588"
dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
 from {username='slave'} at 127.0.0.1:42950
I20260924 14:27:33.152765 23931 raft_consensus.cc:482] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:33.153158 23931 raft_consensus.cc:3037] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.149471 23782 raft_consensus.cc:3037] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.156087 23780 raft_consensus.cc:504] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.161108 23931 raft_consensus.cc:504] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.162084 23855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:33.162699 23855 raft_consensus.cc:3032] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:33.163028 23855 raft_consensus.cc:729] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:33.163128 23782 raft_consensus.cc:2454] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 2.
I20260924 14:27:33.164057 23781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca1ed23241b46cb8c65b57f65f85588" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:33.163957 23855 consensus_queue.cc:260] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.164451 23820 leader_election.cc:302] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: 
I20260924 14:27:33.164634 23781 raft_consensus.cc:3037] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.165506 24055 raft_consensus.cc:2781] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:33.165982 24055 raft_consensus.cc:686] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Becoming Leader. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:33.166832 24055 consensus_queue.cc:237] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.168732 23780 leader_election.cc:288] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 2 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:33.165135 23855 raft_consensus.cc:3037] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.169571 23929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:33.170315 23929 raft_consensus.cc:3037] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.172309 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca1ed23241b46cb8c65b57f65f85588" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:33.172953 23857 raft_consensus.cc:3032] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:33.173282 23857 raft_consensus.cc:729] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:33.174049 23857 consensus_queue.cc:260] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.175163 23857 raft_consensus.cc:3037] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.176548 23855 raft_consensus.cc:2454] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 2.
I20260924 14:27:33.164608 23930 raft_consensus.cc:2454] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 2.
I20260924 14:27:33.177448 23929 raft_consensus.cc:2454] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 2.
I20260924 14:27:33.178790 23745 leader_election.cc:302] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:33.179775 24052 raft_consensus.cc:2781] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:33.180251 24052 raft_consensus.cc:686] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Becoming Leader. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:33.181191 24052 consensus_queue.cc:237] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.182088 23781 raft_consensus.cc:2454] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 2.
I20260924 14:27:33.183727 23857 raft_consensus.cc:2454] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 2.
I20260924 14:27:33.183849 23894 leader_election.cc:302] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:33.184744 24056 raft_consensus.cc:2781] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:33.177492 23931 leader_election.cc:288] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 2 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:33.185225 24056 raft_consensus.cc:686] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 LEADER]: Becoming Leader. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:33.186422 24056 consensus_queue.cc:237] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.191980 23677 catalog_manager.cc:5495] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f reported cstate change: term changed from 1 to 2, leader changed from bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195) to 7744d9495c594ba59d47a45af215ff8f (127.22.151.194). New cstate: current_term: 2 leader_uuid: "7744d9495c594ba59d47a45af215ff8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:33.201941 23679 catalog_manager.cc:5495] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 reported cstate change: term changed from 1 to 2, leader changed from 7744d9495c594ba59d47a45af215ff8f (127.22.151.194) to 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "5b18601c6e1f45358f9a1ad392cff957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:33.209468 23678 catalog_manager.cc:5495] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 reported cstate change: term changed from 1 to 2, leader changed from 7744d9495c594ba59d47a45af215ff8f (127.22.151.194) to bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:33.278898 24055 consensus_queue.cc:1035] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:33.286374 24013 raft_consensus.cc:979] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f: : Instructing follower 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:33.287088 23976 raft_consensus.cc:1067] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Signalling peer 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:33.288679 23780 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:55012
I20260924 14:27:33.291317 23780 raft_consensus.cc:482] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:33.291767 23780 raft_consensus.cc:3037] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.293566 23976 consensus_queue.cc:1035] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:27:33.299487 23780 raft_consensus.cc:504] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.303602 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:33.304339 23857 raft_consensus.cc:3032] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:33.304839 23857 raft_consensus.cc:729] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:33.305608 23857 consensus_queue.cc:260] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.306735 23857 raft_consensus.cc:3037] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.307718 23780 leader_election.cc:288] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 2 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:33.309535 23929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:33.310019 23929 raft_consensus.cc:3037] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.313198 23857 raft_consensus.cc:2454] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 2.
I20260924 14:27:33.314246 23747 leader_election.cc:302] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: 
I20260924 14:27:33.314764 23929 raft_consensus.cc:2454] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 2.
I20260924 14:27:33.314975 24052 raft_consensus.cc:2781] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:33.315426 24052 raft_consensus.cc:686] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Becoming Leader. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:33.316293 24052 consensus_queue.cc:237] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.322700 23679 catalog_manager.cc:5495] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 reported cstate change: term changed from 1 to 2, leader changed from 7744d9495c594ba59d47a45af215ff8f (127.22.151.194) to 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "5b18601c6e1f45358f9a1ad392cff957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:33.403780 23929 raft_consensus.cc:1261] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.405380 24052 consensus_queue.cc:1035] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.413631 23857 raft_consensus.cc:1261] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.415299 23979 consensus_queue.cc:1035] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.446542 23929 raft_consensus.cc:1261] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Refusing update from remote peer 7744d9495c594ba59d47a45af215ff8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.447909 23976 consensus_queue.cc:1035] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.455969 23780 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:33.463635 23780 raft_consensus.cc:1261] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Refusing update from remote peer 7744d9495c594ba59d47a45af215ff8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.465082 23976 consensus_queue.cc:1035] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.630108 23780 raft_consensus.cc:1261] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Refusing update from remote peer 7744d9495c594ba59d47a45af215ff8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.631428 23976 consensus_queue.cc:1035] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.638975 23929 raft_consensus.cc:1261] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Refusing update from remote peer 7744d9495c594ba59d47a45af215ff8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.641551 24013 consensus_queue.cc:1035] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.687695 23780 raft_consensus.cc:1261] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Refusing update from remote peer bc8c63a642ab43efa3b4829ea4294d37: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.689036 24056 consensus_queue.cc:1035] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.698175 23857 raft_consensus.cc:1261] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Refusing update from remote peer bc8c63a642ab43efa3b4829ea4294d37: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.699730 24063 consensus_queue.cc:1035] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.772295 23857 raft_consensus.cc:1261] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.773772 23999 consensus_queue.cc:1035] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.781409 23929 raft_consensus.cc:1261] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.784055 23999 consensus_queue.cc:1035] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.825778 23857 raft_consensus.cc:1261] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.827116 23999 consensus_queue.cc:1035] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.834569 23929 raft_consensus.cc:1261] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:33.836053 23989 consensus_queue.cc:1035] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:33.918056 23989 raft_consensus.cc:482] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:33.918540 23989 raft_consensus.cc:504] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:33.920229 23989 leader_election.cc:288] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:33.921245 23929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:33.921221 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:33.922847 23747 leader_election.cc:302] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957; no voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:33.923487 23989 raft_consensus.cc:2726] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:27:34.242337 24013 raft_consensus.cc:482] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5b18601c6e1f45358f9a1ad392cff957)
I20260924 14:27:34.242835 24013 raft_consensus.cc:504] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:34.244550 24013 leader_election.cc:288] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:34.245548 23929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37" is_pre_election: true
I20260924 14:27:34.245497 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:34.246434 23929 raft_consensus.cc:2379] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7744d9495c594ba59d47a45af215ff8f in current term 2: Already voted for candidate bc8c63a642ab43efa3b4829ea4294d37 in this term.
I20260924 14:27:34.246325 23780 raft_consensus.cc:2454] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 2.
I20260924 14:27:34.247714 23820 leader_election.cc:302] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:34.248466 24013 raft_consensus.cc:2781] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:27:34.248874 24013 raft_consensus.cc:482] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5b18601c6e1f45358f9a1ad392cff957)
I20260924 14:27:34.249281 24013 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:34.254735 24013 raft_consensus.cc:504] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:34.256188 24013 leader_election.cc:288] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 2 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:34.257071 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:34.257241 23929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:34.257906 23929 raft_consensus.cc:2379] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7744d9495c594ba59d47a45af215ff8f in current term 2: Already voted for candidate bc8c63a642ab43efa3b4829ea4294d37 in this term.
I20260924 14:27:34.261662 23780 raft_consensus.cc:2454] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 2.
I20260924 14:27:34.262569 23820 leader_election.cc:302] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:34.263203 24013 raft_consensus.cc:2781] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:34.263620 24013 raft_consensus.cc:686] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Becoming Leader. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:34.264249 24013 consensus_queue.cc:237] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:34.271111 23678 catalog_manager.cc:5495] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f reported cstate change: term changed from 1 to 2, leader changed from 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193) to 7744d9495c594ba59d47a45af215ff8f (127.22.151.194). New cstate: current_term: 2 leader_uuid: "7744d9495c594ba59d47a45af215ff8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:34.662177 24067 raft_consensus.cc:482] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:27:34.662640 24067 raft_consensus.cc:504] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:34.664355 24067 leader_election.cc:288] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:34.665280 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957" is_pre_election: true
I20260924 14:27:34.665457 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f" is_pre_election: true
I20260924 14:27:34.665807 23780 raft_consensus.cc:2454] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 2.
I20260924 14:27:34.666857 23894 leader_election.cc:302] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:34.667505 24067 raft_consensus.cc:2781] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:27:34.667827 24067 raft_consensus.cc:482] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:27:34.668041 24067 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:34.673193 24067 raft_consensus.cc:504] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:34.675419 24067 leader_election.cc:288] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 3 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:34.675864 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:34.676057 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:34.676504 23780 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:34.681061 23780 raft_consensus.cc:2454] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 3.
I20260924 14:27:34.682217 23894 leader_election.cc:302] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:34.682919 24067 raft_consensus.cc:2781] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:34.683429 24067 raft_consensus.cc:686] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Becoming Leader. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:34.684237 24067 consensus_queue.cc:237] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:34.691355 23678 catalog_manager.cc:5495] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 reported cstate change: term changed from 2 to 3, leader changed from 7744d9495c594ba59d47a45af215ff8f (127.22.151.194) to bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:34.786532 23780 raft_consensus.cc:1226] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Rejecting Update request from peer 7744d9495c594ba59d47a45af215ff8f for earlier term 2. Current term is 3. Ops: []
I20260924 14:27:34.787910 24013 consensus_queue.cc:1046] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:34.789826 24013 raft_consensus.cc:3032] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:34.790081 24013 raft_consensus.cc:729] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:34.790745 24013 consensus_queue.cc:260] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:34.792002 24013 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:35.063766 23780 raft_consensus.cc:1261] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Refusing update from remote peer bc8c63a642ab43efa3b4829ea4294d37: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:27:35.065387 24067 consensus_queue.cc:1035] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:35.076476 23857 raft_consensus.cc:1261] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Refusing update from remote peer bc8c63a642ab43efa3b4829ea4294d37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260924 14:27:35.076902 23857 pending_rounds.cc:76] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f: Aborting all ops after (but not including) 1
I20260924 14:27:35.077191 23857 pending_rounds.cc:97] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:27:35.077512 23857 raft_consensus.cc:2866] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:27:35.078902 24067 consensus_queue.cc:1035] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:36.048677 23135 auto_leader_rebalancer-test.cc:339] The leader distribution is 
I20260924 14:27:36.049404 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:36.077103 23781 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
mode: GRACEFUL
new_leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
 from {username='slave'} at 127.0.0.1:53462
I20260924 14:27:36.077819 23781 raft_consensus.cc:595] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Received request to transfer leadership to TS bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:36.079032 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260924 14:27:36.079548 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:36.615258 24081 raft_consensus.cc:979] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957: : Instructing follower bc8c63a642ab43efa3b4829ea4294d37 to start an election
I20260924 14:27:36.615702 23979 raft_consensus.cc:1067] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Signalling peer bc8c63a642ab43efa3b4829ea4294d37 to start an election
I20260924 14:27:36.617221 23929 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57"
dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
 from {username='slave'} at 127.0.0.1:42938
I20260924 14:27:36.617744 23929 raft_consensus.cc:482] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:36.618098 23929 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:36.622707 23929 raft_consensus.cc:504] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:36.624300 23929 leader_election.cc:288] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 3 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675)
I20260924 14:27:36.625341 23781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:36.625368 23857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:36.625877 23781 raft_consensus.cc:3032] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:36.626058 23857 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:36.626199 23781 raft_consensus.cc:729] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:36.626942 23781 consensus_queue.cc:260] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:36.627867 23781 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:36.631132 23857 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 3.
I20260924 14:27:36.632294 23781 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 3.
I20260924 14:27:36.632326 23896 leader_election.cc:302] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:36.633291 24067 raft_consensus.cc:2781] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:36.634110 24067 raft_consensus.cc:686] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Becoming Leader. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:36.634881 24067 consensus_queue.cc:237] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:36.644090 23679 catalog_manager.cc:5495] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 reported cstate change: term changed from 2 to 3, leader changed from 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193) to bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:37.045811 23857 raft_consensus.cc:1261] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Refusing update from remote peer bc8c63a642ab43efa3b4829ea4294d37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:37.047091 24067 consensus_queue.cc:1035] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:37.057103 23781 raft_consensus.cc:1261] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Refusing update from remote peer bc8c63a642ab43efa3b4829ea4294d37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:37.059263 24082 consensus_queue.cc:1035] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:37.080356 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:37.085491 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:37.085856 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:38.086704 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:38.092715 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:38.093236 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:39.094168 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:39.100062 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:39.100476 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:40.114651 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:40.127817 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:40.128300 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:41.129060 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:41.137127 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:41.137709 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:42.138552 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:42.143576 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:42.144028 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:43.144950 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:43.150909 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:43.151384 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:44.152352 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:44.157500 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:44.157953 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:45.158829 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:45.163995 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:45.164444 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:46.197067 23135 auto_leader_rebalancer-test.cc:358] The leader distribution is 
I20260924 14:27:46.202031 23931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57"
dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
mode: GRACEFUL
new_leader_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:58530
I20260924 14:27:46.202977 23931 raft_consensus.cc:595] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Received request to transfer leadership to TS 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:46.218312 23855 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
mode: GRACEFUL
new_leader_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:41104
I20260924 14:27:46.218878 23855 raft_consensus.cc:595] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Received request to transfer leadership to TS 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:46.224159 23855 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
mode: GRACEFUL
new_leader_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:41104
I20260924 14:27:46.224706 23855 raft_consensus.cc:595] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Received request to transfer leadership to TS 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:46.229702 23855 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cb58b7233f5a44daaf2209d0394cd248"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
mode: GRACEFUL
new_leader_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:41104
I20260924 14:27:46.230417 23855 raft_consensus.cc:595] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Received request to transfer leadership to TS 5b18601c6e1f45358f9a1ad392cff957
I20260924 14:27:46.235869 23931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b"
dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
mode: GRACEFUL
new_leader_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:58530
I20260924 14:27:46.236466 23931 raft_consensus.cc:595] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Received request to transfer leadership to TS 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:46.257022 24072 raft_consensus.cc:979] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f: : Instructing follower 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:46.257534 24102 raft_consensus.cc:1067] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Signalling peer 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:46.258951 23781 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:55012
I20260924 14:27:46.259419 23781 raft_consensus.cc:482] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:46.259804 23781 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:46.265223 23781 raft_consensus.cc:504] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.266983 23781 leader_election.cc:288] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 3 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:46.268076 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:46.267946 23855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:46.268740 23931 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:46.268873 23855 raft_consensus.cc:3032] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:46.269351 23855 raft_consensus.cc:729] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:46.270268 23855 consensus_queue.cc:260] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.271755 23855 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:46.275784 23931 raft_consensus.cc:2454] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 3.
I20260924 14:27:46.277151 23745 leader_election.cc:302] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:46.277348 23855 raft_consensus.cc:2454] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 3.
I20260924 14:27:46.278144 24099 raft_consensus.cc:2781] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:46.278589 24099 raft_consensus.cc:686] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 LEADER]: Becoming Leader. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:46.279317 24099 consensus_queue.cc:237] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.286124 23678 catalog_manager.cc:5495] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 reported cstate change: term changed from 2 to 3, leader changed from 7744d9495c594ba59d47a45af215ff8f (127.22.151.194) to 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "5b18601c6e1f45358f9a1ad392cff957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:46.407668 24108 raft_consensus.cc:979] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37: : Instructing follower 7744d9495c594ba59d47a45af215ff8f to start an election
I20260924 14:27:46.408219 24100 raft_consensus.cc:1067] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Signalling peer 7744d9495c594ba59d47a45af215ff8f to start an election
I20260924 14:27:46.409855 23855 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:52536
I20260924 14:27:46.410408 23855 raft_consensus.cc:482] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:46.410732 23855 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:46.415176 24072 raft_consensus.cc:979] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f: : Instructing follower 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:46.415719 24102 raft_consensus.cc:1067] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Signalling peer 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:46.416337 23855 raft_consensus.cc:504] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.417164 23781 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:55012
I20260924 14:27:46.417666 23781 raft_consensus.cc:482] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:46.417944 23781 raft_consensus.cc:3037] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:46.418232 23855 leader_election.cc:288] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 4 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:46.419235 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:46.419473 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac851188d724766ac3d80b1a1a6b98b" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:46.419888 23780 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:46.420073 23931 raft_consensus.cc:3032] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:27:46.420591 23931 raft_consensus.cc:729] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:46.421569 23931 consensus_queue.cc:260] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.423043 23931 raft_consensus.cc:3037] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:46.425329 23781 raft_consensus.cc:504] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.427208 23781 leader_election.cc:288] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 3 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:46.427286 23931 raft_consensus.cc:2454] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 4.
I20260924 14:27:46.428460 23855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:46.428655 23820 leader_election.cc:302] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:46.428880 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05549cf4bfc4950856a68b77823b204" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:46.429132 23855 raft_consensus.cc:3032] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:46.429198 23780 raft_consensus.cc:2454] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 4.
I20260924 14:27:46.429404 23931 raft_consensus.cc:3037] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:46.429543 23855 raft_consensus.cc:729] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:46.429868 24102 raft_consensus.cc:2781] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:27:46.430388 24102 raft_consensus.cc:686] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 4 LEADER]: Becoming Leader. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:46.430473 23855 consensus_queue.cc:260] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.431584 23855 raft_consensus.cc:3037] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:46.431245 24102 consensus_queue.cc:237] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.436138 23855 raft_consensus.cc:2454] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 3.
I20260924 14:27:46.436348 23931 raft_consensus.cc:2454] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 3.
I20260924 14:27:46.437668 23747 leader_election.cc:302] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: 
I20260924 14:27:46.438458 24099 raft_consensus.cc:2781] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:46.438848 24099 raft_consensus.cc:686] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 LEADER]: Becoming Leader. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:46.439589 24099 consensus_queue.cc:237] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.440006 23678 catalog_manager.cc:5495] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f reported cstate change: term changed from 3 to 4, leader changed from bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195) to 7744d9495c594ba59d47a45af215ff8f (127.22.151.194). New cstate: current_term: 4 leader_uuid: "7744d9495c594ba59d47a45af215ff8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:46.447096 23679 catalog_manager.cc:5495] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 reported cstate change: term changed from 2 to 3, leader changed from 7744d9495c594ba59d47a45af215ff8f (127.22.151.194) to 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "5b18601c6e1f45358f9a1ad392cff957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:46.621574 24108 raft_consensus.cc:979] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37: : Instructing follower 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:46.622028 24108 raft_consensus.cc:1067] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Signalling peer 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:46.623649 23780 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:55028
I20260924 14:27:46.624133 23780 raft_consensus.cc:482] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:46.624480 23780 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:46.628989 23780 raft_consensus.cc:504] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:46.630479 23780 leader_election.cc:288] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 4 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:46.631489 23855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:46.631488 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:46.632270 23855 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:46.632270 23931 raft_consensus.cc:3032] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:27:46.632764 23931 raft_consensus.cc:729] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:46.633492 23931 consensus_queue.cc:260] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:46.634655 23931 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:46.638574 23855 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 4.
I20260924 14:27:46.639719 23747 leader_election.cc:302] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, 7744d9495c594ba59d47a45af215ff8f; no voters: 
I20260924 14:27:46.640373 23931 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 4.
I20260924 14:27:46.640475 24099 raft_consensus.cc:2781] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:27:46.641659 24099 raft_consensus.cc:686] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 LEADER]: Becoming Leader. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:46.642501 24099 consensus_queue.cc:237] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:46.644368 24072 raft_consensus.cc:979] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f: : Instructing follower 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:46.644832 24102 raft_consensus.cc:1067] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Signalling peer 5b18601c6e1f45358f9a1ad392cff957 to start an election
I20260924 14:27:46.646342 23780 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb58b7233f5a44daaf2209d0394cd248"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
 from {username='slave'} at 127.0.0.1:55012
I20260924 14:27:46.646859 23780 raft_consensus.cc:482] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:46.647171 23780 raft_consensus.cc:3037] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:46.650563 23679 catalog_manager.cc:5495] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 reported cstate change: term changed from 3 to 4, leader changed from bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195) to 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193). New cstate: current_term: 4 leader_uuid: "5b18601c6e1f45358f9a1ad392cff957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:46.653550 23780 raft_consensus.cc:504] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.655035 23780 leader_election.cc:288] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 2 election: Requested vote from peers 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:46.655879 23855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb58b7233f5a44daaf2209d0394cd248" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:46.656409 23855 raft_consensus.cc:3032] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:27:46.656697 23855 raft_consensus.cc:729] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:46.657011 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb58b7233f5a44daaf2209d0394cd248" candidate_uuid: "5b18601c6e1f45358f9a1ad392cff957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:46.657536 23931 raft_consensus.cc:3037] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:46.657413 23855 consensus_queue.cc:260] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.658516 23855 raft_consensus.cc:3037] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:27:46.663144 23855 raft_consensus.cc:2454] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 2.
I20260924 14:27:46.663282 23931 raft_consensus.cc:2454] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b18601c6e1f45358f9a1ad392cff957 in term 2.
I20260924 14:27:46.664273 23745 leader_election.cc:302] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:46.665050 24099 raft_consensus.cc:2781] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:27:46.665494 24099 raft_consensus.cc:686] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Becoming Leader. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:46.666209 24099 consensus_queue.cc:237] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:46.673132 23679 catalog_manager.cc:5495] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 reported cstate change: term changed from 1 to 2, leader changed from 7744d9495c594ba59d47a45af215ff8f (127.22.151.194) to 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "5b18601c6e1f45358f9a1ad392cff957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:46.701428 23931 raft_consensus.cc:1261] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:46.702631 24099 consensus_queue.cc:1035] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:46.711762 23855 raft_consensus.cc:1261] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:46.713431 24101 consensus_queue.cc:1035] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:27:46.859500 23780 raft_consensus.cc:1261] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Refusing update from remote peer 7744d9495c594ba59d47a45af215ff8f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:27:46.860737 24102 consensus_queue.cc:1035] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:46.869562 23931 raft_consensus.cc:1261] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Refusing update from remote peer 7744d9495c594ba59d47a45af215ff8f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:27:46.871121 24102 consensus_queue.cc:1035] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:46.909441 23931 raft_consensus.cc:1261] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:46.910861 24116 consensus_queue.cc:1035] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:27:46.920578 23855 raft_consensus.cc:1261] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:46.922290 24099 consensus_queue.cc:1035] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:47.088967 23855 raft_consensus.cc:1261] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 4 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:27:47.090528 24114 consensus_queue.cc:1035] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:27:47.102030 23931 raft_consensus.cc:1261] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:27:47.103472 24116 consensus_queue.cc:1035] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:27:47.110579 23929 raft_consensus.cc:1261] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:47.112066 24116 consensus_queue.cc:1035] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:27:47.129956 23855 raft_consensus.cc:1261] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Refusing update from remote peer 5b18601c6e1f45358f9a1ad392cff957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:27:47.131870 24116 consensus_queue.cc:1035] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:27:49.268453 23135 auto_leader_rebalancer-test.cc:373] The leader distribution is 
I20260924 14:27:49.269066 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:49.275579 23780 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
mode: GRACEFUL
new_leader_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:53462
I20260924 14:27:49.276135 23780 raft_consensus.cc:595] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 LEADER]: Received request to transfer leadership to TS 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:49.278708 23780 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
mode: GRACEFUL
new_leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
 from {username='slave'} at 127.0.0.1:53462
I20260924 14:27:49.279294 23780 raft_consensus.cc:595] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Received request to transfer leadership to TS bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:49.281574 23780 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
mode: GRACEFUL
new_leader_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:53462
I20260924 14:27:49.282120 23780 raft_consensus.cc:595] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 LEADER]: Received request to transfer leadership to TS 7744d9495c594ba59d47a45af215ff8f
I20260924 14:27:49.284159 23780 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700"
dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
mode: GRACEFUL
new_leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
 from {username='slave'} at 127.0.0.1:53462
I20260924 14:27:49.284536 23780 raft_consensus.cc:595] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Received request to transfer leadership to TS bc8c63a642ab43efa3b4829ea4294d37
I20260924 14:27:49.285363 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260924 14:27:49.285753 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:49.297451 24116 raft_consensus.cc:979] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957: : Instructing follower 7744d9495c594ba59d47a45af215ff8f to start an election
I20260924 14:27:49.297942 24131 raft_consensus.cc:1067] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 LEADER]: Signalling peer 7744d9495c594ba59d47a45af215ff8f to start an election
I20260924 14:27:49.299204 23855 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:52530
I20260924 14:27:49.299647 23855 raft_consensus.cc:482] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:49.299925 23855 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:49.304373 23855 raft_consensus.cc:504] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:49.306042 23855 leader_election.cc:288] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 4 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467)
I20260924 14:27:49.306739 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:49.306989 23929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141131b2a0824d3cb3c82671609b0e19" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:49.307374 23780 raft_consensus.cc:3032] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:27:49.307495 23929 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:49.307794 23780 raft_consensus.cc:729] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:49.308519 23780 consensus_queue.cc:260] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:49.309617 23780 raft_consensus.cc:3037] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:27:49.312912 23929 raft_consensus.cc:2454] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 4.
I20260924 14:27:49.313787 23820 leader_election.cc:302] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:49.314396 24132 raft_consensus.cc:2781] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:27:49.314814 23780 raft_consensus.cc:2454] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 4.
I20260924 14:27:49.315322 24132 raft_consensus.cc:686] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 4 LEADER]: Becoming Leader. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:49.316215 24132 consensus_queue.cc:237] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:49.322387 23678 catalog_manager.cc:5495] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f reported cstate change: term changed from 3 to 4, leader changed from 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193) to 7744d9495c594ba59d47a45af215ff8f (127.22.151.194). New cstate: current_term: 4 leader_uuid: "7744d9495c594ba59d47a45af215ff8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:49.626849 24116 raft_consensus.cc:979] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957: : Instructing follower 7744d9495c594ba59d47a45af215ff8f to start an election
I20260924 14:27:49.627312 24131 raft_consensus.cc:1067] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 LEADER]: Signalling peer 7744d9495c594ba59d47a45af215ff8f to start an election
I20260924 14:27:49.628672 23855 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57"
dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
 from {username='slave'} at 127.0.0.1:52530
I20260924 14:27:49.629119 23855 raft_consensus.cc:482] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:49.629505 23855 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:27:49.634768 23855 raft_consensus.cc:504] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:49.635751 24131 raft_consensus.cc:979] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957: : Instructing follower bc8c63a642ab43efa3b4829ea4294d37 to start an election
I20260924 14:27:49.636247 24131 raft_consensus.cc:1067] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Signalling peer bc8c63a642ab43efa3b4829ea4294d37 to start an election
I20260924 14:27:49.637274 23931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
I20260924 14:27:49.637760 23929 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669"
dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
 from {username='slave'} at 127.0.0.1:42938
I20260924 14:27:49.637813 23931 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:27:49.638465 23929 raft_consensus.cc:482] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:49.638777 23929 raft_consensus.cc:3037] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:49.638893 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4565b47ba44cc09bc0850742b28c57" candidate_uuid: "7744d9495c594ba59d47a45af215ff8f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:49.639469 23780 raft_consensus.cc:3032] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:27:49.639742 23780 raft_consensus.cc:729] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:49.640358 23780 consensus_queue.cc:260] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:49.641332 23780 raft_consensus.cc:3037] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:27:49.643304 23931 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 5.
I20260924 14:27:49.637945 23855 leader_election.cc:288] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 5 election: Requested vote from peers bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195:41467), 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675)
I20260924 14:27:49.644434 23820 leader_election.cc:302] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:49.645267 24132 raft_consensus.cc:2781] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:27:49.644990 23929 raft_consensus.cc:504] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:49.645727 24132 raft_consensus.cc:686] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 5 LEADER]: Becoming Leader. State: Replica: 7744d9495c594ba59d47a45af215ff8f, State: Running, Role: LEADER
I20260924 14:27:49.646600 24132 consensus_queue.cc:237] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } }
I20260924 14:27:49.647239 23780 raft_consensus.cc:2454] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7744d9495c594ba59d47a45af215ff8f in term 5.
I20260924 14:27:49.650599 24139 raft_consensus.cc:979] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957: : Instructing follower bc8c63a642ab43efa3b4829ea4294d37 to start an election
I20260924 14:27:49.651170 24116 raft_consensus.cc:1067] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Signalling peer bc8c63a642ab43efa3b4829ea4294d37 to start an election
I20260924 14:27:49.652828 23929 leader_election.cc:288] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 3 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:49.653254 23855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:49.652806 23931 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700"
dest_uuid: "bc8c63a642ab43efa3b4829ea4294d37"
 from {username='slave'} at 127.0.0.1:42938
I20260924 14:27:49.653972 23855 raft_consensus.cc:3037] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:49.654043 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458a9bce2fb24e0d81141d72eef8e669" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:49.654640 23931 raft_consensus.cc:482] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:27:49.655126 23931 raft_consensus.cc:3037] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:49.654744 23780 raft_consensus.cc:3032] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:49.658869 23780 raft_consensus.cc:729] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:49.659605 23780 consensus_queue.cc:260] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:49.660817 23780 raft_consensus.cc:3037] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:49.661166 23855 raft_consensus.cc:2454] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 3.
I20260924 14:27:49.662227 23896 leader_election.cc:302] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7744d9495c594ba59d47a45af215ff8f, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:49.662772 23678 catalog_manager.cc:5495] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f reported cstate change: term changed from 4 to 5, leader changed from 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193) to 7744d9495c594ba59d47a45af215ff8f (127.22.151.194). New cstate: current_term: 5 leader_uuid: "7744d9495c594ba59d47a45af215ff8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } }
I20260924 14:27:49.663739 24108 raft_consensus.cc:2781] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:49.663784 23931 raft_consensus.cc:504] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:49.666255 23931 leader_election.cc:288] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 3 election: Requested vote from peers 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675), 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295)
I20260924 14:27:49.666474 23781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b18601c6e1f45358f9a1ad392cff957"
I20260924 14:27:49.666999 23781 raft_consensus.cc:3032] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:27:49.667155 24108 raft_consensus.cc:686] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Becoming Leader. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:49.667320 23781 raft_consensus.cc:729] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b18601c6e1f45358f9a1ad392cff957, State: Running, Role: LEADER
I20260924 14:27:49.667471 23780 raft_consensus.cc:2454] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 3.
I20260924 14:27:49.668532 23855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc8a59ba9b94ffb9cb345e3eef41700" candidate_uuid: "bc8c63a642ab43efa3b4829ea4294d37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7744d9495c594ba59d47a45af215ff8f"
I20260924 14:27:49.668111 24108 consensus_queue.cc:237] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:49.668128 23781 consensus_queue.cc:260] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:49.669083 23855 raft_consensus.cc:3037] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:49.669484 23781 raft_consensus.cc:3037] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:27:49.675967 23781 raft_consensus.cc:2454] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 3.
I20260924 14:27:49.676638 23855 raft_consensus.cc:2454] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8c63a642ab43efa3b4829ea4294d37 in term 3.
I20260924 14:27:49.677340 23894 leader_election.cc:302] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b18601c6e1f45358f9a1ad392cff957, bc8c63a642ab43efa3b4829ea4294d37; no voters: 
I20260924 14:27:49.679026 24141 raft_consensus.cc:2781] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:27:49.679535 24141 raft_consensus.cc:686] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Becoming Leader. State: Replica: bc8c63a642ab43efa3b4829ea4294d37, State: Running, Role: LEADER
I20260924 14:27:49.680490 24141 consensus_queue.cc:237] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } }
I20260924 14:27:49.681926 23678 catalog_manager.cc:5495] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 reported cstate change: term changed from 2 to 3, leader changed from 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193) to bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:49.694326 23678 catalog_manager.cc:5495] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 reported cstate change: term changed from 2 to 3, leader changed from 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193) to bc8c63a642ab43efa3b4829ea4294d37 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "bc8c63a642ab43efa3b4829ea4294d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 } health_report { overall_health: HEALTHY } } }
I20260924 14:27:49.755867 23781 raft_consensus.cc:1261] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Refusing update from remote peer 7744d9495c594ba59d47a45af215ff8f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:27:49.756975 24132 consensus_queue.cc:1035] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:27:49.765781 23931 raft_consensus.cc:1261] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Refusing update from remote peer 7744d9495c594ba59d47a45af215ff8f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:27:49.768880 24132 consensus_queue.cc:1035] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:27:50.077154 23855 raft_consensus.cc:1261] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Refusing update from remote peer bc8c63a642ab43efa3b4829ea4294d37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:50.078224 24141 consensus_queue.cc:1035] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:50.086606 23781 raft_consensus.cc:1261] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Refusing update from remote peer bc8c63a642ab43efa3b4829ea4294d37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:50.089087 24141 consensus_queue.cc:1035] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:50.164192 23855 raft_consensus.cc:1261] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Refusing update from remote peer bc8c63a642ab43efa3b4829ea4294d37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:50.164883 23781 raft_consensus.cc:1261] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Refusing update from remote peer bc8c63a642ab43efa3b4829ea4294d37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:27:50.166427 24141 consensus_queue.cc:1035] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7744d9495c594ba59d47a45af215ff8f" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:27:50.167498 24150 consensus_queue.cc:1035] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:27:50.183758 23931 raft_consensus.cc:1261] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 5 FOLLOWER]: Refusing update from remote peer 7744d9495c594ba59d47a45af215ff8f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:27:50.186111 24132 consensus_queue.cc:1035] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8c63a642ab43efa3b4829ea4294d37" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:27:50.201483 23781 raft_consensus.cc:1261] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 5 FOLLOWER]: Refusing update from remote peer 7744d9495c594ba59d47a45af215ff8f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:27:50.203243 24133 consensus_queue.cc:1035] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b18601c6e1f45358f9a1ad392cff957" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:27:50.286679 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:50.293004 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:50.293432 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:51.294276 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:51.299647 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:51.300067 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:52.737214 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:52.742340 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:52.742764 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260924 14:27:53.216790 23803 debug-util.cc:398] Leaking SignalData structure 0x7b08001c0f60 after lost signal to thread 23138
W20260924 14:27:53.217646 23803 debug-util.cc:398] Leaking SignalData structure 0x7b08001cb980 after lost signal to thread 23714
W20260924 14:27:53.218474 23803 debug-util.cc:398] Leaking SignalData structure 0x7b08001abe00 after lost signal to thread 23806
W20260924 14:27:53.219480 23803 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9f20 after lost signal to thread 23881
W20260924 14:27:53.221017 23803 debug-util.cc:398] Leaking SignalData structure 0x7b08001dbe40 after lost signal to thread 23955
I20260924 14:27:53.743676 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:53.749028 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:53.749478 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:54.750267 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:54.755503 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:54.755927 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:55.756721 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:55.761788 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:55.762198 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:56.763016 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:56.768226 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:56.768611 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:57.769424 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:57.774757 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:57.775218 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:58.776178 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:27:58.782506 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:27:58.782977 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:27:59.815119 23135 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260924 14:27:59.815737 23135 tablet_server.cc:178] TabletServer@127.22.151.193:0 shutting down...
W20260924 14:27:59.820644 23894 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260924 14:27:59.825160 23894 consensus_peers.cc:487] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 -> Peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Couldn't send request to peer 5b18601c6e1f45358f9a1ad392cff957. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:27:59.825827 23894 consensus_peers.cc:487] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 -> Peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Couldn't send request to peer 5b18601c6e1f45358f9a1ad392cff957. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:27:59.847852 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:27:59.848603 23135 tablet_replica.cc:330] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957: stopping tablet replica
I20260924 14:27:59.849632 23135 raft_consensus.cc:2229] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.850289 23135 raft_consensus.cc:2258] T 1ca1ed23241b46cb8c65b57f65f85588 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.853045 23135 tablet_replica.cc:330] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957: stopping tablet replica
I20260924 14:27:59.853637 23135 raft_consensus.cc:2229] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:27:59.858052 23135 raft_consensus.cc:2258] T f05549cf4bfc4950856a68b77823b204 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.860013 23135 tablet_replica.cc:330] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957: stopping tablet replica
I20260924 14:27:59.860569 23135 raft_consensus.cc:2229] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.860978 23135 raft_consensus.cc:2258] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.862604 23135 tablet_replica.cc:330] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957: stopping tablet replica
I20260924 14:27:59.863109 23135 raft_consensus.cc:2229] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:27:59.863903 23135 raft_consensus.cc:2258] T 6528eea31cb945f89f6bbb6507d2b57e P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.865567 23135 tablet_replica.cc:330] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957: stopping tablet replica
I20260924 14:27:59.866077 23135 raft_consensus.cc:2229] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.866473 23135 raft_consensus.cc:2258] T 5a4565b47ba44cc09bc0850742b28c57 P 5b18601c6e1f45358f9a1ad392cff957 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.868099 23135 tablet_replica.cc:330] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957: stopping tablet replica
I20260924 14:27:59.868583 23135 raft_consensus.cc:2229] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:27:59.869343 23135 raft_consensus.cc:2258] T 811a523968ff4d059b0f66e8ee15b059 P 5b18601c6e1f45358f9a1ad392cff957 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.871114 23135 tablet_replica.cc:330] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957: stopping tablet replica
I20260924 14:27:59.871667 23135 raft_consensus.cc:2229] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.872117 23135 raft_consensus.cc:2258] T aac851188d724766ac3d80b1a1a6b98b P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.873836 23135 tablet_replica.cc:330] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957: stopping tablet replica
I20260924 14:27:59.874313 23135 raft_consensus.cc:2229] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.874866 23135 raft_consensus.cc:2258] T 141131b2a0824d3cb3c82671609b0e19 P 5b18601c6e1f45358f9a1ad392cff957 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.876727 23135 tablet_replica.cc:330] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957: stopping tablet replica
I20260924 14:27:59.877247 23135 raft_consensus.cc:2229] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:27:59.878132 23135 raft_consensus.cc:2258] T cb58b7233f5a44daaf2209d0394cd248 P 5b18601c6e1f45358f9a1ad392cff957 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.879910 23135 tablet_replica.cc:330] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957: stopping tablet replica
I20260924 14:27:59.880367 23135 raft_consensus.cc:2229] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.880781 23135 raft_consensus.cc:2258] T 458a9bce2fb24e0d81141d72eef8e669 P 5b18601c6e1f45358f9a1ad392cff957 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.927739 23135 tablet_server.cc:194] TabletServer@127.22.151.193:0 shutdown complete.
I20260924 14:27:59.949306 23135 tablet_server.cc:178] TabletServer@127.22.151.194:0 shutting down...
W20260924 14:27:59.953521 23820 consensus_peers.cc:487] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f -> Peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Couldn't send request to peer 5b18601c6e1f45358f9a1ad392cff957. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:40675: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:27:59.959733 23896 consensus_peers.cc:487] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 -> Peer 7744d9495c594ba59d47a45af215ff8f (127.22.151.194:46295): Couldn't send request to peer 7744d9495c594ba59d47a45af215ff8f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:27:59.979282 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:27:59.979952 23135 tablet_replica.cc:330] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f: stopping tablet replica
I20260924 14:27:59.980571 23135 raft_consensus.cc:2229] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.981041 23135 raft_consensus.cc:2258] T 1ca1ed23241b46cb8c65b57f65f85588 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.982900 23135 tablet_replica.cc:330] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f: stopping tablet replica
I20260924 14:27:59.983407 23135 raft_consensus.cc:2229] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.983893 23135 raft_consensus.cc:2258] T 8fc8a59ba9b94ffb9cb345e3eef41700 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.985448 23135 tablet_replica.cc:330] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f: stopping tablet replica
I20260924 14:27:59.985953 23135 raft_consensus.cc:2229] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.986397 23135 raft_consensus.cc:2258] T 458a9bce2fb24e0d81141d72eef8e669 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.988059 23135 tablet_replica.cc:330] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f: stopping tablet replica
I20260924 14:27:59.988524 23135 raft_consensus.cc:2229] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.988920 23135 raft_consensus.cc:2258] T cb58b7233f5a44daaf2209d0394cd248 P 7744d9495c594ba59d47a45af215ff8f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.990425 23135 tablet_replica.cc:330] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f: stopping tablet replica
I20260924 14:27:59.990888 23135 raft_consensus.cc:2229] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 4 LEADER]: Raft consensus shutting down.
I20260924 14:27:59.991716 23135 raft_consensus.cc:2258] T 141131b2a0824d3cb3c82671609b0e19 P 7744d9495c594ba59d47a45af215ff8f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.993414 23135 tablet_replica.cc:330] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f: stopping tablet replica
I20260924 14:27:59.993904 23135 raft_consensus.cc:2229] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:27:59.994325 23135 raft_consensus.cc:2258] T 811a523968ff4d059b0f66e8ee15b059 P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.995926 23135 tablet_replica.cc:330] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f: stopping tablet replica
I20260924 14:27:59.996376 23135 raft_consensus.cc:2229] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 4 LEADER]: Raft consensus shutting down.
I20260924 14:27:59.997076 23135 raft_consensus.cc:2258] T aac851188d724766ac3d80b1a1a6b98b P 7744d9495c594ba59d47a45af215ff8f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 14:27:59.998737 23135 tablet_replica.cc:330] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f: stopping tablet replica
I20260924 14:27:59.999267 23135 raft_consensus.cc:2229] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 5 LEADER]: Raft consensus shutting down.
I20260924 14:28:00.000030 23135 raft_consensus.cc:2258] T 5a4565b47ba44cc09bc0850742b28c57 P 7744d9495c594ba59d47a45af215ff8f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.001879 23135 tablet_replica.cc:330] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f: stopping tablet replica
I20260924 14:28:00.002620 23135 raft_consensus.cc:2229] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:00.003113 23135 raft_consensus.cc:2258] T 6528eea31cb945f89f6bbb6507d2b57e P 7744d9495c594ba59d47a45af215ff8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.005326 23135 tablet_replica.cc:330] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f: stopping tablet replica
I20260924 14:28:00.005780 23135 raft_consensus.cc:2229] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:00.006173 23135 raft_consensus.cc:2258] T f05549cf4bfc4950856a68b77823b204 P 7744d9495c594ba59d47a45af215ff8f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.036592 23135 tablet_server.cc:194] TabletServer@127.22.151.194:0 shutdown complete.
I20260924 14:28:00.056198 23135 tablet_server.cc:178] TabletServer@127.22.151.195:0 shutting down...
I20260924 14:28:00.084885 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
W20260924 14:28:00.084796 23894 consensus_peers.cc:487] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 -> Peer 5b18601c6e1f45358f9a1ad392cff957 (127.22.151.193:40675): Couldn't send request to peer 5b18601c6e1f45358f9a1ad392cff957. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:40675: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:00.085677 23135 tablet_replica.cc:330] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37: stopping tablet replica
I20260924 14:28:00.086371 23135 raft_consensus.cc:2229] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:00.086863 23135 raft_consensus.cc:2258] T cb58b7233f5a44daaf2209d0394cd248 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.088783 23135 tablet_replica.cc:330] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37: stopping tablet replica
I20260924 14:28:00.089303 23135 raft_consensus.cc:2229] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:28:00.090080 23135 raft_consensus.cc:2258] T 458a9bce2fb24e0d81141d72eef8e669 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.091892 23135 tablet_replica.cc:330] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37: stopping tablet replica
I20260924 14:28:00.092453 23135 raft_consensus.cc:2229] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:00.092916 23135 raft_consensus.cc:2258] T f05549cf4bfc4950856a68b77823b204 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.094624 23135 tablet_replica.cc:330] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37: stopping tablet replica
I20260924 14:28:00.095108 23135 raft_consensus.cc:2229] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:00.095960 23135 raft_consensus.cc:2258] T 1ca1ed23241b46cb8c65b57f65f85588 P bc8c63a642ab43efa3b4829ea4294d37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.097786 23135 tablet_replica.cc:330] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37: stopping tablet replica
I20260924 14:28:00.098276 23135 raft_consensus.cc:2229] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:00.098716 23135 raft_consensus.cc:2258] T 141131b2a0824d3cb3c82671609b0e19 P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.100415 23135 tablet_replica.cc:330] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37: stopping tablet replica
I20260924 14:28:00.100941 23135 raft_consensus.cc:2229] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:00.101359 23135 raft_consensus.cc:2258] T aac851188d724766ac3d80b1a1a6b98b P bc8c63a642ab43efa3b4829ea4294d37 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.103034 23135 tablet_replica.cc:330] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37: stopping tablet replica
I20260924 14:28:00.103776 23135 raft_consensus.cc:2229] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:00.104292 23135 raft_consensus.cc:2258] T 811a523968ff4d059b0f66e8ee15b059 P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.105973 23135 tablet_replica.cc:330] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37: stopping tablet replica
I20260924 14:28:00.106442 23135 raft_consensus.cc:2229] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:28:00.107141 23135 raft_consensus.cc:2258] T 8fc8a59ba9b94ffb9cb345e3eef41700 P bc8c63a642ab43efa3b4829ea4294d37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.108683 23135 tablet_replica.cc:330] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37: stopping tablet replica
I20260924 14:28:00.109126 23135 raft_consensus.cc:2229] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:00.109511 23135 raft_consensus.cc:2258] T 6528eea31cb945f89f6bbb6507d2b57e P bc8c63a642ab43efa3b4829ea4294d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.111002 23135 tablet_replica.cc:330] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37: stopping tablet replica
I20260924 14:28:00.111408 23135 raft_consensus.cc:2229] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:00.111826 23135 raft_consensus.cc:2258] T 5a4565b47ba44cc09bc0850742b28c57 P bc8c63a642ab43efa3b4829ea4294d37 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.141009 23135 tablet_server.cc:194] TabletServer@127.22.151.195:0 shutdown complete.
I20260924 14:28:00.161257 23135 master.cc:537] Master@127.22.151.254:46333 shutting down...
I20260924 14:28:00.196378 23135 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:00.197331 23135 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:00.197641 23135 tablet_replica.cc:330] T 00000000000000000000000000000000 P bfaa0f81c05c413a87b0d9a75b8c7191: stopping tablet replica
I20260924 14:28:00.217676 23135 master.cc:552] Master@127.22.151.254:46333 shutdown complete.
5b18601c6e1f45358f9a1ad392cff957  5
7744d9495c594ba59d47a45af215ff8f  3
bc8c63a642ab43efa3b4829ea4294d37  2
5b18601c6e1f45358f9a1ad392cff957  4
7744d9495c594ba59d47a45af215ff8f  3
bc8c63a642ab43efa3b4829ea4294d37  3
5b18601c6e1f45358f9a1ad392cff957  8
7744d9495c594ba59d47a45af215ff8f  1
bc8c63a642ab43efa3b4829ea4294d37  1
5b18601c6e1f45358f9a1ad392cff957  4
7744d9495c594ba59d47a45af215ff8f  3
bc8c63a642ab43efa3b4829ea4294d37  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (29197 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260924 14:28:00.263430 23135 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.22.151.254:36341
I20260924 14:28:00.264547 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:28:00.270675 24176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:00.271243 24177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:00.271296 24179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:00.272001 23135 server_base.cc:1034] running on GCE node
I20260924 14:28:00.273134 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:28:00.273340 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:28:00.273556 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260080273541 us; error 0 us; skew 500 ppm
I20260924 14:28:00.274108 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:28:00.280299 23135 webserver.cc:460] Webserver started at http://127.22.151.254:39309/ using document root <none> and password file <none>
I20260924 14:28:00.280789 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:28:00.280974 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:28:00.281270 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:28:00.282574 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/master-0-root/instance:
uuid: "77786dfd8b794984995b91ede36bc1b6"
format_stamp: "Formatted at 2026-09-24 14:28:00 on dist-test-slave-pm5x"
I20260924 14:28:00.287307 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260924 14:28:00.290916 24184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:00.291751 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260924 14:28:00.292026 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/master-0-root
uuid: "77786dfd8b794984995b91ede36bc1b6"
format_stamp: "Formatted at 2026-09-24 14:28:00 on dist-test-slave-pm5x"
I20260924 14:28:00.292292 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:28:00.308815 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:28:00.310082 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:28:00.348277 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.254:36341
I20260924 14:28:00.348379 24235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.254:36341 every 8 connection(s)
I20260924 14:28:00.352334 24236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:00.363476 24236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6: Bootstrap starting.
I20260924 14:28:00.368575 24236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:00.373436 24236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6: No bootstrap required, opened a new log
I20260924 14:28:00.375653 24236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77786dfd8b794984995b91ede36bc1b6" member_type: VOTER }
I20260924 14:28:00.376180 24236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:00.376438 24236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77786dfd8b794984995b91ede36bc1b6, State: Initialized, Role: FOLLOWER
I20260924 14:28:00.377059 24236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77786dfd8b794984995b91ede36bc1b6" member_type: VOTER }
I20260924 14:28:00.377472 24236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 14:28:00.377741 24236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 14:28:00.378077 24236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:00.382401 24236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77786dfd8b794984995b91ede36bc1b6" member_type: VOTER }
I20260924 14:28:00.382941 24236 leader_election.cc:302] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77786dfd8b794984995b91ede36bc1b6; no voters: 
I20260924 14:28:00.384151 24236 leader_election.cc:288] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260924 14:28:00.384616 24239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:00.385967 24239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 1 LEADER]: Becoming Leader. State: Replica: 77786dfd8b794984995b91ede36bc1b6, State: Running, Role: LEADER
I20260924 14:28:00.386655 24239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77786dfd8b794984995b91ede36bc1b6" member_type: VOTER }
I20260924 14:28:00.387176 24236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 14:28:00.393987 24241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77786dfd8b794984995b91ede36bc1b6. Latest consensus state: current_term: 1 leader_uuid: "77786dfd8b794984995b91ede36bc1b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77786dfd8b794984995b91ede36bc1b6" member_type: VOTER } }
I20260924 14:28:00.394295 24240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77786dfd8b794984995b91ede36bc1b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77786dfd8b794984995b91ede36bc1b6" member_type: VOTER } }
I20260924 14:28:00.394824 24241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [sys.catalog]: This master's current role is: LEADER
I20260924 14:28:00.394950 24240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [sys.catalog]: This master's current role is: LEADER
I20260924 14:28:00.398474 24249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 14:28:00.406390 24249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 14:28:00.407668 23135 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 14:28:00.416520 24249 catalog_manager.cc:1347] Generated new cluster ID: 9373f63f7e294283ab5618782dee8515
I20260924 14:28:00.416841 24249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 14:28:00.441574 24249 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 14:28:00.443264 24249 catalog_manager.cc:1504] Loading token signing keys...
I20260924 14:28:00.459302 24249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6: Generated new TSK 0
I20260924 14:28:00.460047 24249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 14:28:00.474306 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:28:00.480793 24257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:00.481896 24258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:00.483825 24260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:00.484730 23135 server_base.cc:1034] running on GCE node
I20260924 14:28:00.485813 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:28:00.486028 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:28:00.486197 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260080486183 us; error 0 us; skew 500 ppm
I20260924 14:28:00.486773 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:28:00.490379 23135 webserver.cc:460] Webserver started at http://127.22.151.193:46683/ using document root <none> and password file <none>
I20260924 14:28:00.491068 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:28:00.491297 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:28:00.491676 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:28:00.493646 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/instance:
uuid: "cafc2c5ad4df4224a5ccc4893de70144"
format_stamp: "Formatted at 2026-09-24 14:28:00 on dist-test-slave-pm5x"
I20260924 14:28:00.499341 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260924 14:28:00.502776 24265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:00.503748 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260924 14:28:00.504042 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root
uuid: "cafc2c5ad4df4224a5ccc4893de70144"
format_stamp: "Formatted at 2026-09-24 14:28:00 on dist-test-slave-pm5x"
I20260924 14:28:00.504343 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:28:00.531265 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:28:00.532583 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:28:00.534065 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:28:00.536362 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:28:00.536589 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:00.536849 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:28:00.537024 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:00.579366 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.193:44507
I20260924 14:28:00.579560 24327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.193:44507 every 8 connection(s)
I20260924 14:28:00.584458 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:28:00.593364 24332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:00.596079 24333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:00.601941 24335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:00.602890 23135 server_base.cc:1034] running on GCE node
I20260924 14:28:00.604009 24328 heartbeater.cc:347] Connected to a master server at 127.22.151.254:36341
I20260924 14:28:00.604713 24328 heartbeater.cc:464] Registering TS with master...
I20260924 14:28:00.605650 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260924 14:28:00.605948 24328 heartbeater.cc:511] Master 127.22.151.254:36341 requested a full tablet report, sending...
W20260924 14:28:00.605991 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:28:00.606396 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260080606382 us; error 0 us; skew 500 ppm
I20260924 14:28:00.607231 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:28:00.608634 24201 ts_manager.cc:194] Registered new tserver with Master: cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:00.610273 23135 webserver.cc:460] Webserver started at http://127.22.151.194:34645/ using document root <none> and password file <none>
I20260924 14:28:00.610802 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:28:00.611075 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:28:00.611347 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:28:00.611639 24201 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34764
I20260924 14:28:00.614382 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/instance:
uuid: "61d316eb5dfe405c95019383bbebecaa"
format_stamp: "Formatted at 2026-09-24 14:28:00 on dist-test-slave-pm5x"
I20260924 14:28:00.620534 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:28:00.625349 24340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:00.626253 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260924 14:28:00.626588 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root
uuid: "61d316eb5dfe405c95019383bbebecaa"
format_stamp: "Formatted at 2026-09-24 14:28:00 on dist-test-slave-pm5x"
I20260924 14:28:00.627048 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:28:00.651774 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:28:00.653047 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:28:00.654515 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:28:00.656747 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:28:00.656967 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:00.657202 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:28:00.657363 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:00.702723 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.194:42533
I20260924 14:28:00.702850 24402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.194:42533 every 8 connection(s)
I20260924 14:28:00.707623 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:28:00.715301 24407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:00.715324 24406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:00.719084 23135 server_base.cc:1034] running on GCE node
W20260924 14:28:00.719760 24409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:00.720932 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:28:00.721205 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:28:00.721390 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260080721377 us; error 0 us; skew 500 ppm
I20260924 14:28:00.722100 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:28:00.722855 24403 heartbeater.cc:347] Connected to a master server at 127.22.151.254:36341
I20260924 14:28:00.723331 24403 heartbeater.cc:464] Registering TS with master...
I20260924 14:28:00.724231 24403 heartbeater.cc:511] Master 127.22.151.254:36341 requested a full tablet report, sending...
I20260924 14:28:00.725071 23135 webserver.cc:460] Webserver started at http://127.22.151.195:42555/ using document root <none> and password file <none>
I20260924 14:28:00.725639 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:28:00.725848 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:28:00.726162 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:28:00.726552 24201 ts_manager.cc:194] Registered new tserver with Master: 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:00.727774 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/instance:
uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
format_stamp: "Formatted at 2026-09-24 14:28:00 on dist-test-slave-pm5x"
I20260924 14:28:00.728782 24201 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34774
I20260924 14:28:00.734150 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260924 14:28:00.739295 24414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:00.740289 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260924 14:28:00.740666 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root
uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
format_stamp: "Formatted at 2026-09-24 14:28:00 on dist-test-slave-pm5x"
I20260924 14:28:00.741103 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:28:00.767549 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:28:00.768762 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:28:00.770154 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:28:00.772404 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:28:00.772625 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:00.772854 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:28:00.773023 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:00.814098 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.195:35781
I20260924 14:28:00.814186 24476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.195:35781 every 8 connection(s)
I20260924 14:28:00.830026 24477 heartbeater.cc:347] Connected to a master server at 127.22.151.254:36341
I20260924 14:28:00.830375 24477 heartbeater.cc:464] Registering TS with master...
I20260924 14:28:00.831064 24477 heartbeater.cc:511] Master 127.22.151.254:36341 requested a full tablet report, sending...
I20260924 14:28:00.833079 24201 ts_manager.cc:194] Registered new tserver with Master: 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:00.833523 23135 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016620746s
I20260924 14:28:00.835665 24201 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34786
I20260924 14:28:00.837150 23135 test_util.cc:268] Using random seed: 1747747274
I20260924 14:28:00.863933 24201 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34798:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260924 14:28:00.870137 24201 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 14:28:01.088142 24442 tablet_service.cc:1467] Processing CreateTablet for tablet b0b642240b2a48ff984338b3604080ef (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260924 14:28:01.089620 24442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b642240b2a48ff984338b3604080ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.098513 24441 tablet_service.cc:1467] Processing CreateTablet for tablet fb4aa8a7302f42b688f37d4fdb892b67 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260924 14:28:01.100102 24441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb4aa8a7302f42b688f37d4fdb892b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.101182 24368 tablet_service.cc:1467] Processing CreateTablet for tablet b0b642240b2a48ff984338b3604080ef (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260924 14:28:01.102530 24368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b642240b2a48ff984338b3604080ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.104216 24436 tablet_service.cc:1467] Processing CreateTablet for tablet ce753b9bac814f17a869f4f8d66deca6 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260924 14:28:01.104673 24367 tablet_service.cc:1467] Processing CreateTablet for tablet fb4aa8a7302f42b688f37d4fdb892b67 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260924 14:28:01.105669 24436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce753b9bac814f17a869f4f8d66deca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.106874 24361 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa9c52e97d0444b8ec17719ea5a4ac6 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260924 14:28:01.108639 24437 tablet_service.cc:1467] Processing CreateTablet for tablet cbae9b0be0e4421eab738bd8fd45a1e1 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260924 14:28:01.108726 24365 tablet_service.cc:1467] Processing CreateTablet for tablet b53acab36f9a42838efe3526f4aed03a (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260924 14:28:01.109988 24437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbae9b0be0e4421eab738bd8fd45a1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.110154 24362 tablet_service.cc:1467] Processing CreateTablet for tablet ce753b9bac814f17a869f4f8d66deca6 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260924 14:28:01.111796 24364 tablet_service.cc:1467] Processing CreateTablet for tablet e79eded71d354d4095b73f2f7f7153e5 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260924 14:28:01.112632 24363 tablet_service.cc:1467] Processing CreateTablet for tablet cbae9b0be0e4421eab738bd8fd45a1e1 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260924 14:28:01.107172 24366 tablet_service.cc:1467] Processing CreateTablet for tablet dfe06d46f9904520a8709a2b3824f11c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260924 14:28:01.114499 24439 tablet_service.cc:1467] Processing CreateTablet for tablet b53acab36f9a42838efe3526f4aed03a (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260924 14:28:01.115823 24439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b53acab36f9a42838efe3526f4aed03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.116420 24367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb4aa8a7302f42b688f37d4fdb892b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.119870 24362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce753b9bac814f17a869f4f8d66deca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.120544 24366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfe06d46f9904520a8709a2b3824f11c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.121441 24440 tablet_service.cc:1467] Processing CreateTablet for tablet dfe06d46f9904520a8709a2b3824f11c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260924 14:28:01.123847 24364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e79eded71d354d4095b73f2f7f7153e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.125694 24360 tablet_service.cc:1467] Processing CreateTablet for tablet 3550e2841c1f41939833b5588fb35875 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260924 14:28:01.112670 24438 tablet_service.cc:1467] Processing CreateTablet for tablet e79eded71d354d4095b73f2f7f7153e5 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260924 14:28:01.130092 24435 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa9c52e97d0444b8ec17719ea5a4ac6 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260924 14:28:01.130662 24434 tablet_service.cc:1467] Processing CreateTablet for tablet 3550e2841c1f41939833b5588fb35875 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260924 14:28:01.131577 24360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3550e2841c1f41939833b5588fb35875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.132320 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b53acab36f9a42838efe3526f4aed03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.132936 24361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa9c52e97d0444b8ec17719ea5a4ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.132941 24433 tablet_service.cc:1467] Processing CreateTablet for tablet bec046486fdc464290fafd6e4978632e (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260924 14:28:01.135481 24363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbae9b0be0e4421eab738bd8fd45a1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.123399 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfe06d46f9904520a8709a2b3824f11c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.138176 24293 tablet_service.cc:1467] Processing CreateTablet for tablet b0b642240b2a48ff984338b3604080ef (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260924 14:28:01.138679 24434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3550e2841c1f41939833b5588fb35875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.139549 24293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b642240b2a48ff984338b3604080ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.142287 24438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e79eded71d354d4095b73f2f7f7153e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.143159 24433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bec046486fdc464290fafd6e4978632e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.147814 24435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa9c52e97d0444b8ec17719ea5a4ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.161729 24359 tablet_service.cc:1467] Processing CreateTablet for tablet bec046486fdc464290fafd6e4978632e (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260924 14:28:01.163612 24359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bec046486fdc464290fafd6e4978632e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.174078 24292 tablet_service.cc:1467] Processing CreateTablet for tablet fb4aa8a7302f42b688f37d4fdb892b67 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260924 14:28:01.175213 24290 tablet_service.cc:1467] Processing CreateTablet for tablet b53acab36f9a42838efe3526f4aed03a (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260924 14:28:01.176640 24291 tablet_service.cc:1467] Processing CreateTablet for tablet dfe06d46f9904520a8709a2b3824f11c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260924 14:28:01.177487 24289 tablet_service.cc:1467] Processing CreateTablet for tablet e79eded71d354d4095b73f2f7f7153e5 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260924 14:28:01.178881 24284 tablet_service.cc:1467] Processing CreateTablet for tablet bec046486fdc464290fafd6e4978632e (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260924 14:28:01.179762 24288 tablet_service.cc:1467] Processing CreateTablet for tablet cbae9b0be0e4421eab738bd8fd45a1e1 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260924 14:28:01.181988 24287 tablet_service.cc:1467] Processing CreateTablet for tablet ce753b9bac814f17a869f4f8d66deca6 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260924 14:28:01.184197 24286 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa9c52e97d0444b8ec17719ea5a4ac6 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260924 14:28:01.164863 24441 tablet_service.cc:1467] Processing CreateTablet for tablet 86057fee00d44a4c9607d990f2782813 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260924 14:28:01.187922 24285 tablet_service.cc:1467] Processing CreateTablet for tablet 3550e2841c1f41939833b5588fb35875 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260924 14:28:01.189163 24492 tablet_bootstrap.cc:492] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:01.189498 24292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb4aa8a7302f42b688f37d4fdb892b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.189292 24441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86057fee00d44a4c9607d990f2782813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.190220 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b53acab36f9a42838efe3526f4aed03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.190938 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bec046486fdc464290fafd6e4978632e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.191567 24287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce753b9bac814f17a869f4f8d66deca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.192178 24285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3550e2841c1f41939833b5588fb35875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.199752 24291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfe06d46f9904520a8709a2b3824f11c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.203461 24289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e79eded71d354d4095b73f2f7f7153e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.207782 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbae9b0be0e4421eab738bd8fd45a1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.211799 24286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa9c52e97d0444b8ec17719ea5a4ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.223430 24492 tablet_bootstrap.cc:654] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:01.272773 24436 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdd9163fb2d44068b0f93f1bbfd7387 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260924 14:28:01.274618 24436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bdd9163fb2d44068b0f93f1bbfd7387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.276702 24362 tablet_service.cc:1467] Processing CreateTablet for tablet 86057fee00d44a4c9607d990f2782813 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260924 14:28:01.290158 24362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86057fee00d44a4c9607d990f2782813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.280459 24435 tablet_service.cc:1467] Processing CreateTablet for tablet 5be2b72b568242d3855180c905e78d71 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260924 14:28:01.280529 24434 tablet_service.cc:1467] Processing CreateTablet for tablet 3a06a0e1988d4a53b9590e40ca1481b0 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260924 14:28:01.288753 24363 tablet_service.cc:1467] Processing CreateTablet for tablet 3a06a0e1988d4a53b9590e40ca1481b0 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260924 14:28:01.293275 24361 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdd9163fb2d44068b0f93f1bbfd7387 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260924 14:28:01.294459 24363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a06a0e1988d4a53b9590e40ca1481b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.300184 24287 tablet_service.cc:1467] Processing CreateTablet for tablet 86057fee00d44a4c9607d990f2782813 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260924 14:28:01.303992 24287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86057fee00d44a4c9607d990f2782813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.301265 24365 tablet_service.cc:1467] Processing CreateTablet for tablet 5be2b72b568242d3855180c905e78d71 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260924 14:28:01.304488 24437 tablet_service.cc:1467] Processing CreateTablet for tablet a2f5e6c143534a47b388c7cd35bd1f4f (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260924 14:28:01.307473 24435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be2b72b568242d3855180c905e78d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.314157 24285 tablet_service.cc:1467] Processing CreateTablet for tablet 5be2b72b568242d3855180c905e78d71 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260924 14:28:01.315876 24433 tablet_service.cc:1467] Processing CreateTablet for tablet 4995099d467c4203bc2872f519b99950 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260924 14:28:01.320921 24286 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdd9163fb2d44068b0f93f1bbfd7387 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260924 14:28:01.322991 24441 tablet_service.cc:1467] Processing CreateTablet for tablet c9aebcc9d3ed438daf87d68bf2b8f6a9 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260924 14:28:01.324334 24439 tablet_service.cc:1467] Processing CreateTablet for tablet 0d8ea1ccd8b24bb3a8afb3e6f673c774 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260924 14:28:01.325385 24361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bdd9163fb2d44068b0f93f1bbfd7387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.336396 24285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be2b72b568242d3855180c905e78d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.340533 24368 tablet_service.cc:1467] Processing CreateTablet for tablet 0d8ea1ccd8b24bb3a8afb3e6f673c774 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260924 14:28:01.341969 24368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d8ea1ccd8b24bb3a8afb3e6f673c774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.346678 24362 tablet_service.cc:1467] Processing CreateTablet for tablet 6971512f7f9a4265bb19996677d61320 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260924 14:28:01.348001 24362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6971512f7f9a4265bb19996677d61320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.348750 24288 tablet_service.cc:1467] Processing CreateTablet for tablet 4995099d467c4203bc2872f519b99950 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260924 14:28:01.350080 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4995099d467c4203bc2872f519b99950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.352813 24360 tablet_service.cc:1467] Processing CreateTablet for tablet ce4efbdf1a8c482facd972a93dfea77f (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260924 14:28:01.353466 24366 tablet_service.cc:1467] Processing CreateTablet for tablet 5e48f04a45264e7d8be03ae56ab10560 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260924 14:28:01.348961 24284 tablet_service.cc:1467] Processing CreateTablet for tablet c9aebcc9d3ed438daf87d68bf2b8f6a9 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260924 14:28:01.355020 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9aebcc9d3ed438daf87d68bf2b8f6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.373296 24360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4efbdf1a8c482facd972a93dfea77f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.375057 24434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a06a0e1988d4a53b9590e40ca1481b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.382684 24433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4995099d467c4203bc2872f519b99950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.383826 24492 tablet_bootstrap.cc:492] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:01.384291 24492 ts_tablet_manager.cc:1397] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.196s	user 0.013s	sys 0.007s
I20260924 14:28:01.387099 24492 raft_consensus.cc:348] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:01.387923 24492 raft_consensus.cc:374] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:01.388226 24492 raft_consensus.cc:729] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:01.379429 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be2b72b568242d3855180c905e78d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.380896 24367 tablet_service.cc:1467] Processing CreateTablet for tablet c9aebcc9d3ed438daf87d68bf2b8f6a9 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260924 14:28:01.389062 24494 tablet_bootstrap.cc:492] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:01.389534 24437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2f5e6c143534a47b388c7cd35bd1f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.378552 24286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bdd9163fb2d44068b0f93f1bbfd7387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.395179 24366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e48f04a45264e7d8be03ae56ab10560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.397842 24367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9aebcc9d3ed438daf87d68bf2b8f6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.398660 24284 tablet_service.cc:1467] Processing CreateTablet for tablet 6971512f7f9a4265bb19996677d61320 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260924 14:28:01.399997 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6971512f7f9a4265bb19996677d61320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.390668 24492 consensus_queue.cc:260] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:01.326086 24289 tablet_service.cc:1467] Processing CreateTablet for tablet 3a06a0e1988d4a53b9590e40ca1481b0 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260924 14:28:01.404433 24440 tablet_service.cc:1467] Processing CreateTablet for tablet ce4efbdf1a8c482facd972a93dfea77f (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260924 14:28:01.406395 24441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9aebcc9d3ed438daf87d68bf2b8f6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.416841 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4efbdf1a8c482facd972a93dfea77f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.419765 24439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d8ea1ccd8b24bb3a8afb3e6f673c774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.415081 24359 tablet_service.cc:1467] Processing CreateTablet for tablet 4995099d467c4203bc2872f519b99950 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260924 14:28:01.421434 24359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4995099d467c4203bc2872f519b99950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.408800 24493 tablet_bootstrap.cc:492] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:01.433159 24284 tablet_service.cc:1467] Processing CreateTablet for tablet 5e48f04a45264e7d8be03ae56ab10560 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260924 14:28:01.432721 24293 tablet_service.cc:1467] Processing CreateTablet for tablet ce4efbdf1a8c482facd972a93dfea77f (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260924 14:28:01.402484 24442 tablet_service.cc:1467] Processing CreateTablet for tablet 6971512f7f9a4265bb19996677d61320 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
W20260924 14:28:01.435647 24248 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:01.436738 24442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6971512f7f9a4265bb19996677d61320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.439489 24289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a06a0e1988d4a53b9590e40ca1481b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.425441 24438 tablet_service.cc:1467] Processing CreateTablet for tablet 5e48f04a45264e7d8be03ae56ab10560 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260924 14:28:01.442543 24438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e48f04a45264e7d8be03ae56ab10560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.431784 24494 tablet_bootstrap.cc:654] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:01.448652 24492 ts_tablet_manager.cc:1428] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.001s
I20260924 14:28:01.449474 24477 heartbeater.cc:503] Master 127.22.151.254:36341 was elected leader, sending a full tablet report...
I20260924 14:28:01.391767 24364 tablet_service.cc:1467] Processing CreateTablet for tablet d7a8256035984c7fb87f7e9212b21dc7 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260924 14:28:01.451375 24364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7a8256035984c7fb87f7e9212b21dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.453732 24492 tablet_bootstrap.cc:492] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:01.461403 24290 tablet_service.cc:1467] Processing CreateTablet for tablet a2f5e6c143534a47b388c7cd35bd1f4f (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260924 14:28:01.462798 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2f5e6c143534a47b388c7cd35bd1f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.449474 24493 tablet_bootstrap.cc:654] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:01.463774 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e48f04a45264e7d8be03ae56ab10560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.467870 24292 tablet_service.cc:1467] Processing CreateTablet for tablet 0d8ea1ccd8b24bb3a8afb3e6f673c774 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260924 14:28:01.469197 24292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d8ea1ccd8b24bb3a8afb3e6f673c774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.477981 24437 tablet_service.cc:1467] Processing CreateTablet for tablet d7a8256035984c7fb87f7e9212b21dc7 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260924 14:28:01.486317 24291 tablet_service.cc:1467] Processing CreateTablet for tablet d7a8256035984c7fb87f7e9212b21dc7 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260924 14:28:01.487265 24293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4efbdf1a8c482facd972a93dfea77f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.495817 24363 tablet_service.cc:1467] Processing CreateTablet for tablet a2f5e6c143534a47b388c7cd35bd1f4f (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260924 14:28:01.497154 24363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2f5e6c143534a47b388c7cd35bd1f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.498386 24288 tablet_service.cc:1467] Processing CreateTablet for tablet 2c4bc71d12c14157982c98b4c7ca505c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260924 14:28:01.499794 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c4bc71d12c14157982c98b4c7ca505c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.479327 24437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7a8256035984c7fb87f7e9212b21dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.508172 24286 tablet_service.cc:1467] Processing CreateTablet for tablet e4aee388a7474ac1bb46b761831b809e (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260924 14:28:01.511662 24367 tablet_service.cc:1467] Processing CreateTablet for tablet 2c4bc71d12c14157982c98b4c7ca505c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260924 14:28:01.512954 24367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c4bc71d12c14157982c98b4c7ca505c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.516142 24439 tablet_service.cc:1467] Processing CreateTablet for tablet e4aee388a7474ac1bb46b761831b809e (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260924 14:28:01.525650 24439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4aee388a7474ac1bb46b761831b809e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.525403 24361 tablet_service.cc:1467] Processing CreateTablet for tablet e4aee388a7474ac1bb46b761831b809e (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260924 14:28:01.534942 24492 tablet_bootstrap.cc:654] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:01.534991 24440 tablet_service.cc:1467] Processing CreateTablet for tablet 2c4bc71d12c14157982c98b4c7ca505c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260924 14:28:01.536566 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c4bc71d12c14157982c98b4c7ca505c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.537534 24362 tablet_service.cc:1467] Processing CreateTablet for tablet dd02146c57f9474fba0afbcc145a7a2c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260924 14:28:01.540261 24365 tablet_service.cc:1467] Processing CreateTablet for tablet c775016b57f24851b9c52bd2d708b4ef (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260924 14:28:01.528311 24361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4aee388a7474ac1bb46b761831b809e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.544504 24291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7a8256035984c7fb87f7e9212b21dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.560232 24360 tablet_service.cc:1467] Processing CreateTablet for tablet 7514385d8e4a45e196a8b870b01c1c76 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260924 14:28:01.560072 24366 tablet_service.cc:1467] Processing CreateTablet for tablet c9065d0aec064841b81b70e65f668aac (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260924 14:28:01.563962 24366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9065d0aec064841b81b70e65f668aac. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:01.568315 24478 tablet.cc:2369] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 14:28:01.570111 24360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7514385d8e4a45e196a8b870b01c1c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.584131 24494 tablet_bootstrap.cc:492] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:01.584611 24494 ts_tablet_manager.cc:1397] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.196s	user 0.006s	sys 0.009s
I20260924 14:28:01.584740 24437 tablet_service.cc:1467] Processing CreateTablet for tablet c775016b57f24851b9c52bd2d708b4ef (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260924 14:28:01.586082 24437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c775016b57f24851b9c52bd2d708b4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.563995 24435 tablet_service.cc:1467] Processing CreateTablet for tablet 7514385d8e4a45e196a8b870b01c1c76 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260924 14:28:01.570510 24436 tablet_service.cc:1467] Processing CreateTablet for tablet dd02146c57f9474fba0afbcc145a7a2c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260924 14:28:01.587415 24285 tablet_service.cc:1467] Processing CreateTablet for tablet 7514385d8e4a45e196a8b870b01c1c76 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260924 14:28:01.588739 24435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7514385d8e4a45e196a8b870b01c1c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.572127 24287 tablet_service.cc:1467] Processing CreateTablet for tablet dd02146c57f9474fba0afbcc145a7a2c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260924 14:28:01.592563 24286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4aee388a7474ac1bb46b761831b809e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.597733 24362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd02146c57f9474fba0afbcc145a7a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.574469 24289 tablet_service.cc:1467] Processing CreateTablet for tablet c775016b57f24851b9c52bd2d708b4ef (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260924 14:28:01.607606 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c775016b57f24851b9c52bd2d708b4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.592666 24494 raft_consensus.cc:348] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:01.607255 24436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd02146c57f9474fba0afbcc145a7a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.614148 24493 tablet_bootstrap.cc:492] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:01.614616 24493 ts_tablet_manager.cc:1397] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.207s	user 0.003s	sys 0.014s
I20260924 14:28:01.615381 24328 heartbeater.cc:503] Master 127.22.151.254:36341 was elected leader, sending a full tablet report...
I20260924 14:28:01.617331 24493 raft_consensus.cc:348] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:01.618074 24493 raft_consensus.cc:374] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:01.618359 24493 raft_consensus.cc:729] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:01.623422 24441 tablet_service.cc:1467] Processing CreateTablet for tablet 6fb42513147f483f912eaec2c1151c50 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260924 14:28:01.628939 24365 tablet_service.cc:1467] Processing CreateTablet for tablet 6fb42513147f483f912eaec2c1151c50 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260924 14:28:01.634541 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fb42513147f483f912eaec2c1151c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.639580 24359 tablet_service.cc:1467] Processing CreateTablet for tablet 8b62b316cc05443990141d24402b1f4a (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260924 14:28:01.629129 24493 consensus_queue.cc:260] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:01.640971 24359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b62b316cc05443990141d24402b1f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.641839 24441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fb42513147f483f912eaec2c1151c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.643749 24288 tablet_service.cc:1467] Processing CreateTablet for tablet c9065d0aec064841b81b70e65f668aac (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260924 14:28:01.650774 24365 tablet_service.cc:1467] Processing CreateTablet for tablet 8df40ee44e21496dbc7ef5dd5e624432 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260924 14:28:01.652062 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df40ee44e21496dbc7ef5dd5e624432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.654721 24434 tablet_service.cc:1467] Processing CreateTablet for tablet c9065d0aec064841b81b70e65f668aac (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260924 14:28:01.656069 24434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9065d0aec064841b81b70e65f668aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.665941 24290 tablet_service.cc:1467] Processing CreateTablet for tablet f6bde1e6051745469ca811e5940bd8f9 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260924 14:28:01.633024 24364 tablet_service.cc:1467] Processing CreateTablet for tablet f6bde1e6051745469ca811e5940bd8f9 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260924 14:28:01.676752 24364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6bde1e6051745469ca811e5940bd8f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.679708 24285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7514385d8e4a45e196a8b870b01c1c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.680332 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6bde1e6051745469ca811e5940bd8f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.681025 24433 tablet_service.cc:1467] Processing CreateTablet for tablet f6bde1e6051745469ca811e5940bd8f9 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260924 14:28:01.688004 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9065d0aec064841b81b70e65f668aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.689241 24289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c775016b57f24851b9c52bd2d708b4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.611876 24494 raft_consensus.cc:374] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:01.691847 24494 raft_consensus.cc:729] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:01.692581 24494 consensus_queue.cc:260] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:01.698932 24287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd02146c57f9474fba0afbcc145a7a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.704316 24496 raft_consensus.cc:482] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:01.704790 24496 raft_consensus.cc:504] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:01.706670 24442 tablet_service.cc:1467] Processing CreateTablet for tablet bd5890410f3f4f22a1f26e09cc824bb2 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260924 14:28:01.707190 24496 leader_election.cc:288] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:01.671218 24368 tablet_service.cc:1467] Processing CreateTablet for tablet bd5890410f3f4f22a1f26e09cc824bb2 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260924 14:28:01.723694 24368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5890410f3f4f22a1f26e09cc824bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.736409 24501 raft_consensus.cc:482] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:01.736689 24493 ts_tablet_manager.cc:1428] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.122s	user 0.011s	sys 0.022s
I20260924 14:28:01.736948 24501 raft_consensus.cc:504] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:01.737831 24493 tablet_bootstrap.cc:492] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:01.736465 24438 tablet_service.cc:1467] Processing CreateTablet for tablet 8b62b316cc05443990141d24402b1f4a (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260924 14:28:01.740962 24433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6bde1e6051745469ca811e5940bd8f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.679317 24439 tablet_service.cc:1467] Processing CreateTablet for tablet 8df40ee44e21496dbc7ef5dd5e624432 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260924 14:28:01.703687 24361 tablet_service.cc:1467] Processing CreateTablet for tablet 37e78577921b4cf18418d98b6f6a30a1 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260924 14:28:01.739432 24501 leader_election.cc:288] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:01.744724 24403 heartbeater.cc:503] Master 127.22.151.254:36341 was elected leader, sending a full tablet report...
I20260924 14:28:01.779790 24290 tablet_service.cc:1467] Processing CreateTablet for tablet 8b62b316cc05443990141d24402b1f4a (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260924 14:28:01.780846 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b62b316cc05443990141d24402b1f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.763883 24438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b62b316cc05443990141d24402b1f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.800514 24492 tablet_bootstrap.cc:492] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:01.801060 24492 ts_tablet_manager.cc:1397] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.348s	user 0.020s	sys 0.003s
I20260924 14:28:01.804833 24442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5890410f3f4f22a1f26e09cc824bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.807389 24360 tablet_service.cc:1467] Processing CreateTablet for tablet 095719203855447b9091cc7846d566de (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260924 14:28:01.838511 24439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df40ee44e21496dbc7ef5dd5e624432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.840802 24361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e78577921b4cf18418d98b6f6a30a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.841714 24360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 095719203855447b9091cc7846d566de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.855199 24293 tablet_service.cc:1467] Processing CreateTablet for tablet bd5890410f3f4f22a1f26e09cc824bb2 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260924 14:28:01.858584 24293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5890410f3f4f22a1f26e09cc824bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.869047 24436 tablet_service.cc:1467] Processing CreateTablet for tablet 37e78577921b4cf18418d98b6f6a30a1 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260924 14:28:01.870452 24436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e78577921b4cf18418d98b6f6a30a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.872642 24292 tablet_service.cc:1467] Processing CreateTablet for tablet 6fb42513147f483f912eaec2c1151c50 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260924 14:28:01.874109 24292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fb42513147f483f912eaec2c1151c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.875949 24494 ts_tablet_manager.cc:1428] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.291s	user 0.013s	sys 0.016s
I20260924 14:28:01.884439 24493 tablet_bootstrap.cc:654] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:01.897028 24287 tablet_service.cc:1467] Processing CreateTablet for tablet 37e78577921b4cf18418d98b6f6a30a1 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260924 14:28:01.898375 24287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e78577921b4cf18418d98b6f6a30a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.901497 24289 tablet_service.cc:1467] Processing CreateTablet for tablet 095719203855447b9091cc7846d566de (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260924 14:28:01.902452 24364 tablet_service.cc:1467] Processing CreateTablet for tablet 93b3f9be3c4846a8b75e5e42099e6dbc (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260924 14:28:01.902796 24289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 095719203855447b9091cc7846d566de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.904423 24441 tablet_service.cc:1467] Processing CreateTablet for tablet 317bae4ebac443c8a2fada29c389ab79 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260924 14:28:01.905682 24441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317bae4ebac443c8a2fada29c389ab79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.910040 24435 tablet_service.cc:1467] Processing CreateTablet for tablet 93b3f9be3c4846a8b75e5e42099e6dbc (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260924 14:28:01.912011 24500 raft_consensus.cc:482] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:01.912627 24500 raft_consensus.cc:504] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:01.914736 24284 tablet_service.cc:1467] Processing CreateTablet for tablet d5b55145a82545d9b769ea9d04bf1d99 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260924 14:28:01.873173 24440 tablet_service.cc:1467] Processing CreateTablet for tablet 095719203855447b9091cc7846d566de (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260924 14:28:01.882145 24291 tablet_service.cc:1467] Processing CreateTablet for tablet 8df40ee44e21496dbc7ef5dd5e624432 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260924 14:28:01.886983 24359 tablet_service.cc:1467] Processing CreateTablet for tablet 317bae4ebac443c8a2fada29c389ab79 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260924 14:28:01.934816 24435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93b3f9be3c4846a8b75e5e42099e6dbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.942960 24494 tablet_bootstrap.cc:492] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:01.947342 24364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93b3f9be3c4846a8b75e5e42099e6dbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.972723 24362 tablet_service.cc:1467] Processing CreateTablet for tablet 7959d21c60de4745bebfd367d2367714 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260924 14:28:01.977710 24437 tablet_service.cc:1467] Processing CreateTablet for tablet 7959d21c60de4745bebfd367d2367714 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260924 14:28:01.992367 24437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7959d21c60de4745bebfd367d2367714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.995929 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 095719203855447b9091cc7846d566de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:01.953915 24500 leader_election.cc:288] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781), cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:02.006429 24288 tablet_service.cc:1467] Processing CreateTablet for tablet 93b3f9be3c4846a8b75e5e42099e6dbc (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260924 14:28:02.008662 24492 raft_consensus.cc:348] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.009454 24492 raft_consensus.cc:374] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.009739 24492 raft_consensus.cc:729] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.010536 24492 consensus_queue.cc:260] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:01.971002 24187 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 80026.4us
I20260924 14:28:02.014966 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5b55145a82545d9b769ea9d04bf1d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.035430 24359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317bae4ebac443c8a2fada29c389ab79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.045511 24440 tablet_service.cc:1467] Processing CreateTablet for tablet 8fefeab5a75749a198aa72b89e7e2a53 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260924 14:28:02.046886 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fefeab5a75749a198aa72b89e7e2a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.064328 24366 tablet_service.cc:1467] Processing CreateTablet for tablet d5b55145a82545d9b769ea9d04bf1d99 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260924 14:28:02.064432 24435 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca317d21c5f40f69bc0c4ad009f6e94 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260924 14:28:02.065850 24435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca317d21c5f40f69bc0c4ad009f6e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.070205 24367 tablet_service.cc:1467] Processing CreateTablet for tablet 8fefeab5a75749a198aa72b89e7e2a53 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260924 14:28:02.073020 24365 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca317d21c5f40f69bc0c4ad009f6e94 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260924 14:28:02.075016 24440 tablet_service.cc:1467] Processing CreateTablet for tablet b18bcddd405d4bbd9d39081d45429923 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260924 14:28:02.077579 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b18bcddd405d4bbd9d39081d45429923. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:02.083840 24188 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 54980.4us
I20260924 14:28:02.085680 24362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7959d21c60de4745bebfd367d2367714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.087991 24291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df40ee44e21496dbc7ef5dd5e624432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.097636 24435 tablet_service.cc:1467] Processing CreateTablet for tablet bb1ed21b7eb4447b93b9cd4b579d4031 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260924 14:28:02.098996 24435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1ed21b7eb4447b93b9cd4b579d4031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.100165 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4aa8a7302f42b688f37d4fdb892b67" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:02.102244 24417 leader_election.cc:341] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.107100 24285 tablet_service.cc:1467] Processing CreateTablet for tablet 317bae4ebac443c8a2fada29c389ab79 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260924 14:28:02.109042 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca317d21c5f40f69bc0c4ad009f6e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.113992 24367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fefeab5a75749a198aa72b89e7e2a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.116313 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:02.118059 24268 leader_election.cc:341] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.121708 24494 tablet_bootstrap.cc:654] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.123589 24435 tablet_service.cc:1467] Processing CreateTablet for tablet 5caac6840188451a8e325a37b8ea0e3c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260924 14:28:02.124890 24435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5caac6840188451a8e325a37b8ea0e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.127321 24440 tablet_service.cc:1467] Processing CreateTablet for tablet 252120ce7355481189a4a8c3659997ab (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260924 14:28:02.127707 24434 tablet_service.cc:1467] Processing CreateTablet for tablet d5b55145a82545d9b769ea9d04bf1d99 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260924 14:28:02.128259 24493 tablet_bootstrap.cc:492] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:02.128645 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252120ce7355481189a4a8c3659997ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.128703 24493 ts_tablet_manager.cc:1397] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.391s	user 0.034s	sys 0.068s
I20260924 14:28:02.131757 24366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5b55145a82545d9b769ea9d04bf1d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.131763 24493 raft_consensus.cc:348] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.132803 24493 raft_consensus.cc:374] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.133150 24493 raft_consensus.cc:729] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.133957 24493 consensus_queue.cc:260] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.136274 24286 tablet_service.cc:1467] Processing CreateTablet for tablet 7959d21c60de4745bebfd367d2367714 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260924 14:28:02.138052 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4aa8a7302f42b688f37d4fdb892b67" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.141075 24493 ts_tablet_manager.cc:1428] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260924 14:28:02.142042 24493 tablet_bootstrap.cc:492] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
W20260924 14:28:02.142460 24418 leader_election.cc:341] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.142920 24418 leader_election.cc:302] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:02.143867 24496 raft_consensus.cc:2726] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:02.148296 24492 ts_tablet_manager.cc:1428] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.347s	user 0.004s	sys 0.015s
I20260924 14:28:02.149271 24492 tablet_bootstrap.cc:492] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:02.156519 24363 tablet_service.cc:1467] Processing CreateTablet for tablet bb1ed21b7eb4447b93b9cd4b579d4031 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260924 14:28:02.157924 24363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1ed21b7eb4447b93b9cd4b579d4031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.139818 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93b3f9be3c4846a8b75e5e42099e6dbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.160076 24286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7959d21c60de4745bebfd367d2367714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.163770 24285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317bae4ebac443c8a2fada29c389ab79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.175714 24434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5b55145a82545d9b769ea9d04bf1d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.177170 24365 tablet_service.cc:1467] Processing CreateTablet for tablet 5caac6840188451a8e325a37b8ea0e3c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260924 14:28:02.180612 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5caac6840188451a8e325a37b8ea0e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.180409 24361 tablet_service.cc:1467] Processing CreateTablet for tablet b18bcddd405d4bbd9d39081d45429923 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260924 14:28:02.179312 24367 tablet_service.cc:1467] Processing CreateTablet for tablet 252120ce7355481189a4a8c3659997ab (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260924 14:28:02.181722 24361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b18bcddd405d4bbd9d39081d45429923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.182529 24367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252120ce7355481189a4a8c3659997ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.182735 24493 tablet_bootstrap.cc:654] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.158241 24435 tablet_service.cc:1467] Processing CreateTablet for tablet 870bbbde2a2f4c76bb558d8c43d7587b (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260924 14:28:02.185262 24435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 870bbbde2a2f4c76bb558d8c43d7587b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.186750 24368 tablet_service.cc:1467] Processing CreateTablet for tablet 0e065e0c774647b1a9b127accaebd70a (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260924 14:28:02.198506 24438 tablet_service.cc:1467] Processing CreateTablet for tablet 0e065e0c774647b1a9b127accaebd70a (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260924 14:28:02.199821 24438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e065e0c774647b1a9b127accaebd70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.202210 24500 raft_consensus.cc:482] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.202692 24500 raft_consensus.cc:504] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.204806 24500 leader_election.cc:288] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:02.205144 24292 tablet_service.cc:1467] Processing CreateTablet for tablet 8fefeab5a75749a198aa72b89e7e2a53 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260924 14:28:02.206379 24292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fefeab5a75749a198aa72b89e7e2a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.207006 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:02.207645 24378 raft_consensus.cc:2454] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:02.209075 24268 leader_election.cc:302] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:02.209260 24360 tablet_service.cc:1467] Processing CreateTablet for tablet 870bbbde2a2f4c76bb558d8c43d7587b (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260924 14:28:02.210318 24502 raft_consensus.cc:2781] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:02.210484 24360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 870bbbde2a2f4c76bb558d8c43d7587b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.210709 24502 raft_consensus.cc:482] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:02.211045 24502 raft_consensus.cc:3037] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:02.202759 24439 tablet_service.cc:1467] Processing CreateTablet for tablet 638ffc5970704a53a26967c81af23e52 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260924 14:28:02.218673 24502 raft_consensus.cc:504] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:02.220448 24287 tablet_service.cc:1467] Processing CreateTablet for tablet bb1ed21b7eb4447b93b9cd4b579d4031 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260924 14:28:02.221685 24287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1ed21b7eb4447b93b9cd4b579d4031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.224102 24286 tablet_service.cc:1467] Processing CreateTablet for tablet 252120ce7355481189a4a8c3659997ab (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260924 14:28:02.225322 24286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252120ce7355481189a4a8c3659997ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.243369 24364 tablet_service.cc:1467] Processing CreateTablet for tablet 638ffc5970704a53a26967c81af23e52 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260924 14:28:02.220450 24285 tablet_service.cc:1467] Processing CreateTablet for tablet b18bcddd405d4bbd9d39081d45429923 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260924 14:28:02.244426 24492 tablet_bootstrap.cc:654] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.244661 24285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b18bcddd405d4bbd9d39081d45429923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.247649 24439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638ffc5970704a53a26967c81af23e52. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:02.197023 24187 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 77587.9us
I20260924 14:28:02.252374 24291 tablet_service.cc:1467] Processing CreateTablet for tablet 0e065e0c774647b1a9b127accaebd70a (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260924 14:28:02.253633 24291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e065e0c774647b1a9b127accaebd70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.256947 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce753b9bac814f17a869f4f8d66deca6" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:02.256204 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:02.259372 24343 leader_election.cc:341] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:02.266219 24343 leader_election.cc:341] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.266340 24436 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48732 (request call id 33) took 1025 ms. Trace:
I20260924 14:28:02.266736 24436 rpcz_store.cc:276] 0924 14:28:01.241271 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:01.868091 (+626820us) service_pool.cc:227] Handling call
0924 14:28:01.869969 (+  1878us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:01.870066 (+    97us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.049462 (+179396us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.265835 (+216373us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.266287 (+   452us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31762}
I20260924 14:28:02.269800 24436 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf23ec7c761435d8352db2e38996e74 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260924 14:28:02.219178 24284 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca317d21c5f40f69bc0c4ad009f6e94 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260924 14:28:02.266237 24442 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48732 (request call id 32) took 1025 ms. Trace:
I20260924 14:28:02.239253 24293 tablet_service.cc:1467] Processing CreateTablet for tablet 5caac6840188451a8e325a37b8ea0e3c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260924 14:28:02.282567 24368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e065e0c774647b1a9b127accaebd70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.292680 24436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf23ec7c761435d8352db2e38996e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.291714 24290 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49158 (request call id 29) took 1033 ms. Trace:
I20260924 14:28:02.294327 24290 rpcz_store.cc:276] 0924 14:28:01.257653 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:01.779015 (+521362us) service_pool.cc:227] Handling call
0924 14:28:01.780488 (+  1473us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:01.780559 (+    71us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.124178 (+343619us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.278169 (+153991us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.290931 (+ 12762us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:02.294934 24359 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49698 (request call id 35) took 1024 ms. Trace:
I20260924 14:28:02.285274 24442 rpcz_store.cc:276] 0924 14:28:01.240986 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:01.705781 (+464795us) service_pool.cc:227] Handling call
0924 14:28:01.716278 (+ 10497us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:01.728020 (+ 11742us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.160414 (+432394us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.243562 (+ 83148us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.266167 (+ 22605us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22011}
I20260924 14:28:02.299949 24496 raft_consensus.cc:482] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.299894 24293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5caac6840188451a8e325a37b8ea0e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.296070 24440 tablet_service.cc:1467] Processing CreateTablet for tablet 3822638d266742c4908907a61efe8231 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260924 14:28:02.300460 24496 raft_consensus.cc:504] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.304277 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bec046486fdc464290fafd6e4978632e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.304668 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bec046486fdc464290fafd6e4978632e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:02.305759 24418 leader_election.cc:341] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:02.306295 24417 leader_election.cc:341] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.306730 24417 leader_election.cc:302] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:02.301247 24359 rpcz_store.cc:276] 0924 14:28:01.270348 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:01.886069 (+615721us) service_pool.cc:227] Handling call
0924 14:28:01.956462 (+ 70393us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:01.965422 (+  8960us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.220140 (+254718us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.294475 (+ 74335us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.294877 (+   402us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:02.311036 24361 tablet_service.cc:1467] Processing CreateTablet for tablet 3822638d266742c4908907a61efe8231 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260924 14:28:02.303496 24496 leader_election.cc:288] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:02.311975 24496 raft_consensus.cc:2726] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:02.312543 24442 tablet_service.cc:1467] Processing CreateTablet for tablet f3667aba281046d3b974da9065022529 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260924 14:28:02.313149 24359 tablet_service.cc:1467] Processing CreateTablet for tablet f3667aba281046d3b974da9065022529 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260924 14:28:02.313910 24442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3667aba281046d3b974da9065022529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.316983 24435 tablet_service.cc:1467] Processing CreateTablet for tablet 3a57dc14600c450485f596239b30df0d (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260924 14:28:02.318238 24435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a57dc14600c450485f596239b30df0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.320634 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3822638d266742c4908907a61efe8231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.321292 24436 tablet_service.cc:1467] Processing CreateTablet for tablet 234eb8cdbf4c4e7599789194a0032c00 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260924 14:28:02.322530 24436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 234eb8cdbf4c4e7599789194a0032c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.322916 24493 tablet_bootstrap.cc:492] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:02.323383 24493 ts_tablet_manager.cc:1397] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.182s	user 0.016s	sys 0.003s
I20260924 14:28:02.299248 24365 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf23ec7c761435d8352db2e38996e74 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260924 14:28:02.326148 24493 raft_consensus.cc:348] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.327050 24492 tablet_bootstrap.cc:492] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:02.327921 24441 tablet_service.cc:1467] Processing CreateTablet for tablet e22099296aa6419a924560e9fbe81cfd (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260924 14:28:02.315156 24290 tablet_service.cc:1467] Processing CreateTablet for tablet 870bbbde2a2f4c76bb558d8c43d7587b (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260924 14:28:02.363672 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 870bbbde2a2f4c76bb558d8c43d7587b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.341756 24364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638ffc5970704a53a26967c81af23e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.367848 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca317d21c5f40f69bc0c4ad009f6e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.368475 24361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3822638d266742c4908907a61efe8231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.369146 24494 tablet_bootstrap.cc:492] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:02.369652 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf23ec7c761435d8352db2e38996e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.361402 24493 raft_consensus.cc:374] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.371857 24493 raft_consensus.cc:729] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.348893 24437 tablet_service.cc:1467] Processing CreateTablet for tablet fcf594a126774d39a57f354d4fc7d338 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260924 14:28:02.372125 24441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e22099296aa6419a924560e9fbe81cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.362726 24492 ts_tablet_manager.cc:1397] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.214s	user 0.004s	sys 0.018s
I20260924 14:28:02.372674 24493 consensus_queue.cc:260] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.369626 24494 ts_tablet_manager.cc:1397] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.450s	user 0.019s	sys 0.017s
I20260924 14:28:02.374993 24492 raft_consensus.cc:348] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.375960 24492 raft_consensus.cc:374] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.376281 24492 raft_consensus.cc:729] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.377022 24492 consensus_queue.cc:260] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.379307 24492 ts_tablet_manager.cc:1428] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260924 14:28:02.391052 24285 tablet_service.cc:1467] Processing CreateTablet for tablet 638ffc5970704a53a26967c81af23e52 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260924 14:28:02.392570 24285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638ffc5970704a53a26967c81af23e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.392716 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:02.393318 24434 tablet_service.cc:1467] Processing CreateTablet for tablet e838ce343bad4d42aa930f68fe4a4b2c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260924 14:28:02.394470 24417 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 88378.7us
I20260924 14:28:02.395258 24492 tablet_bootstrap.cc:492] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:02.396711 24493 ts_tablet_manager.cc:1428] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.073s	user 0.004s	sys 0.002s
I20260924 14:28:02.400089 24288 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49158 (request call id 36) took 1003 ms. Trace:
I20260924 14:28:02.415827 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.416080 24288 rpcz_store.cc:276] 0924 14:28:01.396422 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:02.005515 (+609093us) service_pool.cc:227] Handling call
0924 14:28:02.007327 (+  1812us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:02.007432 (+   105us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.238206 (+230774us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.399562 (+161356us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.400039 (+   477us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9328}
I20260924 14:28:02.416663 24303 raft_consensus.cc:2379] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61d316eb5dfe405c95019383bbebecaa in current term 1: Already voted for candidate cafc2c5ad4df4224a5ccc4893de70144 in this term.
I20260924 14:28:02.416731 24493 tablet_bootstrap.cc:492] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:02.417649 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce753b9bac814f17a869f4f8d66deca6" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.418023 24288 tablet_service.cc:1467] Processing CreateTablet for tablet 3822638d266742c4908907a61efe8231 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260924 14:28:02.418859 24502 leader_election.cc:288] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:02.403976 24359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3667aba281046d3b974da9065022529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.419701 24344 leader_election.cc:302] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:02.420991 24362 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49698 (request call id 37) took 1070 ms. Trace:
I20260924 14:28:02.421639 24500 raft_consensus.cc:3037] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:02.421330 24362 rpcz_store.cc:276] 0924 14:28:01.350865 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:01.949362 (+598497us) service_pool.cc:227] Handling call
0924 14:28:01.973891 (+ 24529us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:02.029058 (+ 55167us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.211848 (+182790us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.420502 (+208654us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.420931 (+   429us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:02.419418 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3822638d266742c4908907a61efe8231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.427727 24437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf594a126774d39a57f354d4fc7d338. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:02.421402 24344 leader_election.cc:341] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.429589 24344 leader_election.cc:302] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:02.429690 24438 tablet_service.cc:1467] Processing CreateTablet for tablet 568b498fb63e4638b92d175b8d122c0e (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260924 14:28:02.407577 24366 tablet_service.cc:1467] Processing CreateTablet for tablet 3a57dc14600c450485f596239b30df0d (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260924 14:28:02.410559 24287 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf23ec7c761435d8352db2e38996e74 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260924 14:28:02.413236 24368 tablet_service.cc:1467] Processing CreateTablet for tablet 234eb8cdbf4c4e7599789194a0032c00 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
W20260924 14:28:02.414458 24268 leader_election.cc:341] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.415551 24433 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48732 (request call id 28) took 1200 ms. Trace:
I20260924 14:28:02.376255 24494 raft_consensus.cc:348] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.447947 24494 raft_consensus.cc:374] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.448271 24494 raft_consensus.cc:729] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
W20260924 14:28:02.449347 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 55163.3us
I20260924 14:28:02.449067 24494 consensus_queue.cc:260] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.447671 24433 rpcz_store.cc:276] 0924 14:28:01.214658 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:01.595594 (+380936us) service_pool.cc:227] Handling call
0924 14:28:01.681841 (+ 86247us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:01.700148 (+ 18307us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.156506 (+456358us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.413936 (+257430us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.415480 (+  1544us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:02.423329 24362 tablet_service.cc:1467] Processing CreateTablet for tablet fcf594a126774d39a57f354d4fc7d338 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260924 14:28:02.451969 24433 tablet_service.cc:1467] Processing CreateTablet for tablet b9e83b406c1944efb04163f032a3b4f6 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260924 14:28:02.453405 24367 tablet_service.cc:1467] Processing CreateTablet for tablet 568b498fb63e4638b92d175b8d122c0e (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260924 14:28:02.441754 24248 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:02.442373 24363 tablet_service.cc:1467] Processing CreateTablet for tablet e22099296aa6419a924560e9fbe81cfd (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260924 14:28:02.443789 24366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a57dc14600c450485f596239b30df0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.463728 24368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 234eb8cdbf4c4e7599789194a0032c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.444661 24287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf23ec7c761435d8352db2e38996e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.464475 24362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf594a126774d39a57f354d4fc7d338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.467742 24363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e22099296aa6419a924560e9fbe81cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.456267 24493 tablet_bootstrap.cc:654] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.440323 24500 raft_consensus.cc:2726] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:02.459679 24434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e838ce343bad4d42aa930f68fe4a4b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.484414 24500 raft_consensus.cc:2726] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:02.463865 24289 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49158 (request call id 34) took 1090 ms. Trace:
W20260924 14:28:02.471856 24344 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 50482.9us
I20260924 14:28:02.476022 24367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 568b498fb63e4638b92d175b8d122c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.480489 24442 tablet_service.cc:1467] Processing CreateTablet for tablet c2e4953bd67e40f5823697e67d029f52 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260924 14:28:02.488333 24433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9e83b406c1944efb04163f032a3b4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.496802 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:02.500054 24442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e4953bd67e40f5823697e67d029f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.491448 24289 rpcz_store.cc:276] 0924 14:28:01.374709 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:01.900595 (+525886us) service_pool.cc:227] Handling call
0924 14:28:01.902336 (+  1741us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:01.902444 (+   108us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.387960 (+485516us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.463337 (+ 75377us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.463806 (+   469us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:02.499787 24291 tablet_service.cc:1467] Processing CreateTablet for tablet 3a57dc14600c450485f596239b30df0d (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260924 14:28:02.502593 24291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a57dc14600c450485f596239b30df0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.507671 24292 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49158 (request call id 39) took 1056 ms. Trace:
I20260924 14:28:02.508035 24292 rpcz_store.cc:276] 0924 14:28:01.451609 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:02.203388 (+751779us) service_pool.cc:227] Handling call
0924 14:28:02.205918 (+  2530us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:02.206023 (+   105us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.387913 (+181890us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.499118 (+111205us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.507614 (+  8496us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:02.503862 24289 tablet_service.cc:1467] Processing CreateTablet for tablet 234eb8cdbf4c4e7599789194a0032c00 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260924 14:28:02.509927 24292 tablet_service.cc:1467] Processing CreateTablet for tablet e22099296aa6419a924560e9fbe81cfd (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260924 14:28:02.510939 24284 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49158 (request call id 40) took 1058 ms. Trace:
I20260924 14:28:02.514106 24284 rpcz_store.cc:276] 0924 14:28:01.452320 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:02.218290 (+765970us) service_pool.cc:227] Handling call
0924 14:28:02.327673 (+109383us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b5800001898. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eee82 00007f09bf3078b9 00007f09be21a5d8 00007f09be2da2a0 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2 00007f09b788db01 00007f09b788da7c
0924 14:28:02.327816 (+   143us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:02.362121 (+ 34305us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.472238 (+110117us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.508487 (+ 36249us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.510877 (+  2390us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3390208}
I20260924 14:28:02.511690 24438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 568b498fb63e4638b92d175b8d122c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.516955 24284 tablet_service.cc:1467] Processing CreateTablet for tablet fcf594a126774d39a57f354d4fc7d338 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260924 14:28:02.513837 24492 tablet_bootstrap.cc:654] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.511873 24292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e22099296aa6419a924560e9fbe81cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.518556 24285 tablet_service.cc:1467] Processing CreateTablet for tablet 568b498fb63e4638b92d175b8d122c0e (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260924 14:28:02.511803 24494 ts_tablet_manager.cc:1428] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.138s	user 0.006s	sys 0.001s
I20260924 14:28:02.546892 24494 tablet_bootstrap.cc:492] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:02.533473 24293 tablet_service.cc:1467] Processing CreateTablet for tablet b9e83b406c1944efb04163f032a3b4f6 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260924 14:28:02.533840 24290 tablet_service.cc:1467] Processing CreateTablet for tablet e838ce343bad4d42aa930f68fe4a4b2c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260924 14:28:02.484975 24286 tablet_service.cc:1467] Processing CreateTablet for tablet f3667aba281046d3b974da9065022529 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260924 14:28:02.535944 24360 tablet_service.cc:1467] Processing CreateTablet for tablet b9e83b406c1944efb04163f032a3b4f6 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260924 14:28:02.534102 24366 tablet_service.cc:1467] Processing CreateTablet for tablet e838ce343bad4d42aa930f68fe4a4b2c (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260924 14:28:02.538321 24378 raft_consensus.cc:2454] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:02.552539 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf594a126774d39a57f354d4fc7d338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.556089 24366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e838ce343bad4d42aa930f68fe4a4b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.563150 24365 tablet_service.cc:1467] Processing CreateTablet for tablet c2e4953bd67e40f5823697e67d029f52 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260924 14:28:02.564144 24289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 234eb8cdbf4c4e7599789194a0032c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.564467 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e4953bd67e40f5823697e67d029f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.577296 24494 tablet_bootstrap.cc:654] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.578747 24519 raft_consensus.cc:482] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.578982 24268 leader_election.cc:302] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:02.579234 24519 raft_consensus.cc:504] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.580186 24502 raft_consensus.cc:2781] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:02.581282 24519 leader_election.cc:288] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:02.583046 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:02.583377 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.583830 24285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 568b498fb63e4638b92d175b8d122c0e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:02.584491 24343 leader_election.cc:341] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:02.585067 24344 leader_election.cc:341] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.585479 24344 leader_election.cc:302] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:02.586524 24519 raft_consensus.cc:2726] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:02.592499 24493 tablet_bootstrap.cc:492] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:02.593163 24493 ts_tablet_manager.cc:1397] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.177s	user 0.012s	sys 0.003s
I20260924 14:28:02.597440 24360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9e83b406c1944efb04163f032a3b4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.613623 24502 raft_consensus.cc:686] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:02.617274 24518 raft_consensus.cc:482] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.620734 24502 consensus_queue.cc:237] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:02.622462 24518 raft_consensus.cc:504] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.622814 24494 tablet_bootstrap.cc:492] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:02.624492 24518 leader_election.cc:288] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:02.625432 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9c52e97d0444b8ec17719ea5a4ac6" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.625550 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9c52e97d0444b8ec17719ea5a4ac6" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:02.626083 24302 raft_consensus.cc:2454] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
W20260924 14:28:02.627063 24417 leader_election.cc:341] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.629931 24418 leader_election.cc:302] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:02.630914 24518 raft_consensus.cc:2781] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:02.631294 24518 raft_consensus.cc:482] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:02.631624 24494 ts_tablet_manager.cc:1397] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.085s	user 0.022s	sys 0.011s
I20260924 14:28:02.634310 24292 tablet_service.cc:1467] Processing CreateTablet for tablet c2e4953bd67e40f5823697e67d029f52 (DEFAULT_TABLE table=test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260924 14:28:02.634938 24518 raft_consensus.cc:3037] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:02.635568 24292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e4953bd67e40f5823697e67d029f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.640864 24286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3667aba281046d3b974da9065022529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.641328 24494 raft_consensus.cc:348] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.642385 24494 raft_consensus.cc:374] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.642746 24494 raft_consensus.cc:729] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.644423 24293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9e83b406c1944efb04163f032a3b4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.643882 24494 consensus_queue.cc:260] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.646257 24492 tablet_bootstrap.cc:492] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:02.646719 24492 ts_tablet_manager.cc:1397] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.252s	user 0.013s	sys 0.007s
I20260924 14:28:02.650413 24492 raft_consensus.cc:348] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.651166 24492 raft_consensus.cc:374] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.660013 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e838ce343bad4d42aa930f68fe4a4b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:02.657961 24492 raft_consensus.cc:729] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.664876 24492 consensus_queue.cc:260] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.668417 24492 ts_tablet_manager.cc:1428] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260924 14:28:02.669657 24492 tablet_bootstrap.cc:492] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:02.672313 24493 raft_consensus.cc:348] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.673486 24493 raft_consensus.cc:374] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.673888 24493 raft_consensus.cc:729] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.674896 24493 consensus_queue.cc:260] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.682127 24522 raft_consensus.cc:482] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.682796 24522 raft_consensus.cc:504] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.694977 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:02.696802 24268 leader_election.cc:341] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.696877 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:02.699486 24494 ts_tablet_manager.cc:1428] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.005s
I20260924 14:28:02.701411 24494 tablet_bootstrap.cc:492] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
W20260924 14:28:02.701525 24268 leader_election.cc:341] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.702008 24268 leader_election.cc:302] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:02.701772 24522 leader_election.cc:288] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:02.703027 24522 raft_consensus.cc:2726] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:02.699968 24200 catalog_manager.cc:5495] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:02.709461 24286 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49158 (request call id 50) took 1031 ms. Trace:
I20260924 14:28:02.709883 24286 rpcz_store.cc:276] 0924 14:28:01.677618 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:02.484074 (+806456us) service_pool.cc:227] Handling call
0924 14:28:02.639845 (+155771us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:02.640527 (+   682us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:02.657910 (+ 17383us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:02.708953 (+ 51043us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:02.709403 (+   450us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:02.699314 24493 ts_tablet_manager.cc:1428] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.106s	user 0.007s	sys 0.000s
I20260924 14:28:02.712617 24493 tablet_bootstrap.cc:492] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:02.714426 24494 tablet_bootstrap.cc:654] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.727098 24493 tablet_bootstrap.cc:654] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.736716 24492 tablet_bootstrap.cc:654] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.728286 24518 raft_consensus.cc:504] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.740619 24494 tablet_bootstrap.cc:492] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:02.741194 24494 ts_tablet_manager.cc:1397] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.000s
I20260924 14:28:02.743028 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9c52e97d0444b8ec17719ea5a4ac6" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:02.743677 24302 raft_consensus.cc:3037] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:02.745741 24494 raft_consensus.cc:348] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.746466 24494 raft_consensus.cc:374] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.746740 24494 raft_consensus.cc:729] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.747406 24494 consensus_queue.cc:260] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.749881 24494 ts_tablet_manager.cc:1428] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260924 14:28:02.750819 24494 tablet_bootstrap.cc:492] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:02.753399 24302 raft_consensus.cc:2454] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:02.756035 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9c52e97d0444b8ec17719ea5a4ac6" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:02.758350 24494 tablet_bootstrap.cc:654] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:02.758409 24417 leader_election.cc:341] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.756691 24518 leader_election.cc:288] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:02.760558 24418 leader_election.cc:302] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:02.762135 24518 raft_consensus.cc:2781] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:02.763900 24518 raft_consensus.cc:686] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:02.765091 24518 consensus_queue.cc:237] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.770233 24494 tablet_bootstrap.cc:492] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:02.770747 24494 ts_tablet_manager.cc:1397] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260924 14:28:02.773226 24494 raft_consensus.cc:348] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.773770 24494 raft_consensus.cc:374] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.774012 24494 raft_consensus.cc:729] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.774575 24494 consensus_queue.cc:260] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.777379 24494 ts_tablet_manager.cc:1428] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260924 14:28:02.778419 24494 tablet_bootstrap.cc:492] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:02.779495 24492 tablet_bootstrap.cc:492] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:02.779963 24492 ts_tablet_manager.cc:1397] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.111s	user 0.019s	sys 0.025s
I20260924 14:28:02.782282 24493 tablet_bootstrap.cc:492] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:02.782738 24493 ts_tablet_manager.cc:1397] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.070s	user 0.005s	sys 0.008s
I20260924 14:28:02.782686 24492 raft_consensus.cc:348] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.783577 24492 raft_consensus.cc:374] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.783859 24492 raft_consensus.cc:729] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.784475 24494 tablet_bootstrap.cc:654] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.785569 24493 raft_consensus.cc:348] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.786288 24493 raft_consensus.cc:374] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.786551 24493 raft_consensus.cc:729] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.787330 24493 consensus_queue.cc:260] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.787834 24200 catalog_manager.cc:5495] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:02.789748 24493 ts_tablet_manager.cc:1428] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.008s
I20260924 14:28:02.790735 24493 tablet_bootstrap.cc:492] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:02.784649 24492 consensus_queue.cc:260] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.793408 24492 ts_tablet_manager.cc:1428] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260924 14:28:02.795492 24492 tablet_bootstrap.cc:492] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:02.797008 24493 tablet_bootstrap.cc:654] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.802168 24494 tablet_bootstrap.cc:492] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:02.803099 24492 tablet_bootstrap.cc:654] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.803393 24494 ts_tablet_manager.cc:1397] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.004s
I20260924 14:28:02.806237 24494 raft_consensus.cc:348] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.807019 24494 raft_consensus.cc:374] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.807341 24494 raft_consensus.cc:729] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.808108 24494 consensus_queue.cc:260] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.813470 24494 ts_tablet_manager.cc:1428] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260924 14:28:02.814461 24494 tablet_bootstrap.cc:492] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:02.820322 24494 tablet_bootstrap.cc:654] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.823945 24493 tablet_bootstrap.cc:492] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:02.825296 24492 tablet_bootstrap.cc:492] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:02.825692 24493 ts_tablet_manager.cc:1397] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.015s
I20260924 14:28:02.825884 24492 ts_tablet_manager.cc:1397] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.002s
I20260924 14:28:02.827867 24494 tablet_bootstrap.cc:492] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:02.828320 24494 ts_tablet_manager.cc:1397] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.003s
I20260924 14:28:02.828059 24493 raft_consensus.cc:348] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.828923 24493 raft_consensus.cc:374] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.828744 24492 raft_consensus.cc:348] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.829228 24493 raft_consensus.cc:729] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.829569 24492 raft_consensus.cc:374] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.829874 24492 raft_consensus.cc:729] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.829983 24493 consensus_queue.cc:260] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.830588 24492 consensus_queue.cc:260] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.830955 24494 raft_consensus.cc:348] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.831673 24494 raft_consensus.cc:374] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.831952 24494 raft_consensus.cc:729] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.832212 24493 ts_tablet_manager.cc:1428] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260924 14:28:02.832973 24492 ts_tablet_manager.cc:1428] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:28:02.833209 24493 tablet_bootstrap.cc:492] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:02.832711 24494 consensus_queue.cc:260] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.833875 24492 tablet_bootstrap.cc:492] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:02.838025 24494 ts_tablet_manager.cc:1428] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260924 14:28:02.839174 24494 tablet_bootstrap.cc:492] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:02.839721 24493 tablet_bootstrap.cc:654] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.839816 24492 tablet_bootstrap.cc:654] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.850113 24494 tablet_bootstrap.cc:654] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.851981 24492 tablet_bootstrap.cc:492] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:02.852180 24493 tablet_bootstrap.cc:492] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:02.852453 24492 ts_tablet_manager.cc:1397] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260924 14:28:02.852636 24493 ts_tablet_manager.cc:1397] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.011s
I20260924 14:28:02.856060 24492 raft_consensus.cc:348] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.857100 24492 raft_consensus.cc:374] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.857463 24492 raft_consensus.cc:729] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.858371 24492 consensus_queue.cc:260] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.856081 24493 raft_consensus.cc:348] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.860412 24493 raft_consensus.cc:374] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.860770 24493 raft_consensus.cc:729] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.861763 24493 consensus_queue.cc:260] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.864928 24493 ts_tablet_manager.cc:1428] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260924 14:28:02.865192 24492 ts_tablet_manager.cc:1428] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260924 14:28:02.866087 24493 tablet_bootstrap.cc:492] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:02.868256 24494 tablet_bootstrap.cc:492] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:02.868760 24494 ts_tablet_manager.cc:1397] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.030s	user 0.023s	sys 0.000s
I20260924 14:28:02.872532 24493 tablet_bootstrap.cc:654] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.871368 24494 raft_consensus.cc:348] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.877888 24494 raft_consensus.cc:374] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.878214 24494 raft_consensus.cc:729] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.878360 24492 tablet_bootstrap.cc:492] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:02.878839 24494 consensus_queue.cc:260] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.881134 24494 ts_tablet_manager.cc:1428] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260924 14:28:02.882040 24494 tablet_bootstrap.cc:492] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:02.882558 24493 tablet_bootstrap.cc:492] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:02.882982 24493 ts_tablet_manager.cc:1397] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.004s
I20260924 14:28:02.884207 24518 raft_consensus.cc:482] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.884469 24492 tablet_bootstrap.cc:654] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.884688 24518 raft_consensus.cc:504] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.886773 24518 leader_election.cc:288] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:02.886976 24494 tablet_bootstrap.cc:654] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.888676 24525 raft_consensus.cc:482] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.889251 24525 raft_consensus.cc:504] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.892486 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86057fee00d44a4c9607d990f2782813" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:02.893069 24493 raft_consensus.cc:348] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.894290 24493 raft_consensus.cc:374] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.894631 24493 raft_consensus.cc:729] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.895421 24493 consensus_queue.cc:260] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.896643 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86057fee00d44a4c9607d990f2782813" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.898281 24494 tablet_bootstrap.cc:492] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:02.898813 24494 ts_tablet_manager.cc:1397] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.005s
I20260924 14:28:02.901212 24494 raft_consensus.cc:348] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.901836 24494 raft_consensus.cc:374] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.902105 24494 raft_consensus.cc:729] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.901592 24493 ts_tablet_manager.cc:1428] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260924 14:28:02.903492 24519 raft_consensus.cc:482] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.903687 24494 consensus_queue.cc:260] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.903353 24493 tablet_bootstrap.cc:492] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:02.904075 24519 raft_consensus.cc:504] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.906904 24494 ts_tablet_manager.cc:1428] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260924 14:28:02.907780 24494 tablet_bootstrap.cc:492] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:02.908384 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bec046486fdc464290fafd6e4978632e" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:02.909281 24452 raft_consensus.cc:2454] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:02.910290 24519 leader_election.cc:288] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:02.911782 24417 leader_election.cc:341] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.912571 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bec046486fdc464290fafd6e4978632e" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.913319 24302 raft_consensus.cc:2454] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:02.913552 24494 tablet_bootstrap.cc:654] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.912585 24343 leader_election.cc:302] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
W20260924 14:28:02.915872 24418 leader_election.cc:341] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.916452 24418 leader_election.cc:302] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:02.917819 24496 raft_consensus.cc:482] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.918371 24496 raft_consensus.cc:504] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.919476 24519 raft_consensus.cc:2781] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:02.919962 24519 raft_consensus.cc:482] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:02.920265 24519 raft_consensus.cc:3037] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:02.920830 24493 tablet_bootstrap.cc:654] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.924552 24496 leader_election.cc:288] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:02.925284 24496 raft_consensus.cc:2726] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:02.927841 24500 raft_consensus.cc:482] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.928624 24500 raft_consensus.cc:504] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.929207 24519 raft_consensus.cc:504] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.933070 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b53acab36f9a42838efe3526f4aed03a" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.933238 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bec046486fdc464290fafd6e4978632e" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:02.934161 24303 raft_consensus.cc:3037] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:02.934851 24344 leader_election.cc:341] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.935956 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bec046486fdc464290fafd6e4978632e" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:02.936691 24452 raft_consensus.cc:3037] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:02.937240 24500 leader_election.cc:288] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:02.939213 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b53acab36f9a42838efe3526f4aed03a" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:02.940675 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:02.941570 24378 raft_consensus.cc:2454] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:02.943785 24417 leader_election.cc:302] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:02.945072 24519 leader_election.cc:288] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:02.945447 24343 leader_election.cc:341] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:02.946043 24343 leader_election.cc:302] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:02.950096 24452 raft_consensus.cc:2454] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:02.951686 24343 leader_election.cc:302] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:02.953917 24494 tablet_bootstrap.cc:492] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:02.954516 24494 ts_tablet_manager.cc:1397] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.047s	user 0.011s	sys 0.009s
I20260924 14:28:02.956573 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.957429 24302 raft_consensus.cc:2454] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:02.957722 24494 raft_consensus.cc:348] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.958734 24494 raft_consensus.cc:374] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:02.959060 24494 raft_consensus.cc:729] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:02.959888 24525 leader_election.cc:288] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:02.959908 24494 consensus_queue.cc:260] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.962949 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:02.962800 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:02.963724 24378 raft_consensus.cc:2454] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:02.963824 24452 raft_consensus.cc:2454] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:02.964861 24268 leader_election.cc:302] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:02.965801 24525 raft_consensus.cc:2781] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:02.966231 24525 raft_consensus.cc:482] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:02.966662 24525 raft_consensus.cc:3037] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:02.974825 24500 raft_consensus.cc:2726] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:02.975971 24494 ts_tablet_manager.cc:1428] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.002s
I20260924 14:28:02.976233 24518 raft_consensus.cc:2781] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:02.976773 24518 raft_consensus.cc:482] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:02.977170 24518 raft_consensus.cc:3037] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:02.978404 24525 raft_consensus.cc:504] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.979924 24496 raft_consensus.cc:482] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:02.980599 24496 raft_consensus.cc:504] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.982055 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:02.977190 24494 tablet_bootstrap.cc:492] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:02.985330 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3550e2841c1f41939833b5588fb35875" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:02.986184 24302 raft_consensus.cc:2454] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:02.988140 24418 leader_election.cc:302] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:02.988828 24494 tablet_bootstrap.cc:654] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:02.989333 24500 raft_consensus.cc:2781] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:02.989883 24500 raft_consensus.cc:686] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:02.990085 24547 raft_consensus.cc:2781] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:02.990428 24547 raft_consensus.cc:482] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:02.990728 24547 raft_consensus.cc:3037] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:02.990978 24500 consensus_queue.cc:237] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:02.991598 24525 leader_election.cc:288] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:02.993250 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:02.994165 24378 raft_consensus.cc:3037] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.000761 24547 raft_consensus.cc:504] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.003700 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3550e2841c1f41939833b5588fb35875" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:03.004299 24302 raft_consensus.cc:3037] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.005548 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3550e2841c1f41939833b5588fb35875" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:03.006736 24496 leader_election.cc:288] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
W20260924 14:28:03.008623 24417 leader_election.cc:341] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.016273 24303 raft_consensus.cc:2454] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:03.017375 24496 raft_consensus.cc:482] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.023167 24496 raft_consensus.cc:504] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.021828 24494 tablet_bootstrap.cc:492] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:03.019191 24302 raft_consensus.cc:2454] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:03.044226 24418 leader_election.cc:302] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:03.045482 24494 ts_tablet_manager.cc:1397] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.069s	user 0.017s	sys 0.012s
I20260924 14:28:03.047501 24518 raft_consensus.cc:504] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.047925 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce753b9bac814f17a869f4f8d66deca6" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:03.050177 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce753b9bac814f17a869f4f8d66deca6" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:03.042999 24200 catalog_manager.cc:5495] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:03.056607 24377 raft_consensus.cc:2454] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:03.058027 24417 leader_election.cc:302] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:03.050802 24496 leader_election.cc:288] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:03.053687 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3550e2841c1f41939833b5588fb35875" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:03.055862 24303 raft_consensus.cc:2454] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:03.059921 24494 raft_consensus.cc:348] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.061043 24494 raft_consensus.cc:374] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.061434 24494 raft_consensus.cc:729] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.062376 24494 consensus_queue.cc:260] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.065955 24496 raft_consensus.cc:2781] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:03.067111 24496 raft_consensus.cc:686] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:03.067914 24496 consensus_queue.cc:237] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.069186 24518 leader_election.cc:288] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:03.070019 24518 raft_consensus.cc:2781] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:03.070540 24518 raft_consensus.cc:482] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:03.054700 24547 leader_election.cc:288] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:03.071097 24518 raft_consensus.cc:3037] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.072745 24452 raft_consensus.cc:2379] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cafc2c5ad4df4224a5ccc4893de70144 in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:03.075137 24494 ts_tablet_manager.cc:1428] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260924 14:28:03.076449 24494 tablet_bootstrap.cc:492] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:03.076550 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:03.077515 24303 raft_consensus.cc:2379] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate cafc2c5ad4df4224a5ccc4893de70144 in this term.
W20260924 14:28:03.079226 24417 leader_election.cc:341] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.080852 24525 raft_consensus.cc:482] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.081319 24525 raft_consensus.cc:504] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.081897 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:03.082937 24494 tablet_bootstrap.cc:654] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.085777 24493 tablet_bootstrap.cc:492] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:03.086289 24493 ts_tablet_manager.cc:1397] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.183s	user 0.044s	sys 0.072s
I20260924 14:28:03.086692 24525 leader_election.cc:288] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.089172 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b53acab36f9a42838efe3526f4aed03a" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:03.089095 24493 raft_consensus.cc:348] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.089386 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b53acab36f9a42838efe3526f4aed03a" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:03.090190 24376 raft_consensus.cc:2454] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:03.091146 24493 raft_consensus.cc:374] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.091570 24493 raft_consensus.cc:729] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.092348 24493 consensus_queue.cc:260] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.075381 24378 raft_consensus.cc:2454] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
W20260924 14:28:03.090801 24268 leader_election.cc:341] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.095528 24493 ts_tablet_manager.cc:1428] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260924 14:28:03.096487 24493 tablet_bootstrap.cc:492] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:03.096649 24268 leader_election.cc:302] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:03.097935 24525 raft_consensus.cc:2781] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:03.098289 24525 raft_consensus.cc:482] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:03.092876 24417 leader_election.cc:302] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:03.098603 24525 raft_consensus.cc:3037] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.099282 24268 leader_election.cc:302] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:03.100193 24522 raft_consensus.cc:2781] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:03.100641 24522 raft_consensus.cc:686] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:03.101852 24493 tablet_bootstrap.cc:654] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.101409 24522 consensus_queue.cc:237] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.105991 24502 raft_consensus.cc:482] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.106518 24551 raft_consensus.cc:2726] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 14:28:03.106464 24502 raft_consensus.cc:504] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.106590 24525 raft_consensus.cc:504] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.109748 24502 leader_election.cc:288] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.110770 24492 tablet_bootstrap.cc:492] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:03.110641 24518 raft_consensus.cc:504] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.111266 24492 ts_tablet_manager.cc:1397] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.233s	user 0.083s	sys 0.101s
I20260924 14:28:03.111291 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6971512f7f9a4265bb19996677d61320" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:03.113480 24525 leader_election.cc:288] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.114251 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce753b9bac814f17a869f4f8d66deca6" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:03.114413 24492 raft_consensus.cc:348] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.114837 24303 raft_consensus.cc:3037] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.115108 24492 raft_consensus.cc:374] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.115419 24492 raft_consensus.cc:729] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.115548 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce753b9bac814f17a869f4f8d66deca6" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:03.116393 24378 raft_consensus.cc:3037] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.113590 24500 raft_consensus.cc:482] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.116559 24492 consensus_queue.cc:260] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.117389 24500 raft_consensus.cc:504] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.119937 24492 ts_tablet_manager.cc:1428] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260924 14:28:03.120703 24492 tablet_bootstrap.cc:492] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:03.115612 24518 leader_election.cc:288] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:03.122141 24500 leader_election.cc:288] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:03.126385 24268 leader_election.cc:341] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.130098 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:03.130784 24302 raft_consensus.cc:2454] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:03.131297 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b53acab36f9a42838efe3526f4aed03a" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:03.131753 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6971512f7f9a4265bb19996677d61320" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:03.132130 24376 raft_consensus.cc:3037] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.133822 24344 leader_election.cc:302] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:03.135864 24500 raft_consensus.cc:2781] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:03.136513 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:03.135825 24268 leader_election.cc:341] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.136677 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b53acab36f9a42838efe3526f4aed03a" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:03.137392 24450 raft_consensus.cc:3037] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.137305 24268 leader_election.cc:302] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
W20260924 14:28:03.137955 24343 leader_election.cc:341] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.138116 24525 raft_consensus.cc:2726] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:03.136654 24500 raft_consensus.cc:482] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:03.139849 24500 raft_consensus.cc:3037] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.145642 24378 raft_consensus.cc:2454] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:03.147042 24417 leader_election.cc:302] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:03.148218 24200 catalog_manager.cc:5495] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:03.149909 24494 tablet_bootstrap.cc:492] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:03.150493 24494 ts_tablet_manager.cc:1397] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.074s	user 0.009s	sys 0.003s
I20260924 14:28:03.150921 24201 catalog_manager.cc:5495] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:03.154808 24303 raft_consensus.cc:2454] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:03.158212 24493 tablet_bootstrap.cc:492] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:03.158670 24493 ts_tablet_manager.cc:1397] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.062s	user 0.004s	sys 0.007s
I20260924 14:28:03.159669 24492 tablet_bootstrap.cc:654] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.160832 24494 raft_consensus.cc:348] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.161330 24494 raft_consensus.cc:374] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.161626 24494 raft_consensus.cc:729] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.162514 24496 raft_consensus.cc:2781] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:03.163213 24376 raft_consensus.cc:2454] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:03.165463 24496 raft_consensus.cc:686] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:03.166918 24496 consensus_queue.cc:237] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.170150 24494 consensus_queue.cc:260] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.170336 24500 raft_consensus.cc:504] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.171844 24493 raft_consensus.cc:348] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.172530 24500 leader_election.cc:288] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.173159 24268 leader_election.cc:302] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:03.174060 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:03.174896 24493 raft_consensus.cc:374] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.173977 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:03.175653 24303 raft_consensus.cc:3037] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:03.175879 24343 leader_election.cc:341] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.175215 24493 raft_consensus.cc:729] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.177187 24493 consensus_queue.cc:260] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.177742 24494 ts_tablet_manager.cc:1428] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.003s
I20260924 14:28:03.178462 24522 raft_consensus.cc:2781] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:03.178917 24522 raft_consensus.cc:686] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:03.179096 24494 tablet_bootstrap.cc:492] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:03.179699 24522 consensus_queue.cc:237] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.183261 24493 ts_tablet_manager.cc:1428] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.006s
I20260924 14:28:03.184173 24493 tablet_bootstrap.cc:492] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:03.185472 24519 raft_consensus.cc:482] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.185916 24519 raft_consensus.cc:504] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.189718 24450 raft_consensus.cc:2454] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:03.192414 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9c52e97d0444b8ec17719ea5a4ac6" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:03.193181 24302 raft_consensus.cc:2379] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61d316eb5dfe405c95019383bbebecaa in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:03.194115 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9c52e97d0444b8ec17719ea5a4ac6" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:03.194893 24519 leader_election.cc:288] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.196637 24343 leader_election.cc:302] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:03.198174 24519 raft_consensus.cc:3037] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.198681 24518 consensus_queue.cc:1035] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:03.189752 24303 raft_consensus.cc:2454] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:03.200184 24344 leader_election.cc:302] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:03.204905 24500 raft_consensus.cc:2781] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:03.204125 24201 catalog_manager.cc:5495] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
W20260924 14:28:03.211879 24268 consensus_peers.cc:487] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:03.225023 24494 tablet_bootstrap.cc:654] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.229080 24525 raft_consensus.cc:482] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.230254 24525 raft_consensus.cc:504] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.231858 24493 tablet_bootstrap.cc:654] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.232054 24519 raft_consensus.cc:2726] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:03.233136 24519 raft_consensus.cc:482] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.233762 24519 raft_consensus.cc:504] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.236582 24519 leader_election.cc:288] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.238377 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86057fee00d44a4c9607d990f2782813" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:03.236415 24522 consensus_queue.cc:1035] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 14:28:03.243811 24344 leader_election.cc:341] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.246152 24494 tablet_bootstrap.cc:492] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:03.246642 24494 ts_tablet_manager.cc:1397] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.068s	user 0.013s	sys 0.004s
I20260924 14:28:03.251605 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86057fee00d44a4c9607d990f2782813" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:03.252357 24450 raft_consensus.cc:2454] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:03.247368 24525 leader_election.cc:288] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.256142 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4aa8a7302f42b688f37d4fdb892b67" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:03.260354 24494 raft_consensus.cc:348] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.261222 24494 raft_consensus.cc:374] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.261505 24494 raft_consensus.cc:729] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.261433 24551 consensus_queue.cc:1035] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.262280 24494 consensus_queue.cc:260] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.264782 24452 raft_consensus.cc:2454] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:03.265210 24494 ts_tablet_manager.cc:1428] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260924 14:28:03.266091 24494 tablet_bootstrap.cc:492] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:03.275838 24343 leader_election.cc:302] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:03.276201 24268 leader_election.cc:302] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:03.277347 24559 raft_consensus.cc:2781] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:03.277808 24559 raft_consensus.cc:482] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:03.278204 24559 raft_consensus.cc:3037] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.275539 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4aa8a7302f42b688f37d4fdb892b67" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:03.282579 24376 raft_consensus.cc:2454] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:03.283514 24522 raft_consensus.cc:482] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.283649 24525 raft_consensus.cc:2781] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:03.288980 24522 raft_consensus.cc:504] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.284968 24525 raft_consensus.cc:482] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:03.295876 24525 raft_consensus.cc:3037] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.297851 24522 leader_election.cc:288] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.304488 24500 raft_consensus.cc:686] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:03.307989 24492 tablet_bootstrap.cc:492] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:03.307915 24500 consensus_queue.cc:237] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.308492 24492 ts_tablet_manager.cc:1397] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.188s	user 0.011s	sys 0.031s
I20260924 14:28:03.311635 24492 raft_consensus.cc:348] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.312788 24492 raft_consensus.cc:374] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.313124 24492 raft_consensus.cc:729] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.311254 24201 catalog_manager.cc:5495] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:03.313889 24492 consensus_queue.cc:260] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.316208 24492 ts_tablet_manager.cc:1428] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260924 14:28:03.317075 24492 tablet_bootstrap.cc:492] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:03.317314 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:03.321692 24268 leader_election.cc:341] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.320658 24559 raft_consensus.cc:504] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.333283 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:03.334729 24559 leader_election.cc:288] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:03.335101 24268 leader_election.cc:341] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.338434 24268 leader_election.cc:302] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:03.339136 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86057fee00d44a4c9607d990f2782813" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:03.332432 24551 raft_consensus.cc:482] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.345374 24551 raft_consensus.cc:504] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.343971 24452 raft_consensus.cc:3037] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.346402 24525 raft_consensus.cc:504] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.352743 24525 leader_election.cc:288] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.353351 24525 raft_consensus.cc:2726] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:03.354221 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4aa8a7302f42b688f37d4fdb892b67" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:03.354714 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4aa8a7302f42b688f37d4fdb892b67" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:03.355036 24378 raft_consensus.cc:3037] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.355365 24450 raft_consensus.cc:3037] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.356768 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:03.358372 24302 raft_consensus.cc:2379] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate 61d316eb5dfe405c95019383bbebecaa in this term.
I20260924 14:28:03.356976 24551 leader_election.cc:288] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:03.362812 24494 tablet_bootstrap.cc:654] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.349848 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86057fee00d44a4c9607d990f2782813" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:03.368942 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:03.369591 24344 leader_election.cc:341] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.375525 24417 leader_election.cc:302] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:03.376760 24551 raft_consensus.cc:3037] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.377830 24201 catalog_manager.cc:5495] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:03.381314 24450 raft_consensus.cc:2454] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:03.383419 24268 leader_election.cc:302] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:03.388105 24551 raft_consensus.cc:2726] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:03.391903 24493 tablet_bootstrap.cc:492] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:03.392381 24493 ts_tablet_manager.cc:1397] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.208s	user 0.016s	sys 0.009s
I20260924 14:28:03.394037 24452 raft_consensus.cc:2454] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:03.395555 24343 leader_election.cc:302] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:03.396291 24492 tablet_bootstrap.cc:654] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.396730 24500 raft_consensus.cc:2781] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:03.397209 24500 raft_consensus.cc:686] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:03.397372 24522 raft_consensus.cc:2781] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:03.397814 24522 raft_consensus.cc:686] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:03.398070 24500 consensus_queue.cc:237] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.399258 24569 raft_consensus.cc:482] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.398980 24522 consensus_queue.cc:237] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.399888 24569 raft_consensus.cc:504] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.403739 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:03.404198 24569 leader_election.cc:288] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.408716 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:03.414495 24493 raft_consensus.cc:348] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.415627 24493 raft_consensus.cc:374] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.416230 24493 raft_consensus.cc:729] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.416716 24378 raft_consensus.cc:2454] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:03.419067 24493 consensus_queue.cc:260] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:03.407650 24268 leader_election.cc:341] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:03.424145 24268 leader_election.cc:341] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.424800 24268 leader_election.cc:302] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:03.426111 24493 ts_tablet_manager.cc:1428] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.004s
I20260924 14:28:03.427258 24493 tablet_bootstrap.cc:492] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:03.435060 24572 raft_consensus.cc:2726] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:03.434180 24569 raft_consensus.cc:482] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.438016 24569 raft_consensus.cc:504] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.444403 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe06d46f9904520a8709a2b3824f11c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:03.445380 24378 raft_consensus.cc:2454] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:03.447721 24268 leader_election.cc:302] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:03.447045 24201 catalog_manager.cc:5495] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:03.447887 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe06d46f9904520a8709a2b3824f11c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:03.449273 24572 raft_consensus.cc:2781] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:03.450079 24493 tablet_bootstrap.cc:654] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:03.449522 24268 leader_election.cc:341] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.449990 24572 raft_consensus.cc:482] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:03.453488 24572 raft_consensus.cc:3037] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.456152 24492 tablet_bootstrap.cc:492] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:03.456527 24492 ts_tablet_manager.cc:1397] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.140s	user 0.015s	sys 0.014s
I20260924 14:28:03.450680 24569 leader_election.cc:288] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.452629 24494 tablet_bootstrap.cc:492] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:03.458271 24500 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.459364 24494 ts_tablet_manager.cc:1397] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.193s	user 0.009s	sys 0.006s
W20260924 14:28:03.466748 24248 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:03.469401 24492 raft_consensus.cc:348] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:03.472675 24492 raft_consensus.cc:374] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.473093 24492 raft_consensus.cc:729] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.474068 24492 consensus_queue.cc:260] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:03.476688 24492 ts_tablet_manager.cc:1428] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260924 14:28:03.476346 24199 catalog_manager.cc:5495] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:03.477815 24492 tablet_bootstrap.cc:492] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:03.485342 24492 tablet_bootstrap.cc:654] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.492300 24493 tablet_bootstrap.cc:492] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:03.492923 24493 ts_tablet_manager.cc:1397] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.066s	user 0.020s	sys 0.017s
I20260924 14:28:03.496323 24493 raft_consensus.cc:348] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.497063 24492 tablet_bootstrap.cc:492] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:03.497308 24493 raft_consensus.cc:374] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.497888 24492 ts_tablet_manager.cc:1397] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260924 14:28:03.463135 24494 raft_consensus.cc:348] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.499830 24494 raft_consensus.cc:374] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.500202 24494 raft_consensus.cc:729] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.501024 24494 consensus_queue.cc:260] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.471071 24572 raft_consensus.cc:504] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.497742 24493 raft_consensus.cc:729] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.505129 24493 consensus_queue.cc:260] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.504377 24494 ts_tablet_manager.cc:1428] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.001s
I20260924 14:28:03.506875 24494 tablet_bootstrap.cc:492] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:03.507905 24500 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:03.508926 24493 ts_tablet_manager.cc:1428] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260924 14:28:03.509830 24493 tablet_bootstrap.cc:492] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:03.512105 24378 raft_consensus.cc:3037] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.514447 24492 raft_consensus.cc:348] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.516301 24492 raft_consensus.cc:374] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.516583 24492 raft_consensus.cc:729] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.517346 24492 consensus_queue.cc:260] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.519749 24492 ts_tablet_manager.cc:1428] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.004s
I20260924 14:28:03.520674 24492 tablet_bootstrap.cc:492] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:03.537334 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe06d46f9904520a8709a2b3824f11c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:03.538115 24377 raft_consensus.cc:3037] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.542519 24493 tablet_bootstrap.cc:654] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.546595 24494 tablet_bootstrap.cc:654] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.550099 24492 tablet_bootstrap.cc:654] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.555038 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe06d46f9904520a8709a2b3824f11c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:03.555891 24450 raft_consensus.cc:3037] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.563848 24377 raft_consensus.cc:2454] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:03.563709 24551 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.565151 24268 leader_election.cc:302] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:03.566159 24569 raft_consensus.cc:2781] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:03.566612 24569 raft_consensus.cc:686] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:03.567382 24569 consensus_queue.cc:237] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.562896 24572 leader_election.cc:288] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:03.571128 24450 raft_consensus.cc:2454] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:03.585637 24492 tablet_bootstrap.cc:492] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:03.586341 24492 ts_tablet_manager.cc:1397] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.066s	user 0.018s	sys 0.016s
I20260924 14:28:03.593756 24492 raft_consensus.cc:348] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.594519 24492 raft_consensus.cc:374] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.594776 24492 raft_consensus.cc:729] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.595532 24492 consensus_queue.cc:260] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.597638 24492 ts_tablet_manager.cc:1428] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260924 14:28:03.598551 24492 tablet_bootstrap.cc:492] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:03.599064 24201 catalog_manager.cc:5495] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:03.606030 24492 tablet_bootstrap.cc:654] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.606276 24518 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.613456 24493 tablet_bootstrap.cc:492] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:03.614064 24493 ts_tablet_manager.cc:1397] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.104s	user 0.005s	sys 0.013s
I20260924 14:28:03.616125 24494 tablet_bootstrap.cc:492] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:03.616899 24494 ts_tablet_manager.cc:1397] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.110s	user 0.015s	sys 0.021s
I20260924 14:28:03.618119 24493 raft_consensus.cc:348] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.619259 24493 raft_consensus.cc:374] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.619578 24493 raft_consensus.cc:729] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.620206 24493 consensus_queue.cc:260] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.621300 24494 raft_consensus.cc:348] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.622064 24494 raft_consensus.cc:374] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.622397 24494 raft_consensus.cc:729] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.623188 24494 consensus_queue.cc:260] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.626237 24494 ts_tablet_manager.cc:1428] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260924 14:28:03.635780 24572 consensus_queue.cc:1035] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.677991 24494 tablet_bootstrap.cc:492] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:03.684590 24569 consensus_queue.cc:1035] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.686435 24518 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:03.690915 24493 ts_tablet_manager.cc:1428] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.003s
I20260924 14:28:03.699240 24551 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.760512 24493 tablet_bootstrap.cc:492] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:03.793888 24559 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260924 14:28:03.795601 24417 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 71159.3us
I20260924 14:28:03.797050 24525 consensus_queue.cc:1035] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.799311 24525 consensus_queue.cc:1035] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.800729 24452 raft_consensus.cc:3037] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:03.821908 24549 raft_consensus.cc:482] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.822685 24549 raft_consensus.cc:504] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.824779 24494 tablet_bootstrap.cc:654] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.829172 24492 tablet_bootstrap.cc:492] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:03.829638 24492 ts_tablet_manager.cc:1397] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.231s	user 0.031s	sys 0.026s
I20260924 14:28:03.847427 24563 consensus_queue.cc:1035] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.848368 24493 tablet_bootstrap.cc:654] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:03.858124 24492 raft_consensus.cc:348] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.859092 24492 raft_consensus.cc:374] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:03.859388 24492 raft_consensus.cc:729] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:03.862095 24492 consensus_queue.cc:260] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.860836 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:03.855536 24549 leader_election.cc:288] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:03.867273 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:03.879113 24417 leader_election.cc:341] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:03.900311 24344 consensus_peers.cc:487] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:03.915377 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 53482.2us
W20260924 14:28:03.917152 24418 leader_election.cc:341] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:03.917636 24418 leader_election.cc:302] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:03.933293 24559 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:03.947435 24594 raft_consensus.cc:2726] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:03.959470 24569 consensus_queue.cc:1035] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:03.957361 24525 raft_consensus.cc:482] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:03.963992 24525 raft_consensus.cc:504] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.973625 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdd9163fb2d44068b0f93f1bbfd7387" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:03.993789 24492 ts_tablet_manager.cc:1428] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.164s	user 0.009s	sys 0.011s
I20260924 14:28:03.996654 24492 tablet_bootstrap.cc:492] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:03.998201 24500 raft_consensus.cc:482] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.043490 24559 raft_consensus.cc:482] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.044113 24559 raft_consensus.cc:504] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.048647 24500 raft_consensus.cc:504] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:04.060948 24596 log.cc:924] Time spent T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260924 14:28:04.023327 24268 leader_election.cc:341] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:04.065519 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 54949.2us
I20260924 14:28:04.066753 24502 consensus_queue.cc:1035] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:04.052584 24518 raft_consensus.cc:482] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.069489 24518 raft_consensus.cc:504] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:03.999883 24525 leader_election.cc:288] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.076828 24518 leader_election.cc:288] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:04.079196 24559 leader_election.cc:288] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.051290 24500 leader_election.cc:288] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.081423 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.088701 24494 tablet_bootstrap.cc:492] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:04.089193 24494 ts_tablet_manager.cc:1397] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.412s	user 0.022s	sys 0.019s
I20260924 14:28:04.091032 24493 tablet_bootstrap.cc:492] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:04.091492 24493 ts_tablet_manager.cc:1397] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.332s	user 0.035s	sys 0.051s
I20260924 14:28:04.088583 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:04.092428 24494 raft_consensus.cc:348] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.100334 24494 raft_consensus.cc:374] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.100895 24494 raft_consensus.cc:729] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.101981 24494 consensus_queue.cc:260] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.106653 24494 ts_tablet_manager.cc:1428] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260924 14:28:04.107429 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdd9163fb2d44068b0f93f1bbfd7387" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:04.109977 24493 raft_consensus.cc:348] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:04.111490 24418 leader_election.cc:341] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.110931 24493 raft_consensus.cc:374] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.112505 24493 raft_consensus.cc:729] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.113230 24494 tablet_bootstrap.cc:492] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:04.113368 24525 raft_consensus.cc:482] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.113790 24525 raft_consensus.cc:504] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.114234 24493 consensus_queue.cc:260] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:04.111904 24268 leader_election.cc:341] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.119068 24493 ts_tablet_manager.cc:1428] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260924 14:28:04.116319 24525 leader_election.cc:288] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:04.116642 24417 leader_election.cc:341] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.118094 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:04.122798 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.122550 24417 leader_election.cc:302] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:04.125844 24492 tablet_bootstrap.cc:654] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.128903 24602 raft_consensus.cc:2726] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:28:04.130522 24343 leader_election.cc:341] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.131174 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.120632 24268 leader_election.cc:302] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:04.131381 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:04.133770 24502 raft_consensus.cc:2726] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:04.135347 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be2b72b568242d3855180c905e78d71" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:04.138110 24343 leader_election.cc:341] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.142303 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be2b72b568242d3855180c905e78d71" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:04.143462 24268 leader_election.cc:341] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:04.138515 24344 leader_election.cc:341] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.141597 24502 consensus_queue.cc:1035] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260924 14:28:04.149649 24494 tablet_bootstrap.cc:654] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.149420 24569 consensus_queue.cc:1035] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:04.149570 24344 leader_election.cc:302] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
W20260924 14:28:04.154089 24344 leader_election.cc:341] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.154536 24344 leader_election.cc:302] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:04.155606 24559 raft_consensus.cc:2726] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:04.156281 24493 tablet_bootstrap.cc:492] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:04.157099 24559 raft_consensus.cc:2726] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:28:04.172251 24268 leader_election.cc:341] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.177862 24268 leader_election.cc:302] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:04.178871 24496 raft_consensus.cc:482] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:04.179800 24496 raft_consensus.cc:504] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:04.180799 24502 raft_consensus.cc:2726] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:04.192407 24493 tablet_bootstrap.cc:654] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.229947 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:04.232399 24496 leader_election.cc:288] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:04.233661 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.235332 24418 leader_election.cc:302] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:04.236773 24496 raft_consensus.cc:2726] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:28:04.269450 24492 tablet_bootstrap.cc:492] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:04.270011 24492 ts_tablet_manager.cc:1397] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.274s	user 0.070s	sys 0.037s
I20260924 14:28:04.272686 24494 tablet_bootstrap.cc:492] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:04.273236 24494 ts_tablet_manager.cc:1397] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.160s	user 0.012s	sys 0.005s
I20260924 14:28:04.273120 24492 raft_consensus.cc:348] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.273972 24492 raft_consensus.cc:374] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.274307 24492 raft_consensus.cc:729] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.275022 24492 consensus_queue.cc:260] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.276373 24494 raft_consensus.cc:348] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.278199 24494 raft_consensus.cc:374] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.279690 24494 raft_consensus.cc:729] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.280411 24494 consensus_queue.cc:260] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.282750 24494 ts_tablet_manager.cc:1428] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260924 14:28:04.277388 24492 ts_tablet_manager.cc:1428] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:28:04.287371 24492 tablet_bootstrap.cc:492] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:04.289081 24493 tablet_bootstrap.cc:492] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:04.289558 24493 ts_tablet_manager.cc:1397] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.134s	user 0.017s	sys 0.028s
I20260924 14:28:04.290738 24494 tablet_bootstrap.cc:492] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:04.296515 24492 tablet_bootstrap.cc:654] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.302424 24493 raft_consensus.cc:348] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.303249 24493 raft_consensus.cc:374] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.303543 24493 raft_consensus.cc:729] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.304562 24525 raft_consensus.cc:482] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.304256 24493 consensus_queue.cc:260] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.305078 24525 raft_consensus.cc:504] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.310145 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86057fee00d44a4c9607d990f2782813" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:04.311774 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86057fee00d44a4c9607d990f2782813" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:04.312881 24493 ts_tablet_manager.cc:1428] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260924 14:28:04.313828 24493 tablet_bootstrap.cc:492] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:04.316110 24268 leader_election.cc:302] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:04.317124 24522 raft_consensus.cc:2726] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:04.319581 24493 tablet_bootstrap.cc:654] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.307260 24525 leader_election.cc:288] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.327695 24494 tablet_bootstrap.cc:654] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.336786 24492 tablet_bootstrap.cc:492] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:04.337414 24492 ts_tablet_manager.cc:1397] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.050s	user 0.009s	sys 0.030s
I20260924 14:28:04.341733 24493 tablet_bootstrap.cc:492] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:04.342339 24493 ts_tablet_manager.cc:1397] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.001s
I20260924 14:28:04.344976 24494 tablet_bootstrap.cc:492] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:04.345433 24494 ts_tablet_manager.cc:1397] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.004s
I20260924 14:28:04.345479 24604 raft_consensus.cc:482] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.346028 24492 raft_consensus.cc:348] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.346019 24604 raft_consensus.cc:504] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.346282 24493 raft_consensus.cc:348] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.346989 24492 raft_consensus.cc:374] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.347146 24493 raft_consensus.cc:374] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.347401 24492 raft_consensus.cc:729] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.347592 24493 raft_consensus.cc:729] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.348315 24492 consensus_queue.cc:260] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.348600 24494 raft_consensus.cc:348] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.348467 24493 consensus_queue.cc:260] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.349756 24494 raft_consensus.cc:374] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.350248 24494 raft_consensus.cc:729] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.351815 24493 ts_tablet_manager.cc:1428] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260924 14:28:04.352123 24492 ts_tablet_manager.cc:1428] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260924 14:28:04.352125 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.353055 24301 raft_consensus.cc:2454] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:04.353351 24492 tablet_bootstrap.cc:492] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:04.354218 24344 leader_election.cc:302] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:04.355026 24500 raft_consensus.cc:2781] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:04.355369 24500 raft_consensus.cc:482] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:04.355796 24500 raft_consensus.cc:3037] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.357257 24493 tablet_bootstrap.cc:492] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:04.357908 24494 consensus_queue.cc:260] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.358667 24604 leader_election.cc:288] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.360723 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:04.362733 24343 leader_election.cc:341] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.363921 24492 tablet_bootstrap.cc:654] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.365739 24494 ts_tablet_manager.cc:1428] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260924 14:28:04.366677 24494 tablet_bootstrap.cc:492] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:04.372285 24494 tablet_bootstrap.cc:654] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.375839 24493 tablet_bootstrap.cc:654] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.382036 24492 tablet_bootstrap.cc:492] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:04.382516 24492 ts_tablet_manager.cc:1397] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.005s
I20260924 14:28:04.386332 24492 raft_consensus.cc:348] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.387091 24492 raft_consensus.cc:374] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.387341 24492 raft_consensus.cc:729] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.387977 24492 consensus_queue.cc:260] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.390178 24492 ts_tablet_manager.cc:1428] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:28:04.391642 24492 tablet_bootstrap.cc:492] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:04.392903 24493 tablet_bootstrap.cc:492] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:04.392437 24500 raft_consensus.cc:504] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.393378 24493 ts_tablet_manager.cc:1397] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.008s
I20260924 14:28:04.395618 24500 leader_election.cc:288] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.396008 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:04.396430 24493 raft_consensus.cc:348] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.396522 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:04.397258 24493 raft_consensus.cc:374] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.396888 24302 raft_consensus.cc:3037] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.397599 24493 raft_consensus.cc:729] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
W20260924 14:28:04.398468 24343 leader_election.cc:341] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.398633 24493 consensus_queue.cc:260] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.401731 24493 ts_tablet_manager.cc:1428] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260924 14:28:04.402741 24493 tablet_bootstrap.cc:492] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:04.403717 24494 tablet_bootstrap.cc:492] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:04.404244 24494 ts_tablet_manager.cc:1397] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.004s
I20260924 14:28:04.405334 24492 tablet_bootstrap.cc:654] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.407318 24500 raft_consensus.cc:482] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.407621 24494 raft_consensus.cc:348] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.408363 24494 raft_consensus.cc:374] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.408066 24500 raft_consensus.cc:504] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.412248 24500 leader_election.cc:288] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.408752 24494 raft_consensus.cc:729] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.413030 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdd9163fb2d44068b0f93f1bbfd7387" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:04.413518 24493 tablet_bootstrap.cc:654] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.413965 24448 raft_consensus.cc:2454] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:04.414951 24494 consensus_queue.cc:260] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.415673 24343 leader_election.cc:302] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:04.416476 24500 raft_consensus.cc:2781] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:04.416884 24500 raft_consensus.cc:482] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:04.416533 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdd9163fb2d44068b0f93f1bbfd7387" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.417243 24500 raft_consensus.cc:3037] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.417543 24494 ts_tablet_manager.cc:1428] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260924 14:28:04.417606 24303 raft_consensus.cc:2454] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:04.425134 24500 raft_consensus.cc:504] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.428339 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdd9163fb2d44068b0f93f1bbfd7387" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:04.428644 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdd9163fb2d44068b0f93f1bbfd7387" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:04.429019 24448 raft_consensus.cc:3037] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.429477 24303 raft_consensus.cc:3037] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.397408 24301 raft_consensus.cc:3037] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.438701 24448 raft_consensus.cc:2454] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:04.440935 24343 leader_election.cc:302] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:04.441795 24604 raft_consensus.cc:2781] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:04.442287 24604 raft_consensus.cc:686] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:04.443048 24604 consensus_queue.cc:237] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.451156 24494 tablet_bootstrap.cc:492] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:04.458621 24492 tablet_bootstrap.cc:492] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:04.459125 24492 ts_tablet_manager.cc:1397] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.068s	user 0.006s	sys 0.014s
I20260924 14:28:04.461210 24494 tablet_bootstrap.cc:654] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.462958 24500 leader_election.cc:288] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.465958 24492 raft_consensus.cc:348] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.468600 24492 raft_consensus.cc:374] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.469421 24492 raft_consensus.cc:729] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.470552 24492 consensus_queue.cc:260] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.474702 24492 ts_tablet_manager.cc:1428] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260924 14:28:04.478396 24492 tablet_bootstrap.cc:492] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:04.503301 24492 tablet_bootstrap.cc:654] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.503332 24494 tablet_bootstrap.cc:492] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:04.504310 24494 ts_tablet_manager.cc:1397] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.028s
W20260924 14:28:04.512360 24248 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:04.512794 24563 raft_consensus.cc:482] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.513366 24563 raft_consensus.cc:504] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.531092 24563 leader_election.cc:288] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.541566 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6971512f7f9a4265bb19996677d61320" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:04.544878 24493 tablet_bootstrap.cc:492] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:04.545408 24493 ts_tablet_manager.cc:1397] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.143s	user 0.052s	sys 0.047s
I20260924 14:28:04.545240 24494 raft_consensus.cc:348] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.546059 24494 raft_consensus.cc:374] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.546361 24494 raft_consensus.cc:729] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.545960 24200 catalog_manager.cc:5495] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:04.546995 24494 consensus_queue.cc:260] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:04.549072 24343 leader_election.cc:341] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.538520 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6971512f7f9a4265bb19996677d61320" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.550969 24300 raft_consensus.cc:2454] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:04.550863 24493 raft_consensus.cc:348] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.551730 24302 raft_consensus.cc:2454] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:04.552208 24493 raft_consensus.cc:374] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.552780 24493 raft_consensus.cc:729] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.552809 24344 leader_election.cc:302] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:04.553714 24493 consensus_queue.cc:260] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.554638 24344 leader_election.cc:302] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:04.555503 24604 raft_consensus.cc:2781] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:04.557312 24604 raft_consensus.cc:482] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:04.557693 24494 ts_tablet_manager.cc:1428] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.001s
I20260924 14:28:04.557749 24604 raft_consensus.cc:3037] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.555553 24303 raft_consensus.cc:2454] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:04.556399 24500 raft_consensus.cc:2781] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:04.556972 24493 ts_tablet_manager.cc:1428] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260924 14:28:04.612684 24559 raft_consensus.cc:482] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.623608 24559 raft_consensus.cc:504] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.581560 24496 raft_consensus.cc:482] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.625900 24496 raft_consensus.cc:504] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.640007 24602 raft_consensus.cc:482] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.640727 24602 raft_consensus.cc:504] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.643944 24494 tablet_bootstrap.cc:492] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:04.646955 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:04.648419 24525 raft_consensus.cc:482] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.648964 24525 raft_consensus.cc:504] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.647408 24604 raft_consensus.cc:504] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:04.653725 24417 leader_election.cc:341] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.654624 24602 leader_election.cc:288] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:04.655588 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.655943 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6971512f7f9a4265bb19996677d61320" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:04.656524 24303 raft_consensus.cc:2454] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
W20260924 14:28:04.657397 24343 leader_election.cc:341] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.657860 24522 raft_consensus.cc:482] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.657809 24418 leader_election.cc:302] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:04.658430 24522 raft_consensus.cc:504] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.659329 24602 raft_consensus.cc:2781] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:04.659866 24602 raft_consensus.cc:482] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:04.660257 24602 raft_consensus.cc:3037] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.663424 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c4bc71d12c14157982c98b4c7ca505c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:04.664026 24563 raft_consensus.cc:482] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.667840 24563 raft_consensus.cc:504] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.671214 24522 leader_election.cc:288] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.643944 24493 tablet_bootstrap.cc:492] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:04.672273 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f04a45264e7d8be03ae56ab10560" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:04.673055 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:04.673755 24449 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:04.673908 24563 leader_election.cc:288] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:04.675278 24343 leader_election.cc:341] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.676111 24268 leader_election.cc:302] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:04.677389 24559 leader_election.cc:288] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.678741 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be2b72b568242d3855180c905e78d71" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:04.679576 24449 raft_consensus.cc:2454] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:04.680719 24492 tablet_bootstrap.cc:492] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:04.681191 24492 ts_tablet_manager.cc:1397] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.203s	user 0.015s	sys 0.015s
I20260924 14:28:04.681005 24343 leader_election.cc:302] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:04.683668 24604 leader_election.cc:288] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.683823 24492 raft_consensus.cc:348] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.685708 24626 raft_consensus.cc:2781] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:04.686028 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be2b72b568242d3855180c905e78d71" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:04.686395 24626 raft_consensus.cc:482] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:04.686595 24494 tablet_bootstrap.cc:654] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.686753 24626 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.686990 24496 leader_election.cc:288] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:04.687909 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be2b72b568242d3855180c905e78d71" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.688557 24303 raft_consensus.cc:2454] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:04.688393 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:04.689175 24377 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:04.690212 24525 leader_election.cc:288] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.690696 24418 leader_election.cc:302] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:04.686733 24375 raft_consensus.cc:2454] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:04.692184 24604 raft_consensus.cc:2781] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:04.692729 24492 raft_consensus.cc:374] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:04.693094 24492 raft_consensus.cc:729] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:04.693209 24500 raft_consensus.cc:686] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:04.694056 24492 consensus_queue.cc:260] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.694465 24500 consensus_queue.cc:237] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.694810 24604 raft_consensus.cc:482] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:04.705132 24604 raft_consensus.cc:3037] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.706022 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c4bc71d12c14157982c98b4c7ca505c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:04.710928 24492 ts_tablet_manager.cc:1428] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.005s
I20260924 14:28:04.712098 24492 tablet_bootstrap.cc:492] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:04.713402 24493 tablet_bootstrap.cc:654] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.697264 24496 raft_consensus.cc:2781] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:04.725804 24626 raft_consensus.cc:504] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.749599 24626 leader_election.cc:288] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.744134 24631 raft_consensus.cc:482] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:04.744865 24569 raft_consensus.cc:482] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:28:04.707425 24268 leader_election.cc:341] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.765703 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6971512f7f9a4265bb19996677d61320" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:04.742672 24496 raft_consensus.cc:482] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:04.800767 24496 raft_consensus.cc:3037] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.791927 24302 raft_consensus.cc:3037] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:04.791206 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 83304.6us
I20260924 14:28:04.789952 24569 raft_consensus.cc:504] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.790148 24631 raft_consensus.cc:504] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.835608 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be2b72b568242d3855180c905e78d71" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.851504 24300 raft_consensus.cc:2454] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:04.902324 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f04a45264e7d8be03ae56ab10560" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:04.903640 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:04.904312 24449 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.905439 24631 leader_election.cc:288] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.908080 24604 raft_consensus.cc:504] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.913081 24604 leader_election.cc:288] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.914445 24492 tablet_bootstrap.cc:654] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:04.898109 24569 leader_election.cc:288] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:04.922607 24602 raft_consensus.cc:504] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:04.925125 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:04.922132 24449 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:04.926288 24375 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:04.920051 24268 leader_election.cc:341] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.927901 24268 leader_election.cc:302] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:04.929996 24569 raft_consensus.cc:2726] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:04.932826 24302 raft_consensus.cc:2454] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
W20260924 14:28:04.940778 24344 leader_election.cc:341] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:04.941344 24344 leader_election.cc:302] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:04.942662 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be2b72b568242d3855180c905e78d71" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:04.943411 24604 raft_consensus.cc:2726] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:04.944243 24302 raft_consensus.cc:3037] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.954955 24375 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:04.960041 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be2b72b568242d3855180c905e78d71" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:04.997890 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:04.998721 24300 raft_consensus.cc:3037] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:04.999917 24496 raft_consensus.cc:504] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.001451 24448 raft_consensus.cc:2379] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 61d316eb5dfe405c95019383bbebecaa in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:05.002172 24602 leader_election.cc:288] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:05.011008 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:05.011632 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be2b72b568242d3855180c905e78d71" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
W20260924 14:28:05.013870 24417 leader_election.cc:341] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.014576 24602 raft_consensus.cc:482] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.015077 24602 raft_consensus.cc:504] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.015658 24302 raft_consensus.cc:2454] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:05.016331 24494 tablet_bootstrap.cc:492] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:05.016795 24494 ts_tablet_manager.cc:1397] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.373s	user 0.043s	sys 0.045s
I20260924 14:28:05.018257 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a8256035984c7fb87f7e9212b21dc7" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
W20260924 14:28:05.019807 24418 leader_election.cc:341] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.019469 24494 raft_consensus.cc:348] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.020919 24494 raft_consensus.cc:374] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.021281 24494 raft_consensus.cc:729] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.022305 24494 consensus_queue.cc:260] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.025529 24494 ts_tablet_manager.cc:1428] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260924 14:28:05.021487 24268 leader_election.cc:302] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:05.026902 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a8256035984c7fb87f7e9212b21dc7" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:05.026672 24604 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:05.027824 24602 leader_election.cc:288] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:05.029703 24494 tablet_bootstrap.cc:492] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
W20260924 14:28:05.029938 24417 leader_election.cc:341] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.030531 24569 raft_consensus.cc:2781] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:05.030652 24417 leader_election.cc:302] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:05.031050 24569 raft_consensus.cc:686] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:05.025766 24344 leader_election.cc:302] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:05.032671 24569 consensus_queue.cc:237] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.033147 24633 raft_consensus.cc:2726] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:05.025470 24300 raft_consensus.cc:2454] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:05.036461 24418 leader_election.cc:302] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:05.038141 24633 raft_consensus.cc:2781] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:05.038285 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be2b72b568242d3855180c905e78d71" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:05.038875 24633 raft_consensus.cc:686] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:05.043268 24375 raft_consensus.cc:2379] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate 61d316eb5dfe405c95019383bbebecaa in this term.
I20260924 14:28:05.043377 24633 consensus_queue.cc:237] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.039378 24496 leader_election.cc:288] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:05.045315 24604 raft_consensus.cc:2781] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:05.047889 24604 raft_consensus.cc:686] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:05.048309 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:05.049258 24448 raft_consensus.cc:2454] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:05.049518 24604 consensus_queue.cc:237] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.055970 24344 leader_election.cc:302] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:05.045874 24417 leader_election.cc:302] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:05.046316 24493 tablet_bootstrap.cc:492] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:05.055028 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:05.061558 24493 ts_tablet_manager.cc:1397] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.418s	user 0.011s	sys 0.024s
I20260924 14:28:05.061043 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:05.063647 24496 raft_consensus.cc:2726] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 14:28:05.063908 24630 raft_consensus.cc:2781] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:05.064461 24630 raft_consensus.cc:686] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:05.065239 24630 consensus_queue.cc:237] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.078737 24494 tablet_bootstrap.cc:654] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.069739 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:05.069552 24268 leader_election.cc:302] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:05.084708 24569 raft_consensus.cc:2781] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:05.085114 24569 raft_consensus.cc:482] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:05.085431 24569 raft_consensus.cc:3037] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:05.085947 24268 leader_election.cc:341] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.070501 24377 raft_consensus.cc:2454] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:05.090852 24492 tablet_bootstrap.cc:492] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:05.091588 24492 ts_tablet_manager.cc:1397] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.380s	user 0.017s	sys 0.006s
I20260924 14:28:05.095028 24268 leader_election.cc:302] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:05.096635 24604 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:05.098412 24201 catalog_manager.cc:5495] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:05.103746 24492 raft_consensus.cc:348] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.104280 24492 raft_consensus.cc:374] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.104511 24492 raft_consensus.cc:729] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.105082 24492 consensus_queue.cc:260] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.108040 24492 ts_tablet_manager.cc:1428] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260924 14:28:05.108722 24492 tablet_bootstrap.cc:492] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:05.102233 24198 catalog_manager.cc:5495] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:05.114535 24626 raft_consensus.cc:2781] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:05.115880 24626 raft_consensus.cc:482] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:28:05.116159 24268 leader_election.cc:341] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.116408 24626 raft_consensus.cc:3037] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.121011 24493 raft_consensus.cc:348] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.122268 24493 raft_consensus.cc:374] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.122615 24493 raft_consensus.cc:729] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.124158 24493 consensus_queue.cc:260] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.126533 24493 ts_tablet_manager.cc:1428] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.001s
I20260924 14:28:05.127707 24602 raft_consensus.cc:482] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.128202 24602 raft_consensus.cc:504] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.130039 24493 tablet_bootstrap.cc:492] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:05.136094 24602 leader_election.cc:288] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:05.143141 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:05.153591 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:05.154356 24375 raft_consensus.cc:2454] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:05.143110 24500 consensus_queue.cc:1035] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260924 14:28:05.161576 24417 leader_election.cc:302] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:05.162171 24569 raft_consensus.cc:504] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.162715 24602 raft_consensus.cc:2781] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:05.164500 24602 raft_consensus.cc:482] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:05.165324 24602 raft_consensus.cc:3037] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.178526 24494 tablet_bootstrap.cc:492] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:05.179030 24494 ts_tablet_manager.cc:1397] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.150s	user 0.021s	sys 0.012s
I20260924 14:28:05.181232 24200 catalog_manager.cc:5495] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:05.182430 24200 catalog_manager.cc:5495] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:05.155424 24626 raft_consensus.cc:504] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.183835 24200 catalog_manager.cc:5495] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:05.196952 24302 raft_consensus.cc:2379] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate cafc2c5ad4df4224a5ccc4893de70144 in this term.
I20260924 14:28:05.198287 24569 leader_election.cc:288] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.205374 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:05.207178 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:05.208608 24602 raft_consensus.cc:504] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.209561 24494 raft_consensus.cc:348] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:05.211382 24343 consensus_peers.cc:487] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:05.217046 24494 raft_consensus.cc:374] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.217891 24494 raft_consensus.cc:729] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.218658 24494 consensus_queue.cc:260] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.218607 24448 raft_consensus.cc:3037] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.226787 24494 ts_tablet_manager.cc:1428] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.001s
I20260924 14:28:05.226856 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:05.228142 24302 raft_consensus.cc:2379] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate cafc2c5ad4df4224a5ccc4893de70144 in this term.
I20260924 14:28:05.228189 24494 tablet_bootstrap.cc:492] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:05.229810 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:05.230412 24377 raft_consensus.cc:3037] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.230633 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:05.232169 24492 tablet_bootstrap.cc:654] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:05.231238 24268 leader_election.cc:341] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.233860 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:05.224354 24626 leader_election.cc:288] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.236617 24494 tablet_bootstrap.cc:654] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.235570 24449 raft_consensus.cc:2379] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cafc2c5ad4df4224a5ccc4893de70144 in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:05.234277 24602 leader_election.cc:288] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:05.242457 24268 leader_election.cc:302] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:05.243368 24626 raft_consensus.cc:2726] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 14:28:05.262475 24377 raft_consensus.cc:2454] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:05.264704 24493 tablet_bootstrap.cc:654] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.271915 24417 leader_election.cc:302] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:05.274427 24602 raft_consensus.cc:2781] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:05.276024 24602 raft_consensus.cc:686] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:05.277117 24602 consensus_queue.cc:237] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.298635 24448 raft_consensus.cc:2454] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:05.299003 24492 tablet_bootstrap.cc:492] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:05.299681 24492 ts_tablet_manager.cc:1397] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.191s	user 0.012s	sys 0.007s
I20260924 14:28:05.301268 24268 leader_election.cc:302] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:05.302917 24502 raft_consensus.cc:2781] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:05.303694 24502 raft_consensus.cc:686] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:05.303301 24492 raft_consensus.cc:348] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.304666 24492 raft_consensus.cc:374] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.305455 24492 raft_consensus.cc:729] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.308715 24494 tablet_bootstrap.cc:492] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:05.306388 24492 consensus_queue.cc:260] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.309223 24494 ts_tablet_manager.cc:1397] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.081s	user 0.008s	sys 0.014s
I20260924 14:28:05.310814 24492 ts_tablet_manager.cc:1428] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260924 14:28:05.307901 24502 consensus_queue.cc:237] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.311837 24494 raft_consensus.cc:348] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.312317 24492 tablet_bootstrap.cc:492] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:05.312644 24494 raft_consensus.cc:374] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.312995 24494 raft_consensus.cc:729] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.313750 24494 consensus_queue.cc:260] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.315979 24494 ts_tablet_manager.cc:1428] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260924 14:28:05.317041 24494 tablet_bootstrap.cc:492] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:05.308715 24493 tablet_bootstrap.cc:492] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:05.318127 24493 ts_tablet_manager.cc:1397] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.191s	user 0.027s	sys 0.008s
I20260924 14:28:05.319044 24492 tablet_bootstrap.cc:654] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.326032 24493 raft_consensus.cc:348] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.327211 24493 raft_consensus.cc:374] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.327600 24493 raft_consensus.cc:729] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.328540 24493 consensus_queue.cc:260] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.331650 24493 ts_tablet_manager.cc:1428] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.002s
I20260924 14:28:05.332649 24493 tablet_bootstrap.cc:492] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:05.336040 24492 tablet_bootstrap.cc:492] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:05.336707 24492 ts_tablet_manager.cc:1397] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.004s
I20260924 14:28:05.340891 24492 raft_consensus.cc:348] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.341771 24494 tablet_bootstrap.cc:654] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.340705 24200 catalog_manager.cc:5495] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:05.341724 24492 raft_consensus.cc:374] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.342583 24492 raft_consensus.cc:729] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.343863 24492 consensus_queue.cc:260] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.346359 24492 ts_tablet_manager.cc:1428] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260924 14:28:05.347327 24492 tablet_bootstrap.cc:492] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:05.349943 24493 tablet_bootstrap.cc:654] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.356503 24492 tablet_bootstrap.cc:654] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.373287 24494 tablet_bootstrap.cc:492] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:05.373818 24494 ts_tablet_manager.cc:1397] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.057s	user 0.008s	sys 0.009s
I20260924 14:28:05.377261 24494 raft_consensus.cc:348] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.378166 24494 raft_consensus.cc:374] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.378499 24494 raft_consensus.cc:729] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.379128 24494 consensus_queue.cc:260] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.381361 24494 ts_tablet_manager.cc:1428] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:28:05.381628 24492 tablet_bootstrap.cc:492] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:05.381660 24493 tablet_bootstrap.cc:492] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:05.382120 24492 ts_tablet_manager.cc:1397] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.035s	user 0.022s	sys 0.000s
I20260924 14:28:05.382346 24493 ts_tablet_manager.cc:1397] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.005s
I20260924 14:28:05.382310 24494 tablet_bootstrap.cc:492] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:05.385028 24492 raft_consensus.cc:348] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.385895 24492 raft_consensus.cc:374] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.385664 24493 raft_consensus.cc:348] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.386256 24492 raft_consensus.cc:729] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.386473 24493 raft_consensus.cc:374] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.385776 24198 catalog_manager.cc:5495] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:05.386801 24493 raft_consensus.cc:729] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.387126 24492 consensus_queue.cc:260] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.387687 24493 consensus_queue.cc:260] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.404408 24494 tablet_bootstrap.cc:654] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.404597 24500 raft_consensus.cc:482] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.404924 24492 ts_tablet_manager.cc:1428] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.022s	user 0.015s	sys 0.004s
I20260924 14:28:05.405169 24500 raft_consensus.cc:504] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.405835 24492 tablet_bootstrap.cc:492] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:05.412432 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:05.413398 24300 raft_consensus.cc:2379] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61d316eb5dfe405c95019383bbebecaa in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:05.411103 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:05.415303 24492 tablet_bootstrap.cc:654] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.417403 24344 leader_election.cc:302] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:05.401036 24493 ts_tablet_manager.cc:1428] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.010s
I20260924 14:28:05.418841 24563 raft_consensus.cc:3037] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.419661 24493 tablet_bootstrap.cc:492] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:05.432798 24500 leader_election.cc:288] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.439774 24493 tablet_bootstrap.cc:654] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.446966 24494 tablet_bootstrap.cc:492] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:05.447360 24492 tablet_bootstrap.cc:492] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:05.447736 24494 ts_tablet_manager.cc:1397] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.016s
I20260924 14:28:05.447844 24492 ts_tablet_manager.cc:1397] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.042s	user 0.024s	sys 0.012s
I20260924 14:28:05.450328 24494 raft_consensus.cc:348] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.451012 24494 raft_consensus.cc:374] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.451294 24494 raft_consensus.cc:729] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.451195 24602 raft_consensus.cc:482] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.451174 24492 raft_consensus.cc:348] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.452095 24492 raft_consensus.cc:374] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.452530 24492 raft_consensus.cc:729] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.452054 24494 consensus_queue.cc:260] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.452082 24602 raft_consensus.cc:504] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.452775 24563 raft_consensus.cc:2726] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:05.453182 24492 consensus_queue.cc:260] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.454346 24602 leader_election.cc:288] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:05.455484 24492 ts_tablet_manager.cc:1428] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260924 14:28:05.456475 24493 tablet_bootstrap.cc:492] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:05.456578 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:05.456943 24493 ts_tablet_manager.cc:1397] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.038s	user 0.006s	sys 0.006s
I20260924 14:28:05.458220 24492 tablet_bootstrap.cc:492] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:05.458294 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:05.460333 24417 leader_election.cc:302] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:05.461886 24493 raft_consensus.cc:348] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.462245 24494 ts_tablet_manager.cc:1428] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260924 14:28:05.462579 24493 raft_consensus.cc:374] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.462883 24493 raft_consensus.cc:729] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.462952 24602 raft_consensus.cc:2726] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:05.463701 24494 tablet_bootstrap.cc:492] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:05.474087 24500 consensus_queue.cc:1035] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:05.474629 24493 consensus_queue.cc:260] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.481315 24493 ts_tablet_manager.cc:1428] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260924 14:28:05.482404 24493 tablet_bootstrap.cc:492] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:05.474128 24492 tablet_bootstrap.cc:654] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.483300 24452 raft_consensus.cc:3037] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.493403 24493 tablet_bootstrap.cc:654] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.496939 24626 raft_consensus.cc:482] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.497560 24626 raft_consensus.cc:504] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.498109 24563 consensus_queue.cc:1035] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:05.501009 24494 tablet_bootstrap.cc:654] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.505769 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6bde1e6051745469ca811e5940bd8f9" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:05.508692 24268 leader_election.cc:341] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.511907 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6bde1e6051745469ca811e5940bd8f9" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:05.515589 24268 leader_election.cc:341] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.516064 24268 leader_election.cc:302] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:05.516944 24502 raft_consensus.cc:2726] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:05.500340 24626 leader_election.cc:288] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.542317 24563 consensus_queue.cc:1035] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:05.553751 24493 tablet_bootstrap.cc:492] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:05.554235 24493 ts_tablet_manager.cc:1397] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.072s	user 0.017s	sys 0.017s
I20260924 14:28:05.556703 24493 raft_consensus.cc:348] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.557363 24493 raft_consensus.cc:374] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.557569 24493 raft_consensus.cc:729] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.558072 24493 consensus_queue.cc:260] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.558154 24626 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:05.559651 24493 ts_tablet_manager.cc:1428] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
W20260924 14:28:05.560086 24248 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:05.562386 24492 tablet_bootstrap.cc:492] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:05.562849 24492 ts_tablet_manager.cc:1397] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.105s	user 0.025s	sys 0.017s
I20260924 14:28:05.562731 24500 consensus_queue.cc:1035] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:05.566529 24493 tablet_bootstrap.cc:492] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:05.568665 24492 raft_consensus.cc:348] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.569245 24494 tablet_bootstrap.cc:492] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:05.569376 24492 raft_consensus.cc:374] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.569828 24494 ts_tablet_manager.cc:1397] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.107s	user 0.009s	sys 0.014s
I20260924 14:28:05.569725 24500 raft_consensus.cc:482] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.569746 24492 raft_consensus.cc:729] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.572590 24563 raft_consensus.cc:482] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.572712 24500 raft_consensus.cc:504] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.573259 24563 raft_consensus.cc:504] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.577826 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:05.579787 24343 leader_election.cc:341] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.581152 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:05.581383 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a8256035984c7fb87f7e9212b21dc7" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:05.582195 24301 raft_consensus.cc:2454] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:05.582218 24302 raft_consensus.cc:2454] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:05.586239 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a8256035984c7fb87f7e9212b21dc7" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:05.586949 24449 raft_consensus.cc:2454] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:05.587324 24494 raft_consensus.cc:348] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.588030 24494 raft_consensus.cc:374] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.588291 24494 raft_consensus.cc:729] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.588964 24494 consensus_queue.cc:260] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.590782 24452 raft_consensus.cc:3037] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.591391 24492 consensus_queue.cc:260] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.592299 24344 leader_election.cc:302] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:05.593634 24492 ts_tablet_manager.cc:1428] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.004s
I20260924 14:28:05.582319 24563 leader_election.cc:288] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.593506 24344 leader_election.cc:302] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:05.598760 24492 tablet_bootstrap.cc:492] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:05.601048 24494 ts_tablet_manager.cc:1428] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260924 14:28:05.602038 24494 tablet_bootstrap.cc:492] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:05.595649 24630 raft_consensus.cc:2781] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:05.603193 24626 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:05.603755 24630 raft_consensus.cc:482] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:05.604452 24630 raft_consensus.cc:3037] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.601112 24604 raft_consensus.cc:2781] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:05.607282 24604 raft_consensus.cc:482] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:05.596236 24500 leader_election.cc:288] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.607748 24604 raft_consensus.cc:3037] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.624051 24602 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:05.641294 24604 raft_consensus.cc:504] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.652287 24604 leader_election.cc:288] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.684257 24630 raft_consensus.cc:504] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.689580 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a8256035984c7fb87f7e9212b21dc7" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:05.693058 24302 raft_consensus.cc:3037] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.688190 24494 tablet_bootstrap.cc:654] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.686977 24602 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260924 14:28:05.702080 24492 tablet_bootstrap.cc:654] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.707877 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a8256035984c7fb87f7e9212b21dc7" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:05.708794 24450 raft_consensus.cc:3037] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.718029 24569 raft_consensus.cc:482] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.718930 24569 raft_consensus.cc:504] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.727135 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:05.728142 24448 raft_consensus.cc:2454] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:05.730012 24268 leader_election.cc:302] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:05.730978 24627 raft_consensus.cc:2781] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:05.731287 24569 leader_election.cc:288] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.731633 24627 raft_consensus.cc:482] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:05.747730 24678 raft_consensus.cc:482] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.740388 24627 raft_consensus.cc:3037] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.755232 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:05.756336 24377 raft_consensus.cc:2454] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:05.758378 24450 raft_consensus.cc:2454] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:05.761250 24493 tablet_bootstrap.cc:654] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.738477 24626 consensus_queue.cc:1035] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260924 14:28:05.748737 24678 raft_consensus.cc:504] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.768802 24343 leader_election.cc:302] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:05.776010 24680 raft_consensus.cc:2781] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:05.761317 24302 raft_consensus.cc:2454] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:05.776974 24680 raft_consensus.cc:686] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:05.767364 24630 leader_election.cc:288] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.778695 24680 consensus_queue.cc:237] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.797497 24633 raft_consensus.cc:482] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.798297 24633 raft_consensus.cc:504] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.801079 24678 leader_election.cc:288] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.805008 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:05.807735 24602 raft_consensus.cc:482] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.811712 24602 raft_consensus.cc:504] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.813596 24602 leader_election.cc:288] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:05.816443 24631 raft_consensus.cc:482] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:05.816920 24631 raft_consensus.cc:504] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.817238 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fb42513147f483f912eaec2c1151c50" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:05.818533 24343 leader_election.cc:341] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.818820 24631 leader_election.cc:288] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.817139 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:05.822372 24301 raft_consensus.cc:3037] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.827298 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd02146c57f9474fba0afbcc145a7a2c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:05.819309 24343 leader_election.cc:341] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.831826 24452 raft_consensus.cc:2454] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:05.842298 24268 leader_election.cc:302] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:05.849217 24569 raft_consensus.cc:2781] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:05.849689 24569 raft_consensus.cc:482] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:05.850061 24569 raft_consensus.cc:3037] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.857617 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd02146c57f9474fba0afbcc145a7a2c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:05.859978 24633 leader_election.cc:288] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:05.835292 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4aee388a7474ac1bb46b761831b809e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
W20260924 14:28:05.865084 24268 leader_election.cc:341] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:05.865768 24627 raft_consensus.cc:504] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.868609 24627 leader_election.cc:288] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.870672 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:05.871543 24449 raft_consensus.cc:3037] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:05.879117 24417 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 66628.1us
I20260924 14:28:05.914899 24494 tablet_bootstrap.cc:492] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:05.915563 24494 ts_tablet_manager.cc:1397] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.314s	user 0.015s	sys 0.007s
I20260924 14:28:05.912469 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fb42513147f483f912eaec2c1151c50" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:05.932045 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:05.933723 24377 raft_consensus.cc:3037] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.945982 24301 raft_consensus.cc:2454] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:05.952327 24494 raft_consensus.cc:348] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.959111 24494 raft_consensus.cc:374] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.959558 24494 raft_consensus.cc:729] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.960279 24494 consensus_queue.cc:260] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.953718 24569 raft_consensus.cc:504] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.962476 24569 leader_election.cc:288] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:05.957868 24449 raft_consensus.cc:2454] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:05.964689 24494 ts_tablet_manager.cc:1428] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260924 14:28:05.966146 24494 tablet_bootstrap.cc:492] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:05.972601 24494 tablet_bootstrap.cc:654] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:05.962423 24377 raft_consensus.cc:2454] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:05.977175 24493 tablet_bootstrap.cc:492] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:05.977798 24493 ts_tablet_manager.cc:1397] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.411s	user 0.015s	sys 0.005s
I20260924 14:28:05.980717 24493 raft_consensus.cc:348] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.981642 24493 raft_consensus.cc:374] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:05.981956 24493 raft_consensus.cc:729] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:05.982836 24493 consensus_queue.cc:260] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:05.985802 24493 ts_tablet_manager.cc:1428] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260924 14:28:05.986882 24493 tablet_bootstrap.cc:492] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:05.992736 24494 tablet_bootstrap.cc:492] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:05.993479 24494 ts_tablet_manager.cc:1397] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.002s
I20260924 14:28:05.995005 24492 tablet_bootstrap.cc:492] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:05.995532 24492 ts_tablet_manager.cc:1397] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.401s	user 0.008s	sys 0.024s
I20260924 14:28:05.999084 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f04a45264e7d8be03ae56ab10560" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:05.999764 24301 raft_consensus.cc:2454] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:06.000187 24268 leader_election.cc:302] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:06.000977 24569 raft_consensus.cc:2781] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
W20260924 14:28:06.001282 24418 leader_election.cc:341] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.001401 24569 raft_consensus.cc:686] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:06.002254 24569 consensus_queue.cc:237] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.007830 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd02146c57f9474fba0afbcc145a7a2c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:06.008559 24449 raft_consensus.cc:3037] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:05.997860 24492 raft_consensus.cc:348] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.013608 24492 raft_consensus.cc:374] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.013473 24418 leader_election.cc:302] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:06.014130 24492 raft_consensus.cc:729] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.014984 24687 raft_consensus.cc:2781] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:06.015573 24687 raft_consensus.cc:482] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:06.015853 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4aee388a7474ac1bb46b761831b809e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.015651 24492 consensus_queue.cc:260] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.016192 24687 raft_consensus.cc:3037] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.016574 24377 raft_consensus.cc:2454] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:06.017652 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f04a45264e7d8be03ae56ab10560" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.018323 24374 raft_consensus.cc:2454] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:06.018343 24493 tablet_bootstrap.cc:654] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.019647 24417 leader_election.cc:302] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:06.022198 24686 raft_consensus.cc:2781] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:05.997412 24494 raft_consensus.cc:348] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.022578 24686 raft_consensus.cc:482] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:06.022862 24494 raft_consensus.cc:374] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.022934 24686 raft_consensus.cc:3037] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.023192 24494 raft_consensus.cc:729] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.025492 24492 ts_tablet_manager.cc:1428] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.002s
I20260924 14:28:06.026571 24492 tablet_bootstrap.cc:492] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:06.038283 24449 raft_consensus.cc:2454] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:06.042362 24200 catalog_manager.cc:5495] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:06.045490 24268 leader_election.cc:302] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:06.046583 24627 raft_consensus.cc:2781] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:06.047341 24627 raft_consensus.cc:686] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:06.048954 24627 consensus_queue.cc:237] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.042371 24494 consensus_queue.cc:260] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.053390 24631 raft_consensus.cc:482] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.053920 24631 raft_consensus.cc:504] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:06.056998 24344 leader_election.cc:341] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.057475 24344 leader_election.cc:302] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:06.046999 24687 raft_consensus.cc:504] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.059826 24678 raft_consensus.cc:2726] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:06.060299 24681 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:06.062662 24602 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260924 14:28:06.067543 24631 leader_election.cc:288] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:06.070233 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7514385d8e4a45e196a8b870b01c1c76" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:06.071651 24268 leader_election.cc:341] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.073133 24687 leader_election.cc:288] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:06.074489 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f04a45264e7d8be03ae56ab10560" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:06.075146 24374 raft_consensus.cc:3037] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.076107 24492 tablet_bootstrap.cc:654] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.076671 24344 leader_election.cc:302] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:06.079017 24680 raft_consensus.cc:2781] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:06.079782 24680 raft_consensus.cc:686] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:06.080782 24680 consensus_queue.cc:237] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.080828 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f04a45264e7d8be03ae56ab10560" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:06.081964 24301 raft_consensus.cc:3037] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.082317 24494 ts_tablet_manager.cc:1428] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.001s
I20260924 14:28:06.083230 24494 tablet_bootstrap.cc:492] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:06.077770 24686 raft_consensus.cc:504] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.099174 24686 leader_election.cc:288] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:06.101384 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4aee388a7474ac1bb46b761831b809e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
W20260924 14:28:06.103780 24418 leader_election.cc:341] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.106309 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd02146c57f9474fba0afbcc145a7a2c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:06.106747 24374 raft_consensus.cc:2454] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:06.110270 24301 raft_consensus.cc:2454] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:06.111301 24494 tablet_bootstrap.cc:654] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.111858 24492 tablet_bootstrap.cc:492] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:06.112376 24492 ts_tablet_manager.cc:1397] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.086s	user 0.011s	sys 0.006s
I20260924 14:28:06.111997 24418 leader_election.cc:302] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:06.114560 24686 raft_consensus.cc:2781] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:06.115880 24686 raft_consensus.cc:686] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:06.148101 24492 raft_consensus.cc:348] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.152900 24492 raft_consensus.cc:374] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.153234 24492 raft_consensus.cc:729] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.153851 24492 consensus_queue.cc:260] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.156404 24492 ts_tablet_manager.cc:1428] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.002s
I20260924 14:28:06.160251 24492 tablet_bootstrap.cc:492] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:06.164644 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7514385d8e4a45e196a8b870b01c1c76" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.167225 24494 tablet_bootstrap.cc:492] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
W20260924 14:28:06.167186 24268 consensus_peers.cc:487] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:06.168124 24268 leader_election.cc:341] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:06.170987 24268 leader_election.cc:341] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.148492 24686 consensus_queue.cc:237] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.171511 24268 leader_election.cc:302] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:06.172458 24631 raft_consensus.cc:2726] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:06.172544 24493 tablet_bootstrap.cc:492] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:06.174371 24494 ts_tablet_manager.cc:1397] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.091s	user 0.023s	sys 0.008s
I20260924 14:28:06.176882 24494 raft_consensus.cc:348] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.177637 24494 raft_consensus.cc:374] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.177944 24494 raft_consensus.cc:729] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.178591 24494 consensus_queue.cc:260] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.179430 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4aee388a7474ac1bb46b761831b809e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:06.180248 24374 raft_consensus.cc:3037] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.181190 24494 ts_tablet_manager.cc:1428] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:28:06.184667 24493 ts_tablet_manager.cc:1397] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.198s	user 0.027s	sys 0.019s
I20260924 14:28:06.186785 24493 raft_consensus.cc:348] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.187582 24493 raft_consensus.cc:374] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.187839 24493 raft_consensus.cc:729] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.188508 24493 consensus_queue.cc:260] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.190224 24493 ts_tablet_manager.cc:1428] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260924 14:28:06.191018 24493 tablet_bootstrap.cc:492] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:06.198993 24493 tablet_bootstrap.cc:654] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.200256 24494 tablet_bootstrap.cc:492] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:06.210556 24198 catalog_manager.cc:5495] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:06.218674 24493 tablet_bootstrap.cc:492] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:06.219437 24493 ts_tablet_manager.cc:1397] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.000s
I20260924 14:28:06.214876 24198 catalog_manager.cc:5495] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:06.223069 24493 raft_consensus.cc:348] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.218345 24374 raft_consensus.cc:2454] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:06.223891 24493 raft_consensus.cc:374] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.224188 24493 raft_consensus.cc:729] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.224936 24493 consensus_queue.cc:260] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.229091 24493 ts_tablet_manager.cc:1428] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260924 14:28:06.229916 24492 tablet_bootstrap.cc:654] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.231796 24417 leader_election.cc:302] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:06.234081 24686 raft_consensus.cc:2781] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:06.236971 24686 raft_consensus.cc:686] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:06.238440 24686 consensus_queue.cc:237] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.242800 24602 raft_consensus.cc:482] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.243863 24492 tablet_bootstrap.cc:492] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:06.244498 24492 ts_tablet_manager.cc:1397] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.084s	user 0.014s	sys 0.007s
I20260924 14:28:06.248354 24492 raft_consensus.cc:348] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.249300 24492 raft_consensus.cc:374] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.243367 24602 raft_consensus.cc:504] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.249632 24492 raft_consensus.cc:729] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.250401 24492 consensus_queue.cc:260] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.252676 24602 leader_election.cc:288] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:06.252736 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c4bc71d12c14157982c98b4c7ca505c" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:06.253041 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c4bc71d12c14157982c98b4c7ca505c" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.253703 24302 raft_consensus.cc:2454] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
W20260924 14:28:06.254742 24417 leader_election.cc:341] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.255751 24418 leader_election.cc:302] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:06.256916 24681 raft_consensus.cc:2781] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:06.256779 24500 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:06.257274 24681 raft_consensus.cc:482] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:06.257829 24681 raft_consensus.cc:3037] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.258059 24494 tablet_bootstrap.cc:654] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.252753 24492 ts_tablet_manager.cc:1428] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:28:06.259652 24493 tablet_bootstrap.cc:492] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:06.260224 24492 tablet_bootstrap.cc:492] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:06.267367 24681 raft_consensus.cc:504] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.272133 24201 catalog_manager.cc:5495] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:06.272575 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c4bc71d12c14157982c98b4c7ca505c" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:06.276688 24301 raft_consensus.cc:3037] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.287294 24493 tablet_bootstrap.cc:654] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.290280 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c4bc71d12c14157982c98b4c7ca505c" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:06.291242 24492 tablet_bootstrap.cc:654] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:06.292202 24417 leader_election.cc:341] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.288491 24681 leader_election.cc:288] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:06.300931 24200 catalog_manager.cc:5495] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:06.301621 24301 raft_consensus.cc:2454] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:06.304790 24418 leader_election.cc:302] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:06.306522 24494 tablet_bootstrap.cc:492] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:06.306984 24494 ts_tablet_manager.cc:1397] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.107s	user 0.012s	sys 0.010s
I20260924 14:28:06.310240 24494 raft_consensus.cc:348] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.310894 24494 raft_consensus.cc:374] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.311151 24494 raft_consensus.cc:729] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.311755 24494 consensus_queue.cc:260] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.306665 24681 raft_consensus.cc:2781] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:06.313076 24681 raft_consensus.cc:686] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:06.315412 24681 consensus_queue.cc:237] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.314607 24494 ts_tablet_manager.cc:1428] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260924 14:28:06.318886 24494 tablet_bootstrap.cc:492] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:06.335340 24630 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260924 14:28:06.350703 24494 tablet_bootstrap.cc:654] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.349891 24198 catalog_manager.cc:5495] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:06.357010 24492 tablet_bootstrap.cc:492] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:06.357575 24492 ts_tablet_manager.cc:1397] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.098s	user 0.010s	sys 0.008s
I20260924 14:28:06.370025 24493 tablet_bootstrap.cc:492] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:06.370502 24493 ts_tablet_manager.cc:1397] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.111s	user 0.017s	sys 0.000s
I20260924 14:28:06.370522 24492 raft_consensus.cc:348] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.374795 24493 raft_consensus.cc:348] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.375308 24492 raft_consensus.cc:374] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.375480 24493 raft_consensus.cc:374] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.375625 24492 raft_consensus.cc:729] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.375764 24493 raft_consensus.cc:729] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.376749 24492 consensus_queue.cc:260] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.376749 24493 consensus_queue.cc:260] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.379624 24492 ts_tablet_manager.cc:1428] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260924 14:28:06.379122 24201 catalog_manager.cc:5495] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:06.381247 24492 tablet_bootstrap.cc:492] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:06.387603 24492 tablet_bootstrap.cc:654] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.389820 24493 ts_tablet_manager.cc:1428] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.003s
I20260924 14:28:06.404296 24493 tablet_bootstrap.cc:492] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:06.408830 24492 tablet_bootstrap.cc:492] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:06.409344 24492 ts_tablet_manager.cc:1397] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.008s
I20260924 14:28:06.409818 24494 tablet_bootstrap.cc:492] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:06.410341 24494 ts_tablet_manager.cc:1397] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.027s
I20260924 14:28:06.411320 24493 tablet_bootstrap.cc:654] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.412873 24627 raft_consensus.cc:482] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.413604 24627 raft_consensus.cc:504] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.413957 24492 raft_consensus.cc:348] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.414852 24492 raft_consensus.cc:374] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.415127 24492 raft_consensus.cc:729] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.416101 24492 consensus_queue.cc:260] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.417448 24627 leader_election.cc:288] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:06.419793 24492 ts_tablet_manager.cc:1428] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260924 14:28:06.420037 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.419836 24494 raft_consensus.cc:348] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.420951 24494 raft_consensus.cc:374] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.421239 24492 tablet_bootstrap.cc:492] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:06.421243 24494 raft_consensus.cc:729] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.422823 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:06.423918 24450 raft_consensus.cc:2454] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:06.422793 24494 consensus_queue.cc:260] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.426216 24268 leader_election.cc:302] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:06.429201 24569 raft_consensus.cc:2781] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:06.429790 24494 ts_tablet_manager.cc:1428] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.005s
I20260924 14:28:06.429689 24569 raft_consensus.cc:482] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:06.430249 24569 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.431579 24494 tablet_bootstrap.cc:492] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
W20260924 14:28:06.433125 24268 leader_election.cc:341] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.439042 24681 raft_consensus.cc:482] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.440006 24681 raft_consensus.cc:504] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.442832 24681 leader_election.cc:288] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:06.444609 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fb42513147f483f912eaec2c1151c50" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.444799 24631 consensus_queue.cc:1035] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260924 14:28:06.451473 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fb42513147f483f912eaec2c1151c50" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:06.452780 24301 raft_consensus.cc:2454] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:06.454496 24418 leader_election.cc:302] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:06.456475 24502 raft_consensus.cc:482] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.458909 24681 raft_consensus.cc:2781] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:06.458681 24502 raft_consensus.cc:504] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.458014 24374 raft_consensus.cc:2454] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:06.459384 24681 raft_consensus.cc:482] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:06.459959 24681 raft_consensus.cc:3037] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.460850 24502 leader_election.cc:288] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:06.457422 24500 raft_consensus.cc:482] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.457621 24492 tablet_bootstrap.cc:654] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.463970 24502 consensus_queue.cc:1035] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:06.464079 24500 raft_consensus.cc:504] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.469029 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.471668 24377 raft_consensus.cc:3037] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:06.473423 24268 leader_election.cc:341] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.482676 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:06.509958 24502 consensus_queue.cc:1035] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:06.517163 24493 tablet_bootstrap.cc:492] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:06.520996 24493 ts_tablet_manager.cc:1397] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.117s	user 0.030s	sys 0.017s
I20260924 14:28:06.516408 24631 raft_consensus.cc:482] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.521881 24494 tablet_bootstrap.cc:654] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:06.523306 24268 leader_election.cc:341] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:06.512743 24417 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 47437.3us
I20260924 14:28:06.526784 24602 raft_consensus.cc:482] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.525943 24631 raft_consensus.cc:504] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.527256 24602 raft_consensus.cc:504] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.529299 24631 leader_election.cc:288] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:06.530088 24602 leader_election.cc:288] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:06.525593 24268 leader_election.cc:302] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:06.521770 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7514385d8e4a45e196a8b870b01c1c76" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:06.515880 24569 raft_consensus.cc:504] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.536165 24302 raft_consensus.cc:2454] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:06.534337 24631 raft_consensus.cc:2726] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:06.540038 24680 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:06.543718 24681 raft_consensus.cc:504] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.539625 24500 leader_election.cc:288] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:06.546936 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4aee388a7474ac1bb46b761831b809e" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.547284 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4aee388a7474ac1bb46b761831b809e" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:06.549045 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7514385d8e4a45e196a8b870b01c1c76" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:06.550091 24301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7514385d8e4a45e196a8b870b01c1c76" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:06.556152 24493 raft_consensus.cc:348] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.560328 24493 raft_consensus.cc:374] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.560663 24493 raft_consensus.cc:729] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.561480 24493 consensus_queue.cc:260] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.556954 24681 leader_election.cc:288] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:06.557942 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fb42513147f483f912eaec2c1151c50" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:06.565539 24302 raft_consensus.cc:3037] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.567203 24374 raft_consensus.cc:2379] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cafc2c5ad4df4224a5ccc4893de70144 in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:06.567080 24569 leader_election.cc:288] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:06.568324 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:06.571285 24301 raft_consensus.cc:2454] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:06.569331 24630 raft_consensus.cc:482] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.575702 24630 raft_consensus.cc:504] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.570436 24449 raft_consensus.cc:2454] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:06.577081 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7514385d8e4a45e196a8b870b01c1c76" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.577754 24374 raft_consensus.cc:2454] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:06.578616 24630 leader_election.cc:288] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:06.578687 24373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fb42513147f483f912eaec2c1151c50" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:06.572455 24344 leader_election.cc:302] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:06.579382 24373 raft_consensus.cc:3037] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.580897 24493 ts_tablet_manager.cc:1428] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.060s	user 0.004s	sys 0.005s
I20260924 14:28:06.581792 24493 tablet_bootstrap.cc:492] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:06.568982 24451 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.584430 24680 raft_consensus.cc:2781] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:06.587734 24417 leader_election.cc:302] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:06.587208 24680 raft_consensus.cc:482] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:06.588513 24602 raft_consensus.cc:2781] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:06.588923 24680 raft_consensus.cc:3037] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.588958 24602 raft_consensus.cc:482] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:06.590022 24602 raft_consensus.cc:3037] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.621524 24631 consensus_queue.cc:1035] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:06.640522 24268 leader_election.cc:302] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:06.647176 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd02146c57f9474fba0afbcc145a7a2c" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:06.648670 24631 raft_consensus.cc:3037] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.629027 24373 raft_consensus.cc:2454] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:06.658715 24494 tablet_bootstrap.cc:492] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:06.658538 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:06.659245 24633 raft_consensus.cc:482] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.661376 24494 ts_tablet_manager.cc:1397] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.230s	user 0.009s	sys 0.045s
I20260924 14:28:06.666631 24633 raft_consensus.cc:504] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.667893 24417 leader_election.cc:302] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:06.668798 24688 raft_consensus.cc:2781] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:06.669318 24688 raft_consensus.cc:686] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:06.669641 24633 leader_election.cc:288] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:06.670149 24688 consensus_queue.cc:237] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.670835 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd02146c57f9474fba0afbcc145a7a2c" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:06.670655 24602 raft_consensus.cc:504] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.681006 24451 raft_consensus.cc:2454] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:06.681869 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:06.687820 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7514385d8e4a45e196a8b870b01c1c76" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:06.688496 24300 raft_consensus.cc:3037] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.683730 24302 raft_consensus.cc:2454] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:06.684959 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.686903 24602 leader_election.cc:288] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:06.690379 24686 consensus_queue.cc:1035] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:06.695230 24630 raft_consensus.cc:482] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.695879 24418 leader_election.cc:302] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:06.695820 24630 raft_consensus.cc:504] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.697650 24686 raft_consensus.cc:2726] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:06.700067 24630 leader_election.cc:288] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:06.702039 24268 leader_election.cc:302] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:06.703228 24569 raft_consensus.cc:2781] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:06.703719 24569 raft_consensus.cc:686] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:06.704523 24569 consensus_queue.cc:237] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.705062 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6bde1e6051745469ca811e5940bd8f9" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:06.706343 24248 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260924 14:28:06.709167 24343 leader_election.cc:341] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:06.720458 24268 leader_election.cc:341] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.721067 24678 raft_consensus.cc:482] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.721570 24678 raft_consensus.cc:504] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.723282 24631 raft_consensus.cc:2726] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:06.732031 24450 raft_consensus.cc:3037] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:06.735684 24678 leader_election.cc:288] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:06.730449 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7514385d8e4a45e196a8b870b01c1c76" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:06.737156 24686 consensus_queue.cc:1035] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:06.741859 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b62b316cc05443990141d24402b1f4a" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:06.746673 24300 raft_consensus.cc:2454] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
W20260924 14:28:06.750413 24343 leader_election.cc:341] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.755875 24494 raft_consensus.cc:348] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.756753 24494 raft_consensus.cc:374] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.757077 24494 raft_consensus.cc:729] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.759804 24492 tablet_bootstrap.cc:492] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:06.760254 24492 ts_tablet_manager.cc:1397] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.339s	user 0.044s	sys 0.012s
I20260924 14:28:06.758301 24494 consensus_queue.cc:260] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.764441 24493 tablet_bootstrap.cc:654] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:06.778752 24418 leader_election.cc:302] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:06.789757 24687 raft_consensus.cc:2781] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:06.790548 24687 raft_consensus.cc:686] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:06.789429 24344 leader_election.cc:302] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:06.764426 24681 consensus_queue.cc:1035] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260924 14:28:06.800220 24627 raft_consensus.cc:482] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:06.802796 24492 raft_consensus.cc:348] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.803784 24492 raft_consensus.cc:374] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:06.804132 24492 raft_consensus.cc:729] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:06.804973 24492 consensus_queue.cc:260] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.792918 24630 raft_consensus.cc:2726] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:28:06.809494 24494 ts_tablet_manager.cc:1428] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.147s	user 0.004s	sys 0.005s
I20260924 14:28:06.810606 24494 tablet_bootstrap.cc:492] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
W20260924 14:28:06.767683 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 45002us
W20260924 14:28:06.818951 24725 log.cc:924] Time spent T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260924 14:28:06.800968 24627 raft_consensus.cc:504] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.792196 24687 consensus_queue.cc:237] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.764555 24680 raft_consensus.cc:504] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:06.829968 24492 ts_tablet_manager.cc:1428] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.016s
I20260924 14:28:06.831437 24492 tablet_bootstrap.cc:492] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:06.832636 24201 catalog_manager.cc:5495] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:06.847409 24377 raft_consensus.cc:2379] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate 61d316eb5dfe405c95019383bbebecaa in this term.
I20260924 14:28:06.855396 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:06.846015 24680 leader_election.cc:288] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:06.853278 24718 log.cc:924] Time spent T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa: Append to log took a long time: real 0.106s	user 0.003s	sys 0.003s
I20260924 14:28:06.863897 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:06.853690 24627 leader_election.cc:288] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:06.859999 24299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b62b316cc05443990141d24402b1f4a" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:06.850139 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6bde1e6051745469ca811e5940bd8f9" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:06.881994 24500 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.062s
W20260924 14:28:06.899636 24268 leader_election.cc:341] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:06.989477 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 92538.8us
W20260924 14:28:06.992158 24344 leader_election.cc:341] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:06.876776 24302 raft_consensus.cc:2454] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:06.998306 24494 tablet_bootstrap.cc:654] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:07.003387 24417 consensus_peers.cc:487] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:06.969123 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7514385d8e4a45e196a8b870b01c1c76" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:06.992636 24344 leader_election.cc:302] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:06.967739 24296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7514385d8e4a45e196a8b870b01c1c76" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:07.028905 24633 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.031812 24726 raft_consensus.cc:2726] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.032871 24296 raft_consensus.cc:2379] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 61d316eb5dfe405c95019383bbebecaa in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:07.029393 24633 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.036689 24633 leader_election.cc:288] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:07.038897 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:07.040619 24417 leader_election.cc:341] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.022434 24688 consensus_queue.cc:1035] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
W20260924 14:28:07.027758 24268 leader_election.cc:341] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.052273 24492 tablet_bootstrap.cc:654] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.057386 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:07.062662 24268 leader_election.cc:302] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
W20260924 14:28:07.041833 24344 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 49639.5us
W20260924 14:28:07.068488 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 44473.7us
I20260924 14:28:07.075522 24494 tablet_bootstrap.cc:492] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:07.075901 24727 raft_consensus.cc:2726] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.076140 24494 ts_tablet_manager.cc:1397] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.266s	user 0.011s	sys 0.023s
W20260924 14:28:07.068188 24418 leader_election.cc:341] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.077225 24418 leader_election.cc:302] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:07.078208 24633 raft_consensus.cc:2726] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.079123 24494 raft_consensus.cc:348] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.080058 24494 raft_consensus.cc:374] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.080363 24494 raft_consensus.cc:729] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.079022 24688 consensus_queue.cc:1035] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260924 14:28:07.081085 24494 consensus_queue.cc:260] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.083376 24494 ts_tablet_manager.cc:1428] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260924 14:28:07.084267 24494 tablet_bootstrap.cc:492] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:07.099466 24493 tablet_bootstrap.cc:492] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:07.100136 24493 ts_tablet_manager.cc:1397] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.519s	user 0.012s	sys 0.023s
I20260924 14:28:07.103607 24344 leader_election.cc:302] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:07.107406 24726 raft_consensus.cc:2781] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:07.107987 24726 raft_consensus.cc:482] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:07.108527 24726 raft_consensus.cc:3037] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:07.107430 24494 tablet_bootstrap.cc:654] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.120291 24493 raft_consensus.cc:348] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.121105 24493 raft_consensus.cc:374] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.120733 24344 leader_election.cc:302] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:07.121567 24493 raft_consensus.cc:729] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.122359 24493 consensus_queue.cc:260] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.123095 24630 raft_consensus.cc:2726] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.127568 24493 ts_tablet_manager.cc:1428] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.004s
I20260924 14:28:07.128708 24493 tablet_bootstrap.cc:492] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:07.129601 24726 raft_consensus.cc:504] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.141345 24492 tablet_bootstrap.cc:492] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:07.143996 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6bde1e6051745469ca811e5940bd8f9" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:07.145666 24302 raft_consensus.cc:3037] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:07.147318 24494 tablet_bootstrap.cc:492] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:07.147802 24494 ts_tablet_manager.cc:1397] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.003s
I20260924 14:28:07.144038 24726 leader_election.cc:288] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.144697 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6bde1e6051745469ca811e5940bd8f9" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
W20260924 14:28:07.150648 24343 leader_election.cc:341] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.151368 24494 raft_consensus.cc:348] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.152297 24494 raft_consensus.cc:374] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.152653 24494 raft_consensus.cc:729] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.153491 24494 consensus_queue.cc:260] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.154711 24493 tablet_bootstrap.cc:654] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.155794 24492 ts_tablet_manager.cc:1397] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.325s	user 0.019s	sys 0.003s
I20260924 14:28:07.155910 24494 ts_tablet_manager.cc:1428] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:28:07.156863 24494 tablet_bootstrap.cc:492] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:07.158110 24492 raft_consensus.cc:348] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.158806 24492 raft_consensus.cc:374] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.158350 24200 catalog_manager.cc:5495] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:07.159135 24492 raft_consensus.cc:729] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
W20260924 14:28:07.163341 24268 consensus_peers.cc:487] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:07.164287 24492 consensus_queue.cc:260] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.165356 24494 tablet_bootstrap.cc:654] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.167369 24492 ts_tablet_manager.cc:1428] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260924 14:28:07.170344 24492 tablet_bootstrap.cc:492] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:07.159014 24302 raft_consensus.cc:2454] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:07.180585 24344 leader_election.cc:302] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:07.181573 24680 raft_consensus.cc:2781] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:07.182181 24680 raft_consensus.cc:686] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:07.183266 24680 consensus_queue.cc:237] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.186789 24492 tablet_bootstrap.cc:654] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.197544 24198 catalog_manager.cc:5495] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:07.216014 24602 raft_consensus.cc:482] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.220072 24493 tablet_bootstrap.cc:492] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:07.220729 24493 ts_tablet_manager.cc:1397] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.092s	user 0.005s	sys 0.013s
I20260924 14:28:07.226835 24493 raft_consensus.cc:348] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.227576 24493 raft_consensus.cc:374] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.227829 24493 raft_consensus.cc:729] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.216701 24602 raft_consensus.cc:504] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.245105 24627 raft_consensus.cc:482] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.245615 24627 raft_consensus.cc:504] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.246876 24602 leader_election.cc:288] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:07.247874 24627 leader_election.cc:288] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.248952 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b3f9be3c4846a8b75e5e42099e6dbc" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:07.236698 24492 tablet_bootstrap.cc:492] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:07.249922 24492 ts_tablet_manager.cc:1397] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.080s	user 0.029s	sys 0.018s
W20260924 14:28:07.250727 24418 leader_election.cc:341] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.251746 24493 consensus_queue.cc:260] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.253340 24492 raft_consensus.cc:348] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.253952 24493 ts_tablet_manager.cc:1428] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260924 14:28:07.254074 24492 raft_consensus.cc:374] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.254343 24492 raft_consensus.cc:729] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.241619 24494 tablet_bootstrap.cc:492] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:07.254786 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b3f9be3c4846a8b75e5e42099e6dbc" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:07.255064 24493 tablet_bootstrap.cc:492] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:07.255395 24494 ts_tablet_manager.cc:1397] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.099s	user 0.014s	sys 0.036s
I20260924 14:28:07.255088 24492 consensus_queue.cc:260] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.253568 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:07.257597 24492 ts_tablet_manager.cc:1428] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
W20260924 14:28:07.257711 24268 leader_election.cc:341] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.258236 24494 raft_consensus.cc:348] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.258846 24492 tablet_bootstrap.cc:492] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:07.258929 24494 raft_consensus.cc:374] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.259291 24494 raft_consensus.cc:729] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
W20260924 14:28:07.259826 24417 leader_election.cc:341] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.260232 24417 leader_election.cc:302] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:07.260043 24494 consensus_queue.cc:260] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.260304 24602 consensus_queue.cc:1035] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:07.261260 24681 raft_consensus.cc:2726] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.244093 24200 catalog_manager.cc:5495] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:07.272667 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:07.275763 24494 ts_tablet_manager.cc:1428] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260924 14:28:07.276702 24494 tablet_bootstrap.cc:492] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
W20260924 14:28:07.275310 24268 leader_election.cc:341] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.278424 24268 leader_election.cc:302] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:07.280762 24492 tablet_bootstrap.cc:654] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.282413 24627 raft_consensus.cc:2726] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.285451 24493 tablet_bootstrap.cc:654] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.291488 24602 consensus_queue.cc:1035] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:07.300220 24494 tablet_bootstrap.cc:654] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.297132 24602 consensus_queue.cc:1035] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260924 14:28:07.303704 24627 consensus_queue.cc:1035] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:07.349680 24492 tablet_bootstrap.cc:492] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:07.350147 24492 ts_tablet_manager.cc:1397] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.091s	user 0.013s	sys 0.016s
I20260924 14:28:07.360340 24492 raft_consensus.cc:348] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.361114 24492 raft_consensus.cc:374] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.361398 24492 raft_consensus.cc:729] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.362015 24492 consensus_queue.cc:260] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.364640 24492 ts_tablet_manager.cc:1428] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.014s	user 0.000s	sys 0.008s
I20260924 14:28:07.365532 24492 tablet_bootstrap.cc:492] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:07.372020 24492 tablet_bootstrap.cc:654] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.377308 24687 raft_consensus.cc:482] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.377803 24687 raft_consensus.cc:504] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.379164 24493 tablet_bootstrap.cc:492] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:07.379633 24493 ts_tablet_manager.cc:1397] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.125s	user 0.012s	sys 0.021s
I20260924 14:28:07.379922 24687 leader_election.cc:288] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:07.382377 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fefeab5a75749a198aa72b89e7e2a53" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:07.383963 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fefeab5a75749a198aa72b89e7e2a53" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:07.386794 24418 leader_election.cc:341] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.390928 24378 raft_consensus.cc:3037] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:07.388782 24569 raft_consensus.cc:482] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.392033 24569 raft_consensus.cc:504] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.400271 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b55145a82545d9b769ea9d04bf1d99" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:07.402011 24268 leader_election.cc:341] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:07.402544 24417 leader_election.cc:341] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.402969 24417 leader_election.cc:302] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:07.400377 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b55145a82545d9b769ea9d04bf1d99" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:07.404461 24268 leader_election.cc:341] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.404851 24268 leader_election.cc:302] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:07.405055 24602 raft_consensus.cc:2726] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.405596 24631 raft_consensus.cc:2726] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.398447 24493 raft_consensus.cc:348] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.406674 24493 raft_consensus.cc:374] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.406209 24569 leader_election.cc:288] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.408900 24687 consensus_queue.cc:1035] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:07.408833 24493 raft_consensus.cc:729] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.413160 24493 consensus_queue.cc:260] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.416522 24493 ts_tablet_manager.cc:1428] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.003s
I20260924 14:28:07.417750 24493 tablet_bootstrap.cc:492] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:07.455178 24492 tablet_bootstrap.cc:492] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:07.455811 24492 ts_tablet_manager.cc:1397] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.090s	user 0.025s	sys 0.031s
I20260924 14:28:07.467491 24492 raft_consensus.cc:348] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.469745 24492 raft_consensus.cc:374] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.470103 24492 raft_consensus.cc:729] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.478128 24492 consensus_queue.cc:260] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.479905 24687 raft_consensus.cc:482] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.480389 24687 raft_consensus.cc:504] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.481138 24492 ts_tablet_manager.cc:1428] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260924 14:28:07.482335 24687 leader_election.cc:288] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:07.482401 24492 tablet_bootstrap.cc:492] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:07.485044 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
W20260924 14:28:07.487038 24418 leader_election.cc:341] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.488255 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:07.489624 24417 leader_election.cc:341] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.490015 24417 leader_election.cc:302] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:07.491006 24602 raft_consensus.cc:2726] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.495677 24494 tablet_bootstrap.cc:492] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:07.496165 24494 ts_tablet_manager.cc:1397] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.220s	user 0.023s	sys 0.023s
I20260924 14:28:07.498158 24492 tablet_bootstrap.cc:654] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.498519 24494 raft_consensus.cc:348] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.499228 24494 raft_consensus.cc:374] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.499511 24494 raft_consensus.cc:729] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.500090 24494 consensus_queue.cc:260] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.502218 24493 tablet_bootstrap.cc:654] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.507253 24494 ts_tablet_manager.cc:1428] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260924 14:28:07.508199 24494 tablet_bootstrap.cc:492] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:07.517639 24494 tablet_bootstrap.cc:654] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.536702 24680 raft_consensus.cc:482] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.537312 24680 raft_consensus.cc:504] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.540853 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df40ee44e21496dbc7ef5dd5e624432" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:07.541179 24493 tablet_bootstrap.cc:492] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:07.541723 24493 ts_tablet_manager.cc:1397] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.124s	user 0.013s	sys 0.019s
I20260924 14:28:07.541556 24302 raft_consensus.cc:2454] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:07.542745 24344 leader_election.cc:302] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:07.543521 24726 raft_consensus.cc:2781] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:07.543932 24492 tablet_bootstrap.cc:492] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:07.543895 24726 raft_consensus.cc:482] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:07.544435 24492 ts_tablet_manager.cc:1397] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.062s	user 0.016s	sys 0.027s
I20260924 14:28:07.544404 24726 raft_consensus.cc:3037] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:07.544729 24493 raft_consensus.cc:348] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.545333 24680 leader_election.cc:288] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.545428 24493 raft_consensus.cc:374] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.545841 24493 raft_consensus.cc:729] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.546875 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df40ee44e21496dbc7ef5dd5e624432" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:07.549872 24343 leader_election.cc:341] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.551975 24726 raft_consensus.cc:504] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.555795 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df40ee44e21496dbc7ef5dd5e624432" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:07.555891 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df40ee44e21496dbc7ef5dd5e624432" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:07.556610 24302 raft_consensus.cc:3037] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:07.557145 24343 leader_election.cc:341] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.559307 24492 raft_consensus.cc:348] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.560074 24492 raft_consensus.cc:374] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.560322 24492 raft_consensus.cc:729] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.561038 24602 raft_consensus.cc:482] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.561590 24602 raft_consensus.cc:504] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.564870 24602 leader_election.cc:288] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:07.564296 24493 consensus_queue.cc:260] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.565986 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:07.566644 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:07.567435 24493 ts_tablet_manager.cc:1428] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
W20260924 14:28:07.567320 24417 leader_election.cc:341] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:07.567982 24418 leader_election.cc:341] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.568284 24302 raft_consensus.cc:2454] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:07.568545 24493 tablet_bootstrap.cc:492] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:07.568398 24418 leader_election.cc:302] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:07.569466 24344 leader_election.cc:302] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:07.569224 24492 consensus_queue.cc:260] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.569732 24602 raft_consensus.cc:2726] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.562959 24726 leader_election.cc:288] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.571188 24726 raft_consensus.cc:2781] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:07.571316 24492 ts_tablet_manager.cc:1428] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260924 14:28:07.573256 24492 tablet_bootstrap.cc:492] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:07.574551 24726 raft_consensus.cc:686] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:07.576253 24726 consensus_queue.cc:237] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.582126 24492 tablet_bootstrap.cc:654] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.585157 24494 tablet_bootstrap.cc:492] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:07.585453 24493 tablet_bootstrap.cc:654] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.585608 24494 ts_tablet_manager.cc:1397] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.078s	user 0.010s	sys 0.007s
I20260924 14:28:07.588406 24494 raft_consensus.cc:348] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.589263 24494 raft_consensus.cc:374] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.589603 24494 raft_consensus.cc:729] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.590564 24494 consensus_queue.cc:260] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.597663 24492 tablet_bootstrap.cc:492] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:07.598184 24492 ts_tablet_manager.cc:1397] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.004s
I20260924 14:28:07.600610 24492 raft_consensus.cc:348] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.601432 24492 raft_consensus.cc:374] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.601470 24494 ts_tablet_manager.cc:1428] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260924 14:28:07.601732 24492 raft_consensus.cc:729] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.602551 24494 tablet_bootstrap.cc:492] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:07.601820 24201 catalog_manager.cc:5495] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:07.602552 24492 consensus_queue.cc:260] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.604526 24492 ts_tablet_manager.cc:1428] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:28:07.605746 24492 tablet_bootstrap.cc:492] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:07.614779 24492 tablet_bootstrap.cc:654] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.636227 24687 raft_consensus.cc:482] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.637025 24687 raft_consensus.cc:504] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.640887 24687 leader_election.cc:288] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:07.641858 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:07.644105 24417 leader_election.cc:341] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.654305 24631 raft_consensus.cc:482] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.654843 24631 raft_consensus.cc:504] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.658553 24631 leader_election.cc:288] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.660426 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b62b316cc05443990141d24402b1f4a" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:07.662237 24268 leader_election.cc:341] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.676343 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:07.677605 24492 tablet_bootstrap.cc:492] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:07.678073 24492 ts_tablet_manager.cc:1397] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.072s	user 0.022s	sys 0.018s
W20260924 14:28:07.677798 24418 leader_election.cc:341] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.678467 24418 leader_election.cc:302] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:07.679829 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b62b316cc05443990141d24402b1f4a" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:07.680068 24687 raft_consensus.cc:2726] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.680498 24378 raft_consensus.cc:2454] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:07.681720 24268 leader_election.cc:302] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:07.683517 24493 tablet_bootstrap.cc:492] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:07.684027 24493 ts_tablet_manager.cc:1397] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.116s	user 0.018s	sys 0.005s
I20260924 14:28:07.686203 24492 raft_consensus.cc:348] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.689484 24492 raft_consensus.cc:374] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.689864 24492 raft_consensus.cc:729] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.690712 24492 consensus_queue.cc:260] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.694232 24492 ts_tablet_manager.cc:1428] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260924 14:28:07.695302 24492 tablet_bootstrap.cc:492] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:07.686384 24493 raft_consensus.cc:348] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.689133 24494 tablet_bootstrap.cc:654] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.701495 24493 raft_consensus.cc:374] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.702095 24492 tablet_bootstrap.cc:654] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.702178 24493 raft_consensus.cc:729] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.699214 24631 raft_consensus.cc:2781] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:07.704572 24631 raft_consensus.cc:482] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:07.702896 24493 consensus_queue.cc:260] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.704943 24631 raft_consensus.cc:3037] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:07.708770 24500 consensus_queue.cc:1035] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:07.712585 24493 ts_tablet_manager.cc:1428] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.001s
I20260924 14:28:07.713534 24493 tablet_bootstrap.cc:492] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:07.714336 24492 tablet_bootstrap.cc:492] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
W20260924 14:28:07.715975 24343 consensus_peers.cc:487] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:07.714879 24492 ts_tablet_manager.cc:1397] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.001s
I20260924 14:28:07.720739 24378 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:07.715024 24631 raft_consensus.cc:504] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.746930 24492 raft_consensus.cc:348] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.747671 24492 raft_consensus.cc:374] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.747902 24492 raft_consensus.cc:729] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.748426 24492 consensus_queue.cc:260] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.750228 24727 raft_consensus.cc:482] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.750751 24727 raft_consensus.cc:504] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.753139 24727 leader_election.cc:288] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.754801 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:07.755685 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:07.756520 24268 leader_election.cc:341] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:07.758126 24268 leader_election.cc:341] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.758554 24268 leader_election.cc:302] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:07.760049 24492 ts_tablet_manager.cc:1428] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260924 14:28:07.760792 24492 tablet_bootstrap.cc:492] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:07.759827 24727 raft_consensus.cc:2726] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.765830 24687 raft_consensus.cc:482] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.766309 24687 raft_consensus.cc:504] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.768199 24687 leader_election.cc:288] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:07.768016 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b62b316cc05443990141d24402b1f4a" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:07.770515 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5caac6840188451a8e325a37b8ea0e3c" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:07.770947 24449 raft_consensus.cc:3037] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:07.771289 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5caac6840188451a8e325a37b8ea0e3c" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:07.771713 24418 leader_election.cc:341] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.769732 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b62b316cc05443990141d24402b1f4a" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:07.797850 24376 raft_consensus.cc:3037] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:07.801496 24494 tablet_bootstrap.cc:492] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
W20260924 14:28:07.772755 24417 leader_election.cc:341] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.801955 24417 leader_election.cc:302] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:07.802737 24686 raft_consensus.cc:2726] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:07.801980 24494 ts_tablet_manager.cc:1397] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.200s	user 0.031s	sys 0.032s
I20260924 14:28:07.803294 24492 tablet_bootstrap.cc:654] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.806721 24494 raft_consensus.cc:348] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.807478 24494 raft_consensus.cc:374] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.807746 24494 raft_consensus.cc:729] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.796114 24449 raft_consensus.cc:2454] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:07.774880 24631 leader_election.cc:288] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.800212 24493 tablet_bootstrap.cc:654] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.808519 24494 consensus_queue.cc:260] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.819999 24494 ts_tablet_manager.cc:1428] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260924 14:28:07.822580 24268 leader_election.cc:302] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:07.823499 24494 tablet_bootstrap.cc:492] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:07.826331 24569 raft_consensus.cc:2781] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:07.827116 24569 raft_consensus.cc:686] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:07.831727 24569 consensus_queue.cc:237] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.835371 24376 raft_consensus.cc:2454] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:07.838141 24687 raft_consensus.cc:482] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.838675 24687 raft_consensus.cc:504] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.840675 24687 leader_election.cc:288] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:07.841604 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df40ee44e21496dbc7ef5dd5e624432" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:07.842377 24492 tablet_bootstrap.cc:492] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:07.844624 24492 ts_tablet_manager.cc:1397] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.084s	user 0.010s	sys 0.004s
I20260924 14:28:07.845086 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df40ee44e21496dbc7ef5dd5e624432" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:07.858316 24248 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:07.961011 24199 catalog_manager.cc:5495] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:07.956631 24492 raft_consensus.cc:348] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.842696 24298 raft_consensus.cc:2379] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate 61d316eb5dfe405c95019383bbebecaa in this term.
I20260924 14:28:07.963843 24492 raft_consensus.cc:374] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.964180 24492 raft_consensus.cc:729] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.964840 24492 consensus_queue.cc:260] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.966421 24492 ts_tablet_manager.cc:1428] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.000s
I20260924 14:28:07.967348 24492 tablet_bootstrap.cc:492] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:07.972036 24493 tablet_bootstrap.cc:492] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:07.972627 24493 ts_tablet_manager.cc:1397] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.259s	user 0.017s	sys 0.024s
I20260924 14:28:07.897533 24726 raft_consensus.cc:482] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.972019 24500 raft_consensus.cc:482] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.978780 24500 raft_consensus.cc:504] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.973524 24631 raft_consensus.cc:482] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:07.980146 24493 raft_consensus.cc:348] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.980711 24493 raft_consensus.cc:374] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:07.981001 24493 raft_consensus.cc:729] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:07.981664 24493 consensus_queue.cc:260] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.984616 24493 ts_tablet_manager.cc:1428] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260924 14:28:07.985857 24493 tablet_bootstrap.cc:492] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:07.981843 24631 raft_consensus.cc:504] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:07.988317 24631 leader_election.cc:288] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.990576 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:07.991276 24500 leader_election.cc:288] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.975318 24726 raft_consensus.cc:504] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:07.992028 24268 leader_election.cc:341] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:07.992937 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:07.993474 24726 leader_election.cc:288] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:07.993546 24449 raft_consensus.cc:2454] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:07.994490 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:07.995347 24494 tablet_bootstrap.cc:654] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:07.995854 24268 leader_election.cc:302] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:07.995901 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:07.997570 24300 raft_consensus.cc:2454] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:07.998777 24344 leader_election.cc:302] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:07.994493 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:08.003026 24569 raft_consensus.cc:2781] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:08.011931 24493 tablet_bootstrap.cc:654] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.000947 24500 raft_consensus.cc:2781] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:08.015728 24500 raft_consensus.cc:482] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:08.016090 24500 raft_consensus.cc:3037] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.017210 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:08.023558 24418 leader_election.cc:302] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:08.026297 24686 raft_consensus.cc:3037] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.027046 24500 raft_consensus.cc:504] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.029837 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:08.030723 24500 leader_election.cc:288] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.019140 24569 raft_consensus.cc:482] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:08.031092 24302 raft_consensus.cc:3037] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.031335 24569 raft_consensus.cc:3037] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.034941 24494 tablet_bootstrap.cc:492] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:08.035387 24494 ts_tablet_manager.cc:1397] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.213s	user 0.018s	sys 0.030s
I20260924 14:28:08.062605 24494 raft_consensus.cc:348] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.064210 24494 raft_consensus.cc:374] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.064600 24494 raft_consensus.cc:729] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.067680 24492 tablet_bootstrap.cc:654] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.075093 24343 leader_election.cc:302] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:08.076403 24500 raft_consensus.cc:2726] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:08.069175 24494 consensus_queue.cc:260] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:08.077548 24343 leader_election.cc:341] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.084713 24494 ts_tablet_manager.cc:1428] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.002s
I20260924 14:28:08.086309 24494 tablet_bootstrap.cc:492] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:08.090224 24771 consensus_queue.cc:1035] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:08.074510 24686 raft_consensus.cc:2726] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:08.094743 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
W20260924 14:28:08.102535 24343 leader_election.cc:341] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.105278 24771 consensus_queue.cc:1035] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260924 14:28:08.121773 24493 tablet_bootstrap.cc:492] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:08.122416 24493 ts_tablet_manager.cc:1397] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.137s	user 0.033s	sys 0.017s
I20260924 14:28:08.125828 24493 raft_consensus.cc:348] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.126657 24493 raft_consensus.cc:374] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.126974 24493 raft_consensus.cc:729] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.134150 24494 tablet_bootstrap.cc:654] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.127714 24493 consensus_queue.cc:260] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.136293 24302 raft_consensus.cc:2454] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:08.142920 24344 leader_election.cc:302] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:08.122586 24569 raft_consensus.cc:504] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.145963 24493 ts_tablet_manager.cc:1428] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260924 14:28:08.145130 24727 raft_consensus.cc:482] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.152824 24727 raft_consensus.cc:504] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.154688 24727 leader_election.cc:288] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.145665 24726 raft_consensus.cc:2781] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:08.158789 24726 raft_consensus.cc:686] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:08.152217 24493 tablet_bootstrap.cc:492] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:08.158510 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:08.159718 24726 consensus_queue.cc:237] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.163873 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:08.166159 24268 leader_election.cc:341] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:08.166975 24268 leader_election.cc:341] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.167425 24268 leader_election.cc:302] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:08.168571 24727 raft_consensus.cc:2726] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:08.180361 24492 tablet_bootstrap.cc:492] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:08.181020 24492 ts_tablet_manager.cc:1397] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.214s	user 0.016s	sys 0.026s
I20260924 14:28:08.185045 24492 raft_consensus.cc:348] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.185751 24492 raft_consensus.cc:374] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.186088 24492 raft_consensus.cc:729] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.188992 24569 leader_election.cc:288] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.191329 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:08.191603 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:08.192231 24494 tablet_bootstrap.cc:492] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:08.192183 24450 raft_consensus.cc:3037] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.192896 24492 consensus_queue.cc:260] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.193881 24494 ts_tablet_manager.cc:1397] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.108s	user 0.016s	sys 0.004s
I20260924 14:28:08.199870 24493 tablet_bootstrap.cc:654] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.202265 24494 raft_consensus.cc:348] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.203020 24494 raft_consensus.cc:374] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.203310 24494 raft_consensus.cc:729] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.203949 24494 consensus_queue.cc:260] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.204319 24201 catalog_manager.cc:5495] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:08.196125 24492 ts_tablet_manager.cc:1428] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260924 14:28:08.206298 24492 tablet_bootstrap.cc:492] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:08.211814 24492 tablet_bootstrap.cc:654] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.214720 24494 ts_tablet_manager.cc:1428] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260924 14:28:08.215678 24494 tablet_bootstrap.cc:492] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:08.216605 24726 raft_consensus.cc:482] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.217056 24726 raft_consensus.cc:504] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.219877 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fefeab5a75749a198aa72b89e7e2a53" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
W20260924 14:28:08.221323 24344 leader_election.cc:341] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.222893 24726 leader_election.cc:288] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.236694 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fefeab5a75749a198aa72b89e7e2a53" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:08.237423 24449 raft_consensus.cc:2454] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
W20260924 14:28:08.195886 24268 leader_election.cc:341] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:08.239383 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 43533.7us
I20260924 14:28:08.244423 24569 consensus_queue.cc:1035] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:08.244971 24492 tablet_bootstrap.cc:492] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:08.245573 24492 ts_tablet_manager.cc:1397] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.001s
I20260924 14:28:08.248615 24686 raft_consensus.cc:482] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.248817 24492 raft_consensus.cc:348] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.249121 24686 raft_consensus.cc:504] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.249796 24492 raft_consensus.cc:374] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.250136 24492 raft_consensus.cc:729] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.250449 24343 leader_election.cc:302] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:08.251042 24686 leader_election.cc:288] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:08.252347 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:08.252738 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:08.253366 24302 raft_consensus.cc:2454] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
W20260924 14:28:08.253746 24417 leader_election.cc:341] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.255046 24494 tablet_bootstrap.cc:654] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.254813 24418 leader_election.cc:302] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:08.255914 24686 raft_consensus.cc:2781] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:08.256081 24680 raft_consensus.cc:2781] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:08.256388 24686 raft_consensus.cc:482] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:08.256547 24680 raft_consensus.cc:482] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:08.256911 24686 raft_consensus.cc:3037] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.256991 24680 raft_consensus.cc:3037] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.259363 24569 consensus_queue.cc:1035] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:08.260804 24492 consensus_queue.cc:260] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.264508 24686 raft_consensus.cc:504] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.267256 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:08.267833 24302 raft_consensus.cc:3037] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.267768 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:08.268565 24492 ts_tablet_manager.cc:1428] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260924 14:28:08.269418 24492 tablet_bootstrap.cc:492] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
W20260924 14:28:08.270503 24417 leader_election.cc:341] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.277962 24680 raft_consensus.cc:504] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.282207 24450 raft_consensus.cc:2454] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:08.284161 24378 raft_consensus.cc:3037] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.288504 24680 leader_election.cc:288] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.290493 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fefeab5a75749a198aa72b89e7e2a53" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
W20260924 14:28:08.291884 24344 leader_election.cc:341] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.294490 24492 tablet_bootstrap.cc:654] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.295652 24686 leader_election.cc:288] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:08.296620 24493 tablet_bootstrap.cc:492] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:08.297869 24493 ts_tablet_manager.cc:1397] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.146s	user 0.022s	sys 0.035s
I20260924 14:28:08.297870 24268 leader_election.cc:302] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:08.298780 24777 raft_consensus.cc:2781] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:08.299238 24777 raft_consensus.cc:686] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:08.300030 24777 consensus_queue.cc:237] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.301836 24493 raft_consensus.cc:348] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.314291 24302 raft_consensus.cc:2454] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:08.316604 24418 leader_election.cc:302] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:08.317683 24686 raft_consensus.cc:2781] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:08.317823 24494 tablet_bootstrap.cc:492] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:08.318307 24494 ts_tablet_manager.cc:1397] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.103s	user 0.005s	sys 0.006s
I20260924 14:28:08.318356 24686 raft_consensus.cc:686] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:08.319285 24686 consensus_queue.cc:237] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.308785 24493 raft_consensus.cc:374] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.323547 24494 raft_consensus.cc:348] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.324610 24494 raft_consensus.cc:374] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.325037 24494 raft_consensus.cc:729] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.324172 24493 raft_consensus.cc:729] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.325912 24494 consensus_queue.cc:260] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.328878 24494 ts_tablet_manager.cc:1428] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260924 14:28:08.329504 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fefeab5a75749a198aa72b89e7e2a53" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:08.330307 24494 tablet_bootstrap.cc:492] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:08.330442 24450 raft_consensus.cc:3037] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.326325 24493 consensus_queue.cc:260] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.332643 24493 ts_tablet_manager.cc:1428] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260924 14:28:08.333631 24493 tablet_bootstrap.cc:492] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:08.343123 24569 raft_consensus.cc:482] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.343794 24569 raft_consensus.cc:504] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.363878 24569 leader_election.cc:288] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.382483 24493 tablet_bootstrap.cc:654] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.385871 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:08.388047 24268 leader_election.cc:341] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.375398 24450 raft_consensus.cc:2454] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:08.404124 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:08.404556 24343 leader_election.cc:302] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:08.405311 24450 raft_consensus.cc:2454] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:08.406116 24771 raft_consensus.cc:2781] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:08.407635 24771 raft_consensus.cc:686] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:08.422726 24201 catalog_manager.cc:5495] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:08.416054 24771 consensus_queue.cc:237] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.435894 24492 tablet_bootstrap.cc:492] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:08.436319 24492 ts_tablet_manager.cc:1397] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.167s	user 0.009s	sys 0.007s
I20260924 14:28:08.440059 24268 leader_election.cc:302] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:08.442046 24777 raft_consensus.cc:2781] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:08.442818 24777 raft_consensus.cc:482] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:08.443183 24777 raft_consensus.cc:3037] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.518909 24777 raft_consensus.cc:504] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.527053 24686 raft_consensus.cc:482] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.510784 24726 raft_consensus.cc:482] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.539462 24686 raft_consensus.cc:504] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.540043 24726 raft_consensus.cc:504] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.542526 24726 leader_election.cc:288] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.531719 24727 raft_consensus.cc:482] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.543329 24727 raft_consensus.cc:504] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.545827 24727 leader_election.cc:288] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.559767 24777 leader_election.cc:288] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.618748 24492 raft_consensus.cc:348] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.620205 24492 raft_consensus.cc:374] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.620488 24492 raft_consensus.cc:729] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.549247 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b3f9be3c4846a8b75e5e42099e6dbc" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:08.621225 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252120ce7355481189a4a8c3659997ab" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:08.621860 24686 leader_election.cc:288] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:08.622004 24602 raft_consensus.cc:482] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.628108 24602 raft_consensus.cc:504] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.553125 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
W20260924 14:28:08.631757 24268 leader_election.cc:341] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.631669 24296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:08.632792 24296 raft_consensus.cc:2379] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate 61d316eb5dfe405c95019383bbebecaa in this term.
W20260924 14:28:08.633059 24268 leader_election.cc:341] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.634866 24602 leader_election.cc:288] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:08.636276 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:08.638975 24417 leader_election.cc:302] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:08.623397 24494 tablet_bootstrap.cc:654] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.623023 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252120ce7355481189a4a8c3659997ab" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:08.551298 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:08.610041 24727 raft_consensus.cc:482] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.644678 24602 raft_consensus.cc:3037] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.645128 24727 raft_consensus.cc:504] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.646572 24302 raft_consensus.cc:2454] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:08.623724 24492 consensus_queue.cc:260] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.650683 24727 leader_election.cc:288] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.650970 24418 leader_election.cc:302] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:08.652952 24492 ts_tablet_manager.cc:1428] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.216s	user 0.027s	sys 0.063s
W20260924 14:28:08.654793 24344 leader_election.cc:341] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.655813 24492 tablet_bootstrap.cc:492] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:08.658965 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252120ce7355481189a4a8c3659997ab" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:08.660126 24790 raft_consensus.cc:2781] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:08.661561 24790 raft_consensus.cc:482] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:08.663077 24790 raft_consensus.cc:3037] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.661548 24201 catalog_manager.cc:5495] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
W20260924 14:28:08.663763 24268 leader_election.cc:341] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.679967 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:08.682614 24494 tablet_bootstrap.cc:492] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:08.683095 24494 ts_tablet_manager.cc:1397] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.353s	user 0.013s	sys 0.000s
I20260924 14:28:08.685132 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252120ce7355481189a4a8c3659997ab" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:08.683776 24449 raft_consensus.cc:3037] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.685611 24199 catalog_manager.cc:5495] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:08.683830 24771 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260924 14:28:08.674041 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b3f9be3c4846a8b75e5e42099e6dbc" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:08.690821 24450 raft_consensus.cc:2454] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:08.723472 24493 tablet_bootstrap.cc:492] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:08.724011 24493 ts_tablet_manager.cc:1397] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.391s	user 0.048s	sys 0.060s
W20260924 14:28:08.684818 24417 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 46017.6us
I20260924 14:28:08.706408 24602 raft_consensus.cc:2726] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:08.734674 24493 raft_consensus.cc:348] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.739216 24493 raft_consensus.cc:374] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.739760 24493 raft_consensus.cc:729] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.727082 24771 raft_consensus.cc:482] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.763021 24494 raft_consensus.cc:348] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.763770 24494 raft_consensus.cc:374] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.761920 24493 consensus_queue.cc:260] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.764071 24494 raft_consensus.cc:729] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.767594 24494 consensus_queue.cc:260] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.770291 24493 ts_tablet_manager.cc:1428] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.010s
I20260924 14:28:08.771360 24493 tablet_bootstrap.cc:492] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
W20260924 14:28:08.753667 24417 leader_election.cc:341] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.761368 24790 raft_consensus.cc:504] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.780525 24790 leader_election.cc:288] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:08.781737 24494 ts_tablet_manager.cc:1428] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.098s	user 0.019s	sys 0.003s
I20260924 14:28:08.782125 24449 raft_consensus.cc:2454] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:08.782263 24452 raft_consensus.cc:2379] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cafc2c5ad4df4224a5ccc4893de70144 in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:08.782995 24494 tablet_bootstrap.cc:492] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:08.762717 24771 raft_consensus.cc:504] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.786636 24771 leader_election.cc:288] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.788669 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252120ce7355481189a4a8c3659997ab" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:08.789276 24302 raft_consensus.cc:3037] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.792603 24492 tablet_bootstrap.cc:654] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.804520 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252120ce7355481189a4a8c3659997ab" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:08.798064 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:08.807101 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:08.807695 24300 raft_consensus.cc:2379] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61d316eb5dfe405c95019383bbebecaa in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:08.807912 24450 raft_consensus.cc:2454] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:08.814879 24302 raft_consensus.cc:2454] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:08.816902 24418 leader_election.cc:302] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:08.818281 24602 raft_consensus.cc:2781] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:08.818758 24602 raft_consensus.cc:686] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:08.823961 24493 tablet_bootstrap.cc:654] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:08.828080 24417 leader_election.cc:341] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.830606 24790 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:08.827845 24602 consensus_queue.cc:237] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.836437 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:08.838635 24494 tablet_bootstrap.cc:654] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:08.849057 24268 leader_election.cc:302] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
W20260924 14:28:08.851437 24344 consensus_peers.cc:487] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:08.853401 24268 leader_election.cc:302] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:08.851127 24796 raft_consensus.cc:2781] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:08.855265 24268 leader_election.cc:302] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:08.856735 24797 raft_consensus.cc:2726] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:08.855516 24795 raft_consensus.cc:2781] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:08.857946 24795 raft_consensus.cc:686] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:08.854614 24796 raft_consensus.cc:482] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:08.858754 24796 raft_consensus.cc:3037] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.859045 24795 consensus_queue.cc:237] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.871158 24492 tablet_bootstrap.cc:492] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:08.871281 24493 tablet_bootstrap.cc:492] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:08.872587 24493 ts_tablet_manager.cc:1397] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.101s	user 0.017s	sys 0.005s
I20260924 14:28:08.872226 24492 ts_tablet_manager.cc:1397] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.217s	user 0.007s	sys 0.013s
I20260924 14:28:08.875969 24569 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.876468 24569 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.878168 24771 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:08.878556 24569 leader_election.cc:288] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.875240 24199 catalog_manager.cc:5495] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:08.880920 24493 raft_consensus.cc:348] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.881595 24493 raft_consensus.cc:374] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.881875 24493 raft_consensus.cc:729] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.881611 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:08.882385 24377 raft_consensus.cc:2454] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:08.882601 24493 consensus_queue.cc:260] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.884398 24268 leader_election.cc:302] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:08.885542 24569 raft_consensus.cc:2781] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:08.885893 24569 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:08.886212 24569 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.893832 24343 leader_election.cc:302] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:08.895094 24771 raft_consensus.cc:2781] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:08.895581 24771 raft_consensus.cc:482] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:08.896051 24771 raft_consensus.cc:3037] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.896003 24493 ts_tablet_manager.cc:1428] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260924 14:28:08.902892 24377 raft_consensus.cc:3037] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.906765 24343 leader_election.cc:302] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:08.910918 24493 tablet_bootstrap.cc:492] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
W20260924 14:28:08.913993 24268 consensus_peers.cc:487] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:08.920965 24794 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:08.921564 24794 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.926433 24771 raft_consensus.cc:504] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.928783 24794 leader_election.cc:288] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:08.937855 24492 raft_consensus.cc:348] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.939970 24492 raft_consensus.cc:374] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.940327 24492 raft_consensus.cc:729] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.941146 24492 consensus_queue.cc:260] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.904241 24796 raft_consensus.cc:504] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.951397 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:08.957762 24500 raft_consensus.cc:2726] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:08.950342 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:08.960289 24378 raft_consensus.cc:2454] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:08.963759 24771 leader_election.cc:288] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.964794 24417 leader_election.cc:302] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:08.970325 24796 leader_election.cc:288] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.971189 24687 raft_consensus.cc:2781] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:08.971683 24687 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:08.972080 24687 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.972275 24296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
W20260924 14:28:08.975260 24344 leader_election.cc:341] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:08.976948 24492 ts_tablet_manager.cc:1428] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.103s	user 0.006s	sys 0.001s
I20260924 14:28:08.977747 24492 tablet_bootstrap.cc:492] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:08.964675 24794 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:08.976172 24201 catalog_manager.cc:5495] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:08.938313 24569 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.980568 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b3f9be3c4846a8b75e5e42099e6dbc" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:08.984042 24378 raft_consensus.cc:3037] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:08.985170 24494 tablet_bootstrap.cc:492] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:08.985615 24494 ts_tablet_manager.cc:1397] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.203s	user 0.010s	sys 0.014s
I20260924 14:28:08.986980 24302 raft_consensus.cc:2379] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate cafc2c5ad4df4224a5ccc4893de70144 in this term.
I20260924 14:28:08.988998 24450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:08.989876 24569 leader_election.cc:288] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:08.992573 24627 consensus_queue.cc:1035] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:08.993947 24494 raft_consensus.cc:348] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.994663 24494 raft_consensus.cc:374] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:08.994935 24494 raft_consensus.cc:729] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:08.995548 24494 consensus_queue.cc:260] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:08.996197 24794 raft_consensus.cc:482] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:09.002286 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:09.002955 24451 raft_consensus.cc:3037] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.003772 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b3f9be3c4846a8b75e5e42099e6dbc" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:09.004393 24448 raft_consensus.cc:3037] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.005375 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:09.006047 24376 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.008373 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:08.996610 24794 raft_consensus.cc:504] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.010793 24794 leader_election.cc:288] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:09.021884 24494 ts_tablet_manager.cc:1428] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260924 14:28:09.023063 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870bbbde2a2f4c76bb558d8c43d7587b" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:09.027635 24494 tablet_bootstrap.cc:492] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:09.041939 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870bbbde2a2f4c76bb558d8c43d7587b" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:09.075314 24418 leader_election.cc:341] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.086874 24448 raft_consensus.cc:2454] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:09.089524 24378 raft_consensus.cc:2454] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:09.090991 24268 leader_election.cc:302] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:09.091867 24569 raft_consensus.cc:2781] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:09.092301 24569 raft_consensus.cc:686] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:09.093036 24569 consensus_queue.cc:237] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:09.098877 24417 leader_election.cc:341] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.099435 24417 leader_election.cc:302] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:09.099512 24451 raft_consensus.cc:2454] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:09.103960 24686 raft_consensus.cc:2726] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:09.107247 24343 leader_election.cc:302] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:09.119306 24687 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.112115 24680 raft_consensus.cc:2781] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:09.120427 24492 tablet_bootstrap.cc:654] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:09.120688 24450 raft_consensus.cc:2379] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cafc2c5ad4df4224a5ccc4893de70144 in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:09.120750 24680 raft_consensus.cc:686] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:09.123265 24680 consensus_queue.cc:237] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.123984 24493 tablet_bootstrap.cc:654] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:09.130798 24795 raft_consensus.cc:482] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:09.134569 24726 consensus_queue.cc:1035] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260924 14:28:09.144678 24199 catalog_manager.cc:5495] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:09.133705 24795 raft_consensus.cc:504] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.149981 24795 leader_election.cc:288] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:09.150389 24296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:09.151259 24296 raft_consensus.cc:2379] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate cafc2c5ad4df4224a5ccc4893de70144 in this term.
I20260924 14:28:09.152590 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e065e0c774647b1a9b127accaebd70a" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:09.155035 24268 leader_election.cc:341] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.153470 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e065e0c774647b1a9b127accaebd70a" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:09.160205 24268 leader_election.cc:341] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.160607 24268 leader_election.cc:302] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:09.163678 24569 raft_consensus.cc:2726] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:09.164284 24687 leader_election.cc:288] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
W20260924 14:28:09.168602 24248 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:09.171231 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:09.173079 24417 leader_election.cc:302] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:09.174921 24687 raft_consensus.cc:2726] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 14:28:09.162807 24376 raft_consensus.cc:2454] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:09.180704 24268 leader_election.cc:302] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:09.182617 24569 raft_consensus.cc:2781] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:09.183353 24569 raft_consensus.cc:686] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:09.188269 24569 consensus_queue.cc:237] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.196599 24492 tablet_bootstrap.cc:492] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:09.197118 24492 ts_tablet_manager.cc:1397] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.220s	user 0.012s	sys 0.006s
I20260924 14:28:09.197430 24494 tablet_bootstrap.cc:654] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:09.222769 24199 catalog_manager.cc:5495] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:09.230569 24492 raft_consensus.cc:348] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.231570 24492 raft_consensus.cc:374] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.231886 24492 raft_consensus.cc:729] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.232559 24492 consensus_queue.cc:260] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.234782 24492 ts_tablet_manager.cc:1428] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.005s
I20260924 14:28:09.240629 24492 tablet_bootstrap.cc:492] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:09.270798 24493 tablet_bootstrap.cc:492] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:09.271514 24493 ts_tablet_manager.cc:1397] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.361s	user 0.025s	sys 0.028s
I20260924 14:28:09.275202 24493 raft_consensus.cc:348] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.280807 24493 raft_consensus.cc:374] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.281096 24493 raft_consensus.cc:729] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.281709 24493 consensus_queue.cc:260] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.283257 24493 ts_tablet_manager.cc:1428] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260924 14:28:09.284094 24493 tablet_bootstrap.cc:492] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:09.302186 24492 tablet_bootstrap.cc:654] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:09.310916 24199 catalog_manager.cc:5495] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:09.313694 24794 raft_consensus.cc:482] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:09.319687 24794 raft_consensus.cc:504] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.322192 24794 leader_election.cc:288] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:09.323004 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:09.323664 24302 raft_consensus.cc:2454] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:09.334794 24418 leader_election.cc:302] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:09.336329 24687 raft_consensus.cc:2781] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:09.336699 24687 raft_consensus.cc:482] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:09.337033 24687 raft_consensus.cc:3037] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.338361 24631 raft_consensus.cc:482] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:09.338938 24631 raft_consensus.cc:504] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.343243 24631 leader_election.cc:288] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:09.345301 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:09.345948 24376 raft_consensus.cc:2454] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:09.347206 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b55145a82545d9b769ea9d04bf1d99" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:09.348537 24268 leader_election.cc:341] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.344726 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b55145a82545d9b769ea9d04bf1d99" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:09.348654 24493 tablet_bootstrap.cc:654] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:09.351575 24268 leader_election.cc:341] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.352022 24268 leader_election.cc:302] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:09.352706 24631 raft_consensus.cc:2726] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:09.354638 24494 tablet_bootstrap.cc:492] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:09.355101 24494 ts_tablet_manager.cc:1397] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.328s	user 0.035s	sys 0.051s
I20260924 14:28:09.358004 24494 raft_consensus.cc:348] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.358695 24494 raft_consensus.cc:374] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.358958 24494 raft_consensus.cc:729] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.359529 24494 consensus_queue.cc:260] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.375274 24494 ts_tablet_manager.cc:1428] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260924 14:28:09.386221 24494 tablet_bootstrap.cc:492] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:09.388888 24631 consensus_queue.cc:1035] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260924 14:28:09.391285 24417 consensus_peers.cc:487] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:09.394583 24795 raft_consensus.cc:482] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:09.395248 24795 raft_consensus.cc:504] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.399062 24795 leader_election.cc:288] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:09.399371 24493 tablet_bootstrap.cc:492] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:09.400079 24493 ts_tablet_manager.cc:1397] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.116s	user 0.014s	sys 0.013s
I20260924 14:28:09.403771 24494 tablet_bootstrap.cc:654] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:09.403638 24493 raft_consensus.cc:348] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.404440 24795 raft_consensus.cc:482] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:09.404629 24493 raft_consensus.cc:374] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.405045 24493 raft_consensus.cc:729] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.404942 24795 raft_consensus.cc:504] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.405929 24493 consensus_queue.cc:260] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.406885 24795 leader_election.cc:288] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:09.407778 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:09.407670 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:09.408427 24493 ts_tablet_manager.cc:1428] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:28:09.408525 24447 raft_consensus.cc:2454] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
W20260924 14:28:09.410346 24268 leader_election.cc:341] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.410604 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638ffc5970704a53a26967c81af23e52" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:09.410519 24493 tablet_bootstrap.cc:492] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:09.410959 24268 leader_election.cc:302] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:09.411481 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638ffc5970704a53a26967c81af23e52" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:09.421115 24493 tablet_bootstrap.cc:654] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:09.422279 24494 tablet_bootstrap.cc:492] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:09.422789 24494 ts_tablet_manager.cc:1397] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.007s
I20260924 14:28:09.425865 24494 raft_consensus.cc:348] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.426810 24494 raft_consensus.cc:374] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.427150 24494 raft_consensus.cc:729] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.427175 24790 consensus_queue.cc:1035] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:09.428257 24494 consensus_queue.cc:260] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.461887 24494 ts_tablet_manager.cc:1428] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.005s
I20260924 14:28:09.462031 24302 raft_consensus.cc:3037] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.463296 24494 tablet_bootstrap.cc:492] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:09.475720 24493 tablet_bootstrap.cc:492] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:09.476331 24493 ts_tablet_manager.cc:1397] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.066s	user 0.024s	sys 0.036s
I20260924 14:28:09.479378 24494 tablet_bootstrap.cc:654] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:09.480711 24493 raft_consensus.cc:348] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.481631 24493 raft_consensus.cc:374] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.481969 24493 raft_consensus.cc:729] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.482748 24493 consensus_queue.cc:260] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.485714 24493 ts_tablet_manager.cc:1428] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260924 14:28:09.491665 24494 tablet_bootstrap.cc:492] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:09.492127 24493 tablet_bootstrap.cc:492] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:09.492165 24494 ts_tablet_manager.cc:1397] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.010s
I20260924 14:28:09.497905 24493 tablet_bootstrap.cc:654] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:09.510205 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 99296.2us
I20260924 14:28:09.511247 24795 raft_consensus.cc:2781] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260924 14:28:09.511768 24268 leader_election.cc:341] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.512029 24795 raft_consensus.cc:482] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:28:09.512444 24268 leader_election.cc:341] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.512854 24268 leader_election.cc:302] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:09.514350 24569 raft_consensus.cc:2726] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:09.453859 24687 raft_consensus.cc:504] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.515005 24493 tablet_bootstrap.cc:492] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:09.515547 24493 ts_tablet_manager.cc:1397] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.005s
I20260924 14:28:09.516208 24687 leader_election.cc:288] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:09.518729 24493 raft_consensus.cc:348] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.519681 24493 raft_consensus.cc:374] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.520000 24493 raft_consensus.cc:729] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.520776 24493 consensus_queue.cc:260] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.523197 24493 ts_tablet_manager.cc:1428] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:28:09.524333 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:09.525014 24378 raft_consensus.cc:3037] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.512688 24795 raft_consensus.cc:3037] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.541501 24493 tablet_bootstrap.cc:492] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:09.548194 24494 raft_consensus.cc:348] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.549211 24494 raft_consensus.cc:374] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.549561 24494 raft_consensus.cc:729] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.550321 24494 consensus_queue.cc:260] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.568503 24771 raft_consensus.cc:482] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:09.568847 24494 ts_tablet_manager.cc:1428] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.076s	user 0.006s	sys 0.000s
I20260924 14:28:09.569762 24494 tablet_bootstrap.cc:492] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:09.542703 24378 raft_consensus.cc:2454] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:09.570214 24492 tablet_bootstrap.cc:492] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:09.560599 24296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:09.563325 24680 raft_consensus.cc:482] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:09.570679 24492 ts_tablet_manager.cc:1397] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.330s	user 0.060s	sys 0.134s
I20260924 14:28:09.570817 24680 raft_consensus.cc:504] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.574152 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5caac6840188451a8e325a37b8ea0e3c" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:09.574424 24680 leader_election.cc:288] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:09.569078 24771 raft_consensus.cc:504] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.574653 24795 raft_consensus.cc:504] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.576439 24771 leader_election.cc:288] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:09.576594 24344 leader_election.cc:341] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.577584 24296 raft_consensus.cc:3037] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.577821 24795 leader_election.cc:288] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:09.578303 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:09.579918 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:09.581743 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5caac6840188451a8e325a37b8ea0e3c" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:09.582350 24452 raft_consensus.cc:2454] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:09.582868 24417 leader_election.cc:302] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
W20260924 14:28:09.552294 24268 consensus_peers.cc:487] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:09.583717 24343 leader_election.cc:302] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:09.584342 24790 raft_consensus.cc:2781] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:09.585050 24790 raft_consensus.cc:686] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:09.586130 24790 consensus_queue.cc:237] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.590081 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:09.590813 24447 raft_consensus.cc:3037] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.591843 24771 raft_consensus.cc:2781] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:09.592233 24771 raft_consensus.cc:482] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:09.592561 24771 raft_consensus.cc:3037] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.592592 24492 raft_consensus.cc:348] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.593598 24492 raft_consensus.cc:374] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.593922 24492 raft_consensus.cc:729] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.594776 24492 consensus_queue.cc:260] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.590729 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:09.596992 24492 ts_tablet_manager.cc:1428] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.006s
I20260924 14:28:09.597889 24492 tablet_bootstrap.cc:492] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:09.599756 24344 leader_election.cc:302] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:09.601590 24494 tablet_bootstrap.cc:654] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:09.604193 24680 raft_consensus.cc:2726] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:09.608793 24795 raft_consensus.cc:482] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:28:09.610051 24268 leader_election.cc:341] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.609299 24795 raft_consensus.cc:504] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.616715 24795 leader_election.cc:288] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:09.622165 24296 raft_consensus.cc:2454] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:09.625859 24492 tablet_bootstrap.cc:654] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:09.627353 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:09.627516 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:09.628556 24452 raft_consensus.cc:2379] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cafc2c5ad4df4224a5ccc4893de70144 in current term 1: Already voted for candidate 61d316eb5dfe405c95019383bbebecaa in this term.
I20260924 14:28:09.629966 24268 leader_election.cc:302] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:09.631527 24727 raft_consensus.cc:3037] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.642590 24493 tablet_bootstrap.cc:654] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:09.644860 24727 raft_consensus.cc:2726] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:09.646484 24494 tablet_bootstrap.cc:492] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:09.646961 24494 ts_tablet_manager.cc:1397] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.012s
I20260924 14:28:09.636318 24447 raft_consensus.cc:2454] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:09.649121 24726 raft_consensus.cc:482] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:09.651739 24726 raft_consensus.cc:504] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.652195 24268 leader_election.cc:302] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:09.652987 24727 raft_consensus.cc:2781] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:09.652968 24680 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:09.651145 24771 raft_consensus.cc:504] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.655001 24727 raft_consensus.cc:686] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:09.655649 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b55145a82545d9b769ea9d04bf1d99" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:09.650386 24201 catalog_manager.cc:5495] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:09.655948 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b55145a82545d9b769ea9d04bf1d99" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:09.657192 24296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5caac6840188451a8e325a37b8ea0e3c" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:09.659176 24302 raft_consensus.cc:2454] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
W20260924 14:28:09.660040 24344 leader_election.cc:341] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.655949 24727 consensus_queue.cc:237] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:09.656949 24343 leader_election.cc:341] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.656155 24726 leader_election.cc:288] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:09.662403 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5caac6840188451a8e325a37b8ea0e3c" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:09.662990 24452 raft_consensus.cc:3037] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.663923 24344 leader_election.cc:302] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:09.664665 24726 raft_consensus.cc:2781] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:09.665048 24726 raft_consensus.cc:482] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:09.665371 24726 raft_consensus.cc:3037] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.671229 24452 raft_consensus.cc:2454] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:09.672879 24343 leader_election.cc:302] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:09.673935 24680 raft_consensus.cc:2781] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:09.674397 24680 raft_consensus.cc:686] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:09.674993 24726 raft_consensus.cc:504] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.677178 24726 leader_election.cc:288] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:09.657574 24771 leader_election.cc:288] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:09.678486 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b55145a82545d9b769ea9d04bf1d99" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:09.681363 24569 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:09.681131 24795 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260924 14:28:09.684302 24296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b55145a82545d9b769ea9d04bf1d99" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:09.684965 24296 raft_consensus.cc:3037] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.675159 24680 consensus_queue.cc:237] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:09.686466 24343 leader_election.cc:341] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.710999 24494 raft_consensus.cc:348] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.711732 24494 raft_consensus.cc:374] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.711982 24494 raft_consensus.cc:729] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.712507 24494 consensus_queue.cc:260] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.717276 24494 ts_tablet_manager.cc:1428] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.002s
I20260924 14:28:09.718822 24494 tablet_bootstrap.cc:492] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:09.724058 24569 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:09.721547 24296 raft_consensus.cc:2454] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:09.742143 24344 leader_election.cc:302] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:09.744665 24500 raft_consensus.cc:2781] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:09.745110 24500 raft_consensus.cc:686] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:09.745764 24500 consensus_queue.cc:237] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.761231 24492 tablet_bootstrap.cc:492] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:09.761752 24492 ts_tablet_manager.cc:1397] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.164s	user 0.013s	sys 0.020s
I20260924 14:28:09.775424 24201 catalog_manager.cc:5495] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
W20260924 14:28:09.786046 24343 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 72371.9us
I20260924 14:28:09.827840 24790 raft_consensus.cc:482] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:09.828356 24790 raft_consensus.cc:504] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.835665 24790 leader_election.cc:288] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:09.842342 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:09.843426 24726 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:09.845389 24727 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:09.846467 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:09.850274 24494 tablet_bootstrap.cc:654] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:09.852522 24417 leader_election.cc:341] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:09.856364 24418 leader_election.cc:341] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:09.856818 24418 leader_election.cc:302] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:09.859290 24602 raft_consensus.cc:2726] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:09.863648 24199 catalog_manager.cc:5495] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:09.865423 24492 raft_consensus.cc:348] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:09.868750 24375 raft_consensus.cc:3037] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:09.868860 24492 raft_consensus.cc:374] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:09.869349 24492 raft_consensus.cc:729] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:09.870386 24492 consensus_queue.cc:260] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.020051 24492 ts_tablet_manager.cc:1428] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.258s	user 0.026s	sys 0.009s
I20260924 14:28:10.027948 24494 tablet_bootstrap.cc:492] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:10.028461 24493 tablet_bootstrap.cc:492] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:10.028626 24494 ts_tablet_manager.cc:1397] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.310s	user 0.006s	sys 0.023s
I20260924 14:28:10.028982 24493 ts_tablet_manager.cc:1397] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.488s	user 0.007s	sys 0.028s
I20260924 14:28:10.031493 24492 tablet_bootstrap.cc:492] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:10.031809 24493 raft_consensus.cc:348] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.032756 24493 raft_consensus.cc:374] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:10.033020 24493 raft_consensus.cc:729] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:10.033617 24493 consensus_queue.cc:260] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.035802 24493 ts_tablet_manager.cc:1428] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260924 14:28:10.038836 24492 tablet_bootstrap.cc:654] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:10.043334 24494 raft_consensus.cc:348] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.044270 24494 raft_consensus.cc:374] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:10.044574 24494 raft_consensus.cc:729] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:10.045220 24494 consensus_queue.cc:260] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.047721 24494 ts_tablet_manager.cc:1428] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260924 14:28:10.048673 24494 tablet_bootstrap.cc:492] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:10.064699 24493 tablet_bootstrap.cc:492] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:10.068344 24687 consensus_queue.cc:1035] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:10.072978 24493 tablet_bootstrap.cc:654] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:10.074165 24492 tablet_bootstrap.cc:492] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:10.084465 24492 ts_tablet_manager.cc:1397] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.010s
I20260924 14:28:10.075369 24494 tablet_bootstrap.cc:654] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:10.083246 24792 consensus_queue.cc:1035] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:10.087249 24492 raft_consensus.cc:348] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.088107 24492 raft_consensus.cc:374] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:10.088397 24492 raft_consensus.cc:729] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:10.089025 24492 consensus_queue.cc:260] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.091192 24492 ts_tablet_manager.cc:1428] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260924 14:28:10.099673 24199 catalog_manager.cc:5495] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:10.100487 24492 tablet_bootstrap.cc:492] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:10.124789 24796 raft_consensus.cc:482] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:10.125389 24796 raft_consensus.cc:504] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.129693 24796 leader_election.cc:288] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:10.132994 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870bbbde2a2f4c76bb558d8c43d7587b" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:10.133720 24447 raft_consensus.cc:2454] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:10.137804 24268 leader_election.cc:302] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:10.139531 24796 raft_consensus.cc:2781] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:10.139961 24796 raft_consensus.cc:482] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:10.140326 24796 raft_consensus.cc:3037] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:10.140780 24687 raft_consensus.cc:482] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:10.143219 24687 raft_consensus.cc:504] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.149922 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870bbbde2a2f4c76bb558d8c43d7587b" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:10.150002 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b55145a82545d9b769ea9d04bf1d99" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:10.151031 24300 raft_consensus.cc:2379] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 1: Already voted for candidate 61d316eb5dfe405c95019383bbebecaa in this term.
I20260924 14:28:10.150803 24796 raft_consensus.cc:504] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.153038 24796 leader_election.cc:288] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:10.153304 24493 tablet_bootstrap.cc:492] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:10.153823 24493 ts_tablet_manager.cc:1397] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.089s	user 0.016s	sys 0.002s
W20260924 14:28:10.153867 24268 leader_election.cc:341] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:10.154467 24494 tablet_bootstrap.cc:492] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:10.155066 24494 ts_tablet_manager.cc:1397] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.107s	user 0.023s	sys 0.013s
I20260924 14:28:10.155340 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870bbbde2a2f4c76bb558d8c43d7587b" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:10.156049 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870bbbde2a2f4c76bb558d8c43d7587b" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:10.161576 24447 raft_consensus.cc:3037] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:10.162590 24492 tablet_bootstrap.cc:654] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:10.163784 24268 leader_election.cc:341] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:10.169304 24447 raft_consensus.cc:2454] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:10.175634 24494 raft_consensus.cc:348] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.178913 24268 leader_election.cc:302] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:10.182417 24569 raft_consensus.cc:2781] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:10.182869 24569 raft_consensus.cc:686] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:10.183604 24569 consensus_queue.cc:237] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.156941 24687 leader_election.cc:288] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:10.157227 24375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b55145a82545d9b769ea9d04bf1d99" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:10.158309 24493 raft_consensus.cc:348] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.187809 24493 raft_consensus.cc:374] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:10.188122 24493 raft_consensus.cc:729] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:10.188759 24493 consensus_queue.cc:260] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.191685 24493 ts_tablet_manager.cc:1428] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.004s
I20260924 14:28:10.176774 24494 raft_consensus.cc:374] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:10.192631 24493 tablet_bootstrap.cc:492] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:10.188791 24417 leader_election.cc:302] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:10.196761 24494 raft_consensus.cc:729] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:10.200381 24494 consensus_queue.cc:260] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.185173 24492 tablet_bootstrap.cc:492] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:10.202299 24492 ts_tablet_manager.cc:1397] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.102s	user 0.004s	sys 0.010s
I20260924 14:28:10.202168 24687 raft_consensus.cc:3037] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:10.204985 24492 raft_consensus.cc:348] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.205686 24492 raft_consensus.cc:374] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:10.206000 24492 raft_consensus.cc:729] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:10.208879 24680 raft_consensus.cc:482] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:10.211084 24493 tablet_bootstrap.cc:654] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:10.212883 24680 raft_consensus.cc:504] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.213888 24200 catalog_manager.cc:5495] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:10.217269 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:10.217578 24680 leader_election.cc:288] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:10.208554 24494 ts_tablet_manager.cc:1428] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.039s	user 0.003s	sys 0.003s
I20260924 14:28:10.217904 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:10.219167 24494 tablet_bootstrap.cc:492] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:10.227618 24680 consensus_queue.cc:1035] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:10.226825 24492 consensus_queue.cc:260] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.231031 24302 raft_consensus.cc:3037] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:10.232183 24492 ts_tablet_manager.cc:1428] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260924 14:28:10.235105 24492 tablet_bootstrap.cc:492] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:10.236320 24344 leader_election.cc:302] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:10.237931 24837 raft_consensus.cc:2726] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:10.241600 24687 raft_consensus.cc:2726] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:10.244513 24494 tablet_bootstrap.cc:654] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:10.259208 24447 raft_consensus.cc:3037] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:10.263425 24680 consensus_queue.cc:1035] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:10.268932 24837 consensus_queue.cc:1035] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:10.270910 24837 consensus_queue.cc:1035] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260924 14:28:10.294809 24492 tablet_bootstrap.cc:654] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:10.300233 24268 consensus_peers.cc:487] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:10.322268 24494 tablet_bootstrap.cc:492] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:10.322809 24494 ts_tablet_manager.cc:1397] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.104s	user 0.009s	sys 0.014s
I20260924 14:28:10.324537 24569 consensus_queue.cc:1035] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:10.314663 24837 raft_consensus.cc:482] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:10.335058 24837 raft_consensus.cc:504] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.349716 24837 leader_election.cc:288] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:10.391929 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 62123.3us
I20260924 14:28:10.409623 24861 raft_consensus.cc:482] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:10.411067 24861 raft_consensus.cc:504] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.416038 24861 leader_election.cc:288] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:10.426707 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e065e0c774647b1a9b127accaebd70a" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:10.428392 24448 raft_consensus.cc:2454] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:10.424177 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e065e0c774647b1a9b127accaebd70a" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
W20260924 14:28:10.434011 24857 log.cc:924] Time spent T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260924 14:28:10.463423 24302 raft_consensus.cc:2454] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:10.510461 24790 raft_consensus.cc:482] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 61d316eb5dfe405c95019383bbebecaa)
I20260924 14:28:10.511327 24790 raft_consensus.cc:504] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.545603 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22099296aa6419a924560e9fbe81cfd" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:10.561721 24494 raft_consensus.cc:348] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.562970 24494 raft_consensus.cc:374] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:10.563369 24494 raft_consensus.cc:729] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:10.564186 24494 consensus_queue.cc:260] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.610996 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22099296aa6419a924560e9fbe81cfd" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:10.614079 24790 leader_election.cc:288] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:10.615561 24493 tablet_bootstrap.cc:492] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
W20260924 14:28:10.765760 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 160373us
I20260924 14:28:10.714764 24494 ts_tablet_manager.cc:1428] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.392s	user 0.008s	sys 0.004s
I20260924 14:28:10.620021 24492 tablet_bootstrap.cc:492] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:10.771893 24493 ts_tablet_manager.cc:1397] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.579s	user 0.039s	sys 0.103s
I20260924 14:28:10.783665 24343 leader_election.cc:302] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
W20260924 14:28:10.791092 24248 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260924 14:28:10.796347 24268 leader_election.cc:341] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:10.804571 24493 raft_consensus.cc:348] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.805815 24493 raft_consensus.cc:374] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:10.805897 24726 raft_consensus.cc:2781] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:10.806185 24493 raft_consensus.cc:729] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:10.806389 24726 raft_consensus.cc:482] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:10.806756 24726 raft_consensus.cc:3037] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:10.807098 24493 consensus_queue.cc:260] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.809993 24493 ts_tablet_manager.cc:1428] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260924 14:28:10.811764 24493 tablet_bootstrap.cc:492] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:10.830590 24492 ts_tablet_manager.cc:1397] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.596s	user 0.039s	sys 0.017s
I20260924 14:28:10.833340 24494 tablet_bootstrap.cc:492] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:10.844115 24492 raft_consensus.cc:348] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.852061 24492 raft_consensus.cc:374] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:10.852325 24492 raft_consensus.cc:729] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:10.854473 24377 raft_consensus.cc:3037] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:10.862998 24493 tablet_bootstrap.cc:654] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:10.859259 24492 consensus_queue.cc:260] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.881958 24492 ts_tablet_manager.cc:1428] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.002s
I20260924 14:28:10.883569 24492 tablet_bootstrap.cc:492] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:10.887612 24378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6bde1e6051745469ca811e5940bd8f9" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:10.901072 24494 tablet_bootstrap.cc:654] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:10.913992 24268 leader_election.cc:341] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:10.900990 24726 raft_consensus.cc:504] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.926131 24726 leader_election.cc:288] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:10.915690 24268 leader_election.cc:302] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:10.945148 24862 raft_consensus.cc:2726] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:10.972745 24376 raft_consensus.cc:3037] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:10.981047 24268 consensus_peers.cc:487] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:10.984786 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e065e0c774647b1a9b127accaebd70a" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:10.985505 24298 raft_consensus.cc:3037] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:10.985936 24493 tablet_bootstrap.cc:492] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:10.986498 24493 ts_tablet_manager.cc:1397] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.011s
I20260924 14:28:10.990629 24869 raft_consensus.cc:482] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:10.991216 24869 raft_consensus.cc:504] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:10.994081 24869 leader_election.cc:288] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:10.998134 24298 raft_consensus.cc:2454] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:11.005800 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3667aba281046d3b974da9065022529" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:11.007655 24417 leader_election.cc:341] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:11.012249 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 215173us
I20260924 14:28:11.023332 24492 tablet_bootstrap.cc:654] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.023872 24493 raft_consensus.cc:348] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.049912 24493 raft_consensus.cc:374] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.050216 24493 raft_consensus.cc:729] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.056481 24493 consensus_queue.cc:260] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.062352 24494 tablet_bootstrap.cc:492] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:11.064395 24494 ts_tablet_manager.cc:1397] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.231s	user 0.024s	sys 0.031s
I20260924 14:28:11.066205 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e065e0c774647b1a9b127accaebd70a" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:11.066938 24445 raft_consensus.cc:3037] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.073125 24494 raft_consensus.cc:348] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.077316 24493 ts_tablet_manager.cc:1428] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.090s	user 0.008s	sys 0.000s
I20260924 14:28:11.078128 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6bde1e6051745469ca811e5940bd8f9" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:11.078423 24493 tablet_bootstrap.cc:492] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:11.077243 24494 raft_consensus.cc:374] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.084010 24494 raft_consensus.cc:729] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.086100 24494 consensus_queue.cc:260] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.104977 24494 ts_tablet_manager.cc:1428] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260924 14:28:11.106281 24494 tablet_bootstrap.cc:492] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:11.110867 24876 raft_consensus.cc:482] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:11.111480 24876 raft_consensus.cc:504] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.116598 24796 raft_consensus.cc:482] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.117200 24796 raft_consensus.cc:504] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.119395 24796 leader_election.cc:288] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.119585 24445 raft_consensus.cc:2454] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:11.121032 24372 raft_consensus.cc:3037] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.123559 24727 consensus_queue.cc:1035] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260924 14:28:11.121658 24344 leader_election.cc:302] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:11.128473 24873 raft_consensus.cc:2781] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:11.129006 24873 raft_consensus.cc:686] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:11.129971 24873 consensus_queue.cc:237] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.132958 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3667aba281046d3b974da9065022529" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:11.134662 24876 leader_election.cc:288] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.138186 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:11.145603 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a57dc14600c450485f596239b30df0d" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:11.149209 24296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:11.153947 24493 tablet_bootstrap.cc:654] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:11.160441 24268 leader_election.cc:341] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.159809 24418 leader_election.cc:302] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:11.161190 24344 leader_election.cc:302] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:11.168774 24874 raft_consensus.cc:2726] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:28:11.171106 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a57dc14600c450485f596239b30df0d" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:11.183983 24492 tablet_bootstrap.cc:492] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:11.184608 24492 ts_tablet_manager.cc:1397] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.301s	user 0.021s	sys 0.017s
I20260924 14:28:11.185504 24494 tablet_bootstrap.cc:654] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.187058 24727 raft_consensus.cc:482] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.187662 24883 raft_consensus.cc:2726] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:28:11.187567 24727 raft_consensus.cc:504] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.190192 24727 leader_election.cc:288] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:11.172361 24418 leader_election.cc:341] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.195428 24418 leader_election.cc:302] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:11.195973 24492 raft_consensus.cc:348] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.196750 24492 raft_consensus.cc:374] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.197016 24492 raft_consensus.cc:729] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.197321 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:11.202450 24602 raft_consensus.cc:2726] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:11.202805 24727 raft_consensus.cc:482] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.203313 24727 raft_consensus.cc:504] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.202504 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:11.204594 24445 raft_consensus.cc:2454] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:11.218884 24492 consensus_queue.cc:260] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.205473 24727 leader_election.cc:288] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:11.226305 24268 leader_election.cc:341] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:11.227324 24268 leader_election.cc:341] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.227837 24268 leader_election.cc:302] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:11.229100 24796 raft_consensus.cc:2726] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:11.229202 24268 leader_election.cc:302] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:11.231626 24727 raft_consensus.cc:2781] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:11.232084 24727 raft_consensus.cc:482] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:11.232448 24727 raft_consensus.cc:3037] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.240456 24492 ts_tablet_manager.cc:1428] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.001s
I20260924 14:28:11.241127 24444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638ffc5970704a53a26967c81af23e52" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:11.243193 24493 tablet_bootstrap.cc:492] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:11.243930 24493 ts_tablet_manager.cc:1397] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.166s	user 0.018s	sys 0.015s
I20260924 14:28:11.243186 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638ffc5970704a53a26967c81af23e52" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:11.253377 24492 tablet_bootstrap.cc:492] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:11.254882 24494 tablet_bootstrap.cc:492] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:11.255383 24494 ts_tablet_manager.cc:1397] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.149s	user 0.012s	sys 0.005s
I20260924 14:28:11.258113 24727 raft_consensus.cc:504] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.261387 24494 raft_consensus.cc:348] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.262204 24494 raft_consensus.cc:374] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.262475 24494 raft_consensus.cc:729] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.261950 24201 catalog_manager.cc:5495] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:11.265429 24493 raft_consensus.cc:348] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.266162 24493 raft_consensus.cc:374] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.266419 24493 raft_consensus.cc:729] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.264810 24727 leader_election.cc:288] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.267127 24494 consensus_queue.cc:260] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.269330 24494 ts_tablet_manager.cc:1428] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.001s
I20260924 14:28:11.270248 24494 tablet_bootstrap.cc:492] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
W20260924 14:28:11.271543 24268 leader_election.cc:341] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:11.272256 24268 leader_election.cc:341] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.272624 24268 leader_election.cc:302] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:11.273485 24493 consensus_queue.cc:260] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.277961 24493 ts_tablet_manager.cc:1428] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.034s	user 0.001s	sys 0.005s
I20260924 14:28:11.278959 24493 tablet_bootstrap.cc:492] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:11.285095 24492 tablet_bootstrap.cc:654] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.290241 24494 tablet_bootstrap.cc:654] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.296358 24493 tablet_bootstrap.cc:654] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.301426 24727 raft_consensus.cc:2726] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:11.310029 24444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:11.310708 24444 raft_consensus.cc:3037] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.314487 24494 tablet_bootstrap.cc:492] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:11.314982 24494 ts_tablet_manager.cc:1397] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.000s
I20260924 14:28:11.319224 24494 raft_consensus.cc:348] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.320060 24494 raft_consensus.cc:374] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.320338 24494 raft_consensus.cc:729] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.323470 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:11.326534 24444 raft_consensus.cc:2454] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:11.327955 24494 consensus_queue.cc:260] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.329974 24268 leader_election.cc:302] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
W20260924 14:28:11.334975 24268 leader_election.cc:341] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.337569 24492 tablet_bootstrap.cc:492] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:11.338310 24492 ts_tablet_manager.cc:1397] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.001s
I20260924 14:28:11.333038 24861 raft_consensus.cc:2781] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:11.340135 24861 raft_consensus.cc:686] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:11.342239 24861 consensus_queue.cc:237] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.344537 24493 tablet_bootstrap.cc:492] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:11.345041 24493 ts_tablet_manager.cc:1397] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.006s
I20260924 14:28:11.346418 24494 ts_tablet_manager.cc:1428] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260924 14:28:11.355007 24493 raft_consensus.cc:348] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.356503 24493 raft_consensus.cc:374] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.356870 24493 raft_consensus.cc:729] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.357522 24493 consensus_queue.cc:260] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.359163 24494 tablet_bootstrap.cc:492] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:11.359915 24493 ts_tablet_manager.cc:1428] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.003s
I20260924 14:28:11.363664 24493 tablet_bootstrap.cc:492] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:11.356541 24492 raft_consensus.cc:348] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.369616 24492 raft_consensus.cc:374] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.369890 24492 raft_consensus.cc:729] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.370471 24492 consensus_queue.cc:260] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.379418 24494 tablet_bootstrap.cc:654] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.388180 24602 raft_consensus.cc:482] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.388826 24602 raft_consensus.cc:504] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.392904 24602 leader_election.cc:288] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:11.392568 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638ffc5970704a53a26967c81af23e52" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:11.395619 24417 leader_election.cc:341] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.410718 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638ffc5970704a53a26967c81af23e52" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:11.411592 24298 raft_consensus.cc:2454] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:11.412767 24418 leader_election.cc:302] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:11.414597 24874 raft_consensus.cc:2781] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:11.415081 24874 raft_consensus.cc:482] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:11.415517 24874 raft_consensus.cc:3037] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.423007 24492 ts_tablet_manager.cc:1428] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.002s
I20260924 14:28:11.425410 24372 raft_consensus.cc:3037] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.429495 24870 raft_consensus.cc:482] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.429629 24492 tablet_bootstrap.cc:492] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:11.430117 24870 raft_consensus.cc:504] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.436834 24870 leader_election.cc:288] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.439421 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a57dc14600c450485f596239b30df0d" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:11.442147 24343 leader_election.cc:341] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.453014 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a57dc14600c450485f596239b30df0d" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:11.453751 24298 raft_consensus.cc:2454] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 0.
I20260924 14:28:11.458743 24874 raft_consensus.cc:504] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.459175 24344 leader_election.cc:302] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:11.461387 24872 raft_consensus.cc:2781] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:11.461807 24872 raft_consensus.cc:482] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:11.462146 24872 raft_consensus.cc:3037] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.465500 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638ffc5970704a53a26967c81af23e52" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:11.465690 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638ffc5970704a53a26967c81af23e52" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:11.466089 24874 leader_election.cc:288] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
W20260924 14:28:11.466979 24417 leader_election.cc:341] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.467551 24861 raft_consensus.cc:482] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.468039 24861 raft_consensus.cc:504] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.470067 24861 leader_election.cc:288] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.471855 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234eb8cdbf4c4e7599789194a0032c00" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:11.472532 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234eb8cdbf4c4e7599789194a0032c00" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:11.473986 24268 leader_election.cc:341] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:11.476065 24268 leader_election.cc:341] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.476465 24268 leader_election.cc:302] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:11.479089 24861 raft_consensus.cc:2726] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:11.476873 24298 raft_consensus.cc:3037] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.486904 24493 tablet_bootstrap.cc:654] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.483040 24861 raft_consensus.cc:482] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.495930 24298 raft_consensus.cc:2454] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:11.497107 24418 leader_election.cc:302] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:11.498277 24874 raft_consensus.cc:2781] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:11.500504 24874 raft_consensus.cc:686] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:11.493050 24861 raft_consensus.cc:504] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.499111 24872 raft_consensus.cc:504] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.504410 24492 tablet_bootstrap.cc:654] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.502836 24874 consensus_queue.cc:237] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.506237 24861 leader_election.cc:288] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.509032 24444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a57dc14600c450485f596239b30df0d" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
W20260924 14:28:11.510548 24343 leader_election.cc:341] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.509162 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3822638d266742c4908907a61efe8231" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:11.507539 24872 leader_election.cc:288] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.510778 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3822638d266742c4908907a61efe8231" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:11.511067 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a57dc14600c450485f596239b30df0d" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
W20260924 14:28:11.513784 24268 leader_election.cc:341] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.514189 24493 tablet_bootstrap.cc:492] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
W20260924 14:28:11.514786 24268 leader_election.cc:341] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.514874 24493 ts_tablet_manager.cc:1397] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.151s	user 0.014s	sys 0.005s
I20260924 14:28:11.515216 24268 leader_election.cc:302] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:11.516520 24861 raft_consensus.cc:2726] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:11.516494 24298 raft_consensus.cc:3037] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.521461 24494 tablet_bootstrap.cc:492] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:11.521999 24494 ts_tablet_manager.cc:1397] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.163s	user 0.020s	sys 0.000s
I20260924 14:28:11.533080 24494 raft_consensus.cc:348] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.534026 24494 raft_consensus.cc:374] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.534336 24494 raft_consensus.cc:729] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.535108 24494 consensus_queue.cc:260] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.544472 24494 ts_tablet_manager.cc:1428] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260924 14:28:11.546635 24494 tablet_bootstrap.cc:492] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:11.550441 24298 raft_consensus.cc:2454] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:11.556133 24344 leader_election.cc:302] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:11.557344 24872 raft_consensus.cc:2781] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:11.557931 24872 raft_consensus.cc:686] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:11.553145 24200 catalog_manager.cc:5495] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:11.559059 24872 consensus_queue.cc:237] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.562134 24493 raft_consensus.cc:348] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.569079 24493 raft_consensus.cc:374] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.569732 24492 tablet_bootstrap.cc:492] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:11.570197 24492 ts_tablet_manager.cc:1397] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.141s	user 0.018s	sys 0.014s
I20260924 14:28:11.573639 24602 raft_consensus.cc:482] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.565881 24494 tablet_bootstrap.cc:654] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.574163 24602 raft_consensus.cc:504] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.576023 24602 leader_election.cc:288] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:11.566165 24198 catalog_manager.cc:5495] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:11.579138 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22099296aa6419a924560e9fbe81cfd" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:11.583194 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22099296aa6419a924560e9fbe81cfd" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:11.583273 24492 raft_consensus.cc:348] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.584131 24492 raft_consensus.cc:374] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.584051 24298 raft_consensus.cc:2454] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:11.584506 24492 raft_consensus.cc:729] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.586125 24418 leader_election.cc:302] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:11.586956 24869 raft_consensus.cc:2781] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:11.587343 24869 raft_consensus.cc:482] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:11.587741 24869 raft_consensus.cc:3037] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:11.592036 24417 leader_election.cc:341] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.592334 24870 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:11.585160 24492 consensus_queue.cc:260] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.600638 24869 raft_consensus.cc:504] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.569401 24493 raft_consensus.cc:729] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.602375 24493 consensus_queue.cc:260] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.604221 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22099296aa6419a924560e9fbe81cfd" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:11.605363 24298 raft_consensus.cc:3037] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.603921 24492 ts_tablet_manager.cc:1428] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260924 14:28:11.607622 24492 tablet_bootstrap.cc:492] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:11.612691 24493 ts_tablet_manager.cc:1428] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.002s
I20260924 14:28:11.615312 24493 tablet_bootstrap.cc:492] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:11.611819 24200 catalog_manager.cc:5495] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
W20260924 14:28:11.622399 24902 log.cc:924] Time spent T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260924 14:28:11.636436 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22099296aa6419a924560e9fbe81cfd" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
W20260924 14:28:11.639312 24417 leader_election.cc:341] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.639889 24771 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:11.634728 24869 leader_election.cc:288] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:11.647861 24298 raft_consensus.cc:2454] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:11.649804 24418 leader_election.cc:302] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:11.654210 24494 tablet_bootstrap.cc:492] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:11.654929 24494 ts_tablet_manager.cc:1397] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.109s	user 0.022s	sys 0.003s
I20260924 14:28:11.655685 24874 raft_consensus.cc:2781] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:11.658270 24874 raft_consensus.cc:686] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:11.659307 24874 consensus_queue.cc:237] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.669152 24493 tablet_bootstrap.cc:654] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.669400 24492 tablet_bootstrap.cc:654] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.668948 24494 raft_consensus.cc:348] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.671885 24494 raft_consensus.cc:374] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.672232 24494 raft_consensus.cc:729] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.673966 24494 consensus_queue.cc:260] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.676398 24494 ts_tablet_manager.cc:1428] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260924 14:28:11.683163 24494 tablet_bootstrap.cc:492] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:11.692399 24200 catalog_manager.cc:5495] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:11.720139 24492 tablet_bootstrap.cc:492] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:11.720650 24492 ts_tablet_manager.cc:1397] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.006s
I20260924 14:28:11.722361 24494 tablet_bootstrap.cc:654] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.724099 24493 tablet_bootstrap.cc:492] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:11.724573 24493 ts_tablet_manager.cc:1397] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.110s	user 0.019s	sys 0.029s
I20260924 14:28:11.727627 24861 raft_consensus.cc:482] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.728144 24861 raft_consensus.cc:504] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.728870 24492 raft_consensus.cc:348] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.729813 24492 raft_consensus.cc:374] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.730017 24861 leader_election.cc:288] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.730152 24492 raft_consensus.cc:729] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.731053 24492 consensus_queue.cc:260] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.731822 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e4953bd67e40f5823697e67d029f52" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:11.734062 24492 ts_tablet_manager.cc:1428] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260924 14:28:11.734624 24493 raft_consensus.cc:348] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:11.735459 24268 leader_election.cc:341] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.736285 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e4953bd67e40f5823697e67d029f52" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:11.735550 24492 tablet_bootstrap.cc:492] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:11.735354 24493 raft_consensus.cc:374] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.737708 24493 raft_consensus.cc:729] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.738359 24493 consensus_queue.cc:260] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:11.740813 24268 leader_election.cc:341] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.741259 24268 leader_election.cc:302] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:11.742244 24861 raft_consensus.cc:2726] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:11.746308 24492 tablet_bootstrap.cc:654] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.750922 24493 ts_tablet_manager.cc:1428] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.002s
I20260924 14:28:11.752046 24493 tablet_bootstrap.cc:492] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:11.754233 24494 tablet_bootstrap.cc:492] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:11.754786 24494 ts_tablet_manager.cc:1397] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.072s	user 0.013s	sys 0.007s
I20260924 14:28:11.757311 24494 raft_consensus.cc:348] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.759621 24494 raft_consensus.cc:374] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.759931 24494 raft_consensus.cc:729] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.761718 24493 tablet_bootstrap.cc:654] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.765676 24861 consensus_queue.cc:1035] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:11.766212 24494 consensus_queue.cc:260] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.842272 24796 raft_consensus.cc:482] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.842788 24796 raft_consensus.cc:504] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.850425 24492 tablet_bootstrap.cc:492] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:11.851034 24492 ts_tablet_manager.cc:1397] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.116s	user 0.013s	sys 0.004s
I20260924 14:28:11.854655 24771 raft_consensus.cc:482] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.856361 24771 raft_consensus.cc:504] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.857750 24492 raft_consensus.cc:348] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.858233 24771 leader_election.cc:288] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.858448 24492 raft_consensus.cc:374] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:11.858703 24492 raft_consensus.cc:729] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:11.859493 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e83b406c1944efb04163f032a3b4f6" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:11.860251 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e83b406c1944efb04163f032a3b4f6" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:11.860852 24447 raft_consensus.cc:2454] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:11.862613 24447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:11.863929 24494 ts_tablet_manager.cc:1428] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.050s
I20260924 14:28:11.859302 24492 consensus_queue.cc:260] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.850317 24796 leader_election.cc:288] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.867820 24268 leader_election.cc:302] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:11.870061 24860 raft_consensus.cc:2781] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:11.870430 24860 raft_consensus.cc:482] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:11.870129 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:11.870754 24860 raft_consensus.cc:3037] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.873961 24492 ts_tablet_manager.cc:1428] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.003s
I20260924 14:28:11.874706 24344 leader_election.cc:302] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:11.877781 24771 raft_consensus.cc:2726] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:28:11.876633 24268 leader_election.cc:341] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:11.882014 24869 raft_consensus.cc:482] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.882619 24869 raft_consensus.cc:504] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.899757 24494 tablet_bootstrap.cc:492] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:11.886135 24869 leader_election.cc:288] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:11.909145 24494 tablet_bootstrap.cc:654] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.915903 24492 tablet_bootstrap.cc:492] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:11.925292 24296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3822638d266742c4908907a61efe8231" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:11.926007 24296 raft_consensus.cc:2454] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:11.931393 24418 leader_election.cc:302] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:11.932312 24869 raft_consensus.cc:2781] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:11.932685 24869 raft_consensus.cc:482] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:11.940915 24869 raft_consensus.cc:3037] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:11.934146 24860 raft_consensus.cc:504] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.946604 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3822638d266742c4908907a61efe8231" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:11.947580 24868 raft_consensus.cc:482] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.948031 24868 raft_consensus.cc:504] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.950229 24868 leader_election.cc:288] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:11.957352 24492 tablet_bootstrap.cc:654] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:11.952939 24860 leader_election.cc:288] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:11.968386 24248 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:11.961171 24869 raft_consensus.cc:504] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:11.995343 24417 leader_election.cc:341] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:11.996320 24417 consensus_peers.cc:487] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:11.997187 24874 raft_consensus.cc:482] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:11.997622 24874 raft_consensus.cc:504] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:11.999074 24869 leader_election.cc:288] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:11.999428 24874 leader_election.cc:288] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:12.004593 24493 tablet_bootstrap.cc:492] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:12.004505 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3822638d266742c4908907a61efe8231" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:12.005103 24493 ts_tablet_manager.cc:1397] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.253s	user 0.065s	sys 0.083s
I20260924 14:28:12.005179 24302 raft_consensus.cc:3037] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.007762 24493 raft_consensus.cc:348] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.008450 24493 raft_consensus.cc:374] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.008704 24493 raft_consensus.cc:729] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.009275 24493 consensus_queue.cc:260] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.011361 24493 ts_tablet_manager.cc:1428] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260924 14:28:12.012543 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf594a126774d39a57f354d4fc7d338" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:12.013585 24493 tablet_bootstrap.cc:492] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:12.012492 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3822638d266742c4908907a61efe8231" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:12.014089 24376 raft_consensus.cc:3037] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.016510 24302 raft_consensus.cc:2454] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:12.019389 24493 tablet_bootstrap.cc:654] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
W20260924 14:28:12.019487 24417 leader_election.cc:341] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:12.021566 24418 leader_election.cc:302] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:12.022493 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3667aba281046d3b974da9065022529" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:12.023128 24372 raft_consensus.cc:2454] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:12.023640 24370 raft_consensus.cc:3037] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.023857 24869 raft_consensus.cc:2781] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:12.024396 24869 raft_consensus.cc:686] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:12.025285 24869 consensus_queue.cc:237] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.027838 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e83b406c1944efb04163f032a3b4f6" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:12.028162 24494 tablet_bootstrap.cc:492] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:12.028622 24494 ts_tablet_manager.cc:1397] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.142s	user 0.011s	sys 0.016s
I20260924 14:28:12.029734 24874 consensus_queue.cc:1035] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:12.030988 24494 raft_consensus.cc:348] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.031895 24494 raft_consensus.cc:374] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.032166 24494 raft_consensus.cc:729] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.032757 24494 consensus_queue.cc:260] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.033470 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf594a126774d39a57f354d4fc7d338" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:12.034709 24494 ts_tablet_manager.cc:1428] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260924 14:28:12.035740 24494 tablet_bootstrap.cc:492] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:12.038926 24302 raft_consensus.cc:2454] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:12.045919 24493 tablet_bootstrap.cc:492] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:12.046437 24493 ts_tablet_manager.cc:1397] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.000s
I20260924 14:28:12.048328 24268 leader_election.cc:302] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:12.049326 24860 raft_consensus.cc:2781] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:12.049801 24860 raft_consensus.cc:482] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:12.050164 24860 raft_consensus.cc:3037] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:12.051371 24268 leader_election.cc:341] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:12.054826 24493 raft_consensus.cc:348] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.055649 24493 raft_consensus.cc:374] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.055541 24796 consensus_queue.cc:1035] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:12.056030 24493 raft_consensus.cc:729] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.056488 24376 raft_consensus.cc:2454] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:12.056717 24493 consensus_queue.cc:260] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.062567 24494 tablet_bootstrap.cc:654] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.063024 24493 ts_tablet_manager.cc:1428] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260924 14:28:12.064055 24493 tablet_bootstrap.cc:492] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:12.071735 24418 leader_election.cc:302] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:12.073422 24869 raft_consensus.cc:2781] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:12.073799 24869 raft_consensus.cc:482] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:12.074115 24869 raft_consensus.cc:3037] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.074684 24860 raft_consensus.cc:504] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.076762 24860 leader_election.cc:288] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:12.083842 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3667aba281046d3b974da9065022529" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:12.085777 24371 raft_consensus.cc:3037] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.087431 24493 tablet_bootstrap.cc:654] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.090003 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3667aba281046d3b974da9065022529" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:12.090729 24445 raft_consensus.cc:2454] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:12.091087 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3667aba281046d3b974da9065022529" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:12.096611 24444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e83b406c1944efb04163f032a3b4f6" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:12.097335 24200 catalog_manager.cc:5495] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:12.104665 24446 raft_consensus.cc:3037] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.107719 24444 raft_consensus.cc:3037] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.127399 24494 tablet_bootstrap.cc:492] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:12.127939 24494 ts_tablet_manager.cc:1397] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.010s
W20260924 14:28:12.129370 24417 consensus_peers.cc:487] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:12.135016 24492 tablet_bootstrap.cc:492] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:12.135517 24492 ts_tablet_manager.cc:1397] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.246s	user 0.017s	sys 0.016s
I20260924 14:28:12.140738 24494 raft_consensus.cc:348] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.141589 24494 raft_consensus.cc:374] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.141882 24494 raft_consensus.cc:729] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.142553 24494 consensus_queue.cc:260] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.147996 24493 tablet_bootstrap.cc:492] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:12.148499 24493 ts_tablet_manager.cc:1397] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.085s	user 0.016s	sys 0.007s
I20260924 14:28:12.154799 24444 raft_consensus.cc:2454] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:12.144269 24869 raft_consensus.cc:504] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.160315 24493 raft_consensus.cc:348] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.161023 24493 raft_consensus.cc:374] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.161288 24493 raft_consensus.cc:729] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.155879 24492 raft_consensus.cc:348] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.162097 24492 raft_consensus.cc:374] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.162429 24492 raft_consensus.cc:729] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.162724 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf594a126774d39a57f354d4fc7d338" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:12.163090 24492 consensus_queue.cc:260] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:12.163959 24417 leader_election.cc:341] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:12.166128 24492 ts_tablet_manager.cc:1428] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260924 14:28:12.159092 24446 raft_consensus.cc:2454] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:12.161340 24302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf594a126774d39a57f354d4fc7d338" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:12.161561 24869 leader_election.cc:288] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:12.169377 24302 raft_consensus.cc:3037] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.164781 24494 ts_tablet_manager.cc:1428] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.002s
I20260924 14:28:12.167728 24268 leader_election.cc:302] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:12.157941 24371 raft_consensus.cc:2454] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:12.168438 24493 consensus_queue.cc:260] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.183681 24771 consensus_queue.cc:1035] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:12.184095 24302 raft_consensus.cc:2454] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:12.185189 24492 tablet_bootstrap.cc:492] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:12.186939 24268 leader_election.cc:302] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:12.177820 24860 raft_consensus.cc:2781] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:12.193135 24860 raft_consensus.cc:686] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:12.188503 24861 raft_consensus.cc:2781] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:12.196413 24860 consensus_queue.cc:237] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.197294 24861 raft_consensus.cc:686] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
W20260924 14:28:12.200253 24343 consensus_peers.cc:487] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:12.198583 24861 consensus_queue.cc:237] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.203079 24771 raft_consensus.cc:482] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:12.203606 24771 raft_consensus.cc:504] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.205224 24494 tablet_bootstrap.cc:492] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144: Bootstrap starting.
I20260924 14:28:12.205745 24771 leader_election.cc:288] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:12.207649 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22099296aa6419a924560e9fbe81cfd" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:12.208477 24297 raft_consensus.cc:2379] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61d316eb5dfe405c95019383bbebecaa in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:12.213039 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22099296aa6419a924560e9fbe81cfd" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:12.216588 24493 ts_tablet_manager.cc:1428] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260924 14:28:12.217702 24492 tablet_bootstrap.cc:654] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.224263 24494 tablet_bootstrap.cc:654] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.232519 24418 leader_election.cc:302] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:12.263392 24918 raft_consensus.cc:482] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:12.305835 24869 raft_consensus.cc:2781] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:12.238796 24344 leader_election.cc:302] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:12.306391 24869 raft_consensus.cc:686] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:12.300209 24492 tablet_bootstrap.cc:492] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:12.305714 24918 raft_consensus.cc:504] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:12.305676 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 73246.3us
I20260924 14:28:12.307256 24869 consensus_queue.cc:237] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.307319 24492 ts_tablet_manager.cc:1397] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.122s	user 0.014s	sys 0.006s
I20260924 14:28:12.310102 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:12.311517 24417 leader_election.cc:341] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:12.313287 24344 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 74649.7us
I20260924 14:28:12.314622 24771 raft_consensus.cc:3037] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.315694 24874 consensus_queue.cc:1035] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:12.275635 24493 tablet_bootstrap.cc:492] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:12.319938 24918 leader_election.cc:288] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:12.321568 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:12.322526 24297 raft_consensus.cc:2454] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:12.320437 24492 raft_consensus.cc:348] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.323066 24492 raft_consensus.cc:374] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.323606 24492 raft_consensus.cc:729] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.324319 24492 consensus_queue.cc:260] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.332295 24418 leader_election.cc:302] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:12.333775 24918 raft_consensus.cc:2781] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:12.334136 24918 raft_consensus.cc:482] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:12.334506 24918 raft_consensus.cc:3037] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.341806 24918 raft_consensus.cc:504] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.345279 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:12.345956 24297 raft_consensus.cc:3037] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.347957 24492 ts_tablet_manager.cc:1428] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260924 14:28:12.354336 24873 raft_consensus.cc:482] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:12.360913 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
W20260924 14:28:12.363032 24417 leader_election.cc:341] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:12.355731 24873 raft_consensus.cc:504] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.360415 24492 tablet_bootstrap.cc:492] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:12.367587 24873 leader_election.cc:288] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:12.368551 24296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638ffc5970704a53a26967c81af23e52" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:12.372354 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638ffc5970704a53a26967c81af23e52" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:12.369722 24201 catalog_manager.cc:5495] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:12.373430 24297 raft_consensus.cc:2454] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:12.374529 24418 leader_election.cc:302] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:12.375371 24869 raft_consensus.cc:2781] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:12.375857 24869 raft_consensus.cc:686] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:12.377059 24494 tablet_bootstrap.cc:492] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144: No bootstrap required, opened a new log
I20260924 14:28:12.376634 24869 consensus_queue.cc:237] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.377437 24493 tablet_bootstrap.cc:654] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.377522 24494 ts_tablet_manager.cc:1397] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144: Time spent bootstrapping tablet: real 0.173s	user 0.015s	sys 0.009s
I20260924 14:28:12.352495 24918 leader_election.cc:288] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:12.381793 24344 leader_election.cc:302] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:12.383040 24872 raft_consensus.cc:2726] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:12.381623 24494 raft_consensus.cc:348] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.384652 24494 raft_consensus.cc:374] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.384959 24494 raft_consensus.cc:729] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.385703 24494 consensus_queue.cc:260] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.387722 24771 raft_consensus.cc:2726] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:12.391181 24494 ts_tablet_manager.cc:1428] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260924 14:28:12.400444 24201 catalog_manager.cc:5495] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:12.404594 24201 catalog_manager.cc:5495] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:12.427312 24492 tablet_bootstrap.cc:654] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.448261 24493 tablet_bootstrap.cc:492] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:12.448724 24493 ts_tablet_manager.cc:1397] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.185s	user 0.022s	sys 0.021s
I20260924 14:28:12.451155 24493 raft_consensus.cc:348] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.451907 24493 raft_consensus.cc:374] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.452205 24493 raft_consensus.cc:729] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.453097 24492 tablet_bootstrap.cc:492] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:12.452883 24493 consensus_queue.cc:260] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.453544 24492 ts_tablet_manager.cc:1397] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.003s
I20260924 14:28:12.455004 24493 ts_tablet_manager.cc:1428] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:28:12.455981 24493 tablet_bootstrap.cc:492] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:12.465262 24493 tablet_bootstrap.cc:654] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.467936 24198 catalog_manager.cc:5495] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:12.463434 24492 raft_consensus.cc:348] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.471832 24492 raft_consensus.cc:374] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.472246 24492 raft_consensus.cc:729] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.473209 24492 consensus_queue.cc:260] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.476147 24492 ts_tablet_manager.cc:1428] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260924 14:28:12.487635 24492 tablet_bootstrap.cc:492] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:12.495280 24492 tablet_bootstrap.cc:654] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.506335 24377 raft_consensus.cc:3037] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.509622 24869 consensus_queue.cc:1035] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:12.540830 24602 consensus_queue.cc:1035] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:12.544397 24492 tablet_bootstrap.cc:492] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:12.545395 24492 ts_tablet_manager.cc:1397] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.022s
I20260924 14:28:12.547868 24492 raft_consensus.cc:348] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.549149 24492 raft_consensus.cc:374] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.549463 24492 raft_consensus.cc:729] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.550226 24492 consensus_queue.cc:260] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.553083 24492 ts_tablet_manager.cc:1428] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260924 14:28:12.555006 24492 tablet_bootstrap.cc:492] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Bootstrap starting.
I20260924 14:28:12.561314 24492 tablet_bootstrap.cc:654] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.578660 24492 tablet_bootstrap.cc:492] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: No bootstrap required, opened a new log
I20260924 14:28:12.579253 24492 ts_tablet_manager.cc:1397] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.009s
I20260924 14:28:12.583093 24492 raft_consensus.cc:348] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.584038 24492 raft_consensus.cc:374] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.584370 24492 raft_consensus.cc:729] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.585170 24492 consensus_queue.cc:260] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.587720 24492 ts_tablet_manager.cc:1428] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260924 14:28:12.594712 24493 tablet_bootstrap.cc:492] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:12.595227 24493 ts_tablet_manager.cc:1397] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.139s	user 0.013s	sys 0.055s
I20260924 14:28:12.596266 24602 raft_consensus.cc:482] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:12.596704 24602 raft_consensus.cc:504] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.597795 24493 raft_consensus.cc:348] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.598565 24493 raft_consensus.cc:374] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.598816 24493 raft_consensus.cc:729] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.598989 24602 leader_election.cc:288] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:12.598903 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a57dc14600c450485f596239b30df0d" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
W20260924 14:28:12.599795 24268 consensus_peers.cc:487] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:12.599560 24493 consensus_queue.cc:260] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.601084 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a57dc14600c450485f596239b30df0d" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:12.601673 24493 ts_tablet_manager.cc:1428] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:28:12.602655 24493 tablet_bootstrap.cc:492] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:12.603519 24417 leader_election.cc:302] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:12.604189 24602 raft_consensus.cc:2726] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:12.608770 24493 tablet_bootstrap.cc:654] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.618745 24916 raft_consensus.cc:482] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:12.619205 24916 raft_consensus.cc:504] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.621208 24916 leader_election.cc:288] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:12.622074 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:12.622347 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:12.622916 24493 tablet_bootstrap.cc:492] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:12.622918 24300 raft_consensus.cc:2379] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61d316eb5dfe405c95019383bbebecaa in current term 1: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:12.623370 24493 ts_tablet_manager.cc:1397] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.005s
I20260924 14:28:12.624262 24344 leader_election.cc:302] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:12.624729 24446 raft_consensus.cc:3037] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.625221 24916 raft_consensus.cc:3037] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.626214 24493 raft_consensus.cc:348] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.626971 24493 raft_consensus.cc:374] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.627251 24493 raft_consensus.cc:729] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.627893 24493 consensus_queue.cc:260] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.630040 24493 ts_tablet_manager.cc:1428] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:28:12.630928 24493 tablet_bootstrap.cc:492] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:12.633944 24916 raft_consensus.cc:2726] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:12.634631 24727 consensus_queue.cc:1035] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:12.636426 24493 tablet_bootstrap.cc:654] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.659107 24493 tablet_bootstrap.cc:492] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:12.659659 24493 ts_tablet_manager.cc:1397] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.010s
I20260924 14:28:12.662078 24493 raft_consensus.cc:348] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.662911 24493 raft_consensus.cc:374] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.663213 24493 raft_consensus.cc:729] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.663859 24493 consensus_queue.cc:260] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.666183 24493 ts_tablet_manager.cc:1428] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260924 14:28:12.667090 24493 tablet_bootstrap.cc:492] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:12.668576 24602 raft_consensus.cc:482] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:12.669062 24602 raft_consensus.cc:504] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.672412 24493 tablet_bootstrap.cc:654] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.672008 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234eb8cdbf4c4e7599789194a0032c00" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:12.673554 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234eb8cdbf4c4e7599789194a0032c00" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:12.674070 24300 raft_consensus.cc:2454] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:12.674105 24370 raft_consensus.cc:2454] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:12.676290 24417 leader_election.cc:302] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:12.677198 24919 raft_consensus.cc:2781] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:12.677541 24919 raft_consensus.cc:482] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:12.677855 24919 raft_consensus.cc:3037] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.688282 24861 consensus_queue.cc:1035] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:12.689848 24919 raft_consensus.cc:504] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.701920 24602 leader_election.cc:288] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:12.708916 24919 leader_election.cc:288] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:12.709923 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234eb8cdbf4c4e7599789194a0032c00" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:12.709815 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "234eb8cdbf4c4e7599789194a0032c00" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:12.710711 24300 raft_consensus.cc:3037] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.710711 24370 raft_consensus.cc:3037] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.715377 24493 tablet_bootstrap.cc:492] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:12.715863 24493 ts_tablet_manager.cc:1397] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.033s
I20260924 14:28:12.718183 24493 raft_consensus.cc:348] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.718866 24493 raft_consensus.cc:374] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.719115 24493 raft_consensus.cc:729] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.719714 24493 consensus_queue.cc:260] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.721817 24493 ts_tablet_manager.cc:1428] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:28:12.722751 24493 tablet_bootstrap.cc:492] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa: Bootstrap starting.
I20260924 14:28:12.723762 24727 consensus_queue.cc:1035] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:12.725064 24727 raft_consensus.cc:482] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:12.724984 24370 raft_consensus.cc:2454] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:12.725567 24727 raft_consensus.cc:504] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.726023 24300 raft_consensus.cc:2454] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:12.727197 24418 leader_election.cc:302] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:12.728317 24493 tablet_bootstrap.cc:654] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:12.728905 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e838ce343bad4d42aa930f68fe4a4b2c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:12.729715 24370 raft_consensus.cc:2454] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:12.730036 24602 raft_consensus.cc:2781] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:12.730518 24602 raft_consensus.cc:686] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:12.730751 24268 leader_election.cc:302] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:12.731237 24602 consensus_queue.cc:237] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.731840 24860 raft_consensus.cc:2781] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:12.732527 24860 raft_consensus.cc:482] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:12.733022 24860 raft_consensus.cc:3037] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.735623 24727 leader_election.cc:288] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:12.737059 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e838ce343bad4d42aa930f68fe4a4b2c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:12.737730 24446 raft_consensus.cc:2454] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 0.
I20260924 14:28:12.740407 24860 raft_consensus.cc:504] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.744801 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e838ce343bad4d42aa930f68fe4a4b2c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:12.745481 24370 raft_consensus.cc:3037] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.747001 24860 leader_election.cc:288] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:12.747225 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e838ce343bad4d42aa930f68fe4a4b2c" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:12.750815 24446 raft_consensus.cc:3037] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.766426 24602 raft_consensus.cc:482] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:12.766920 24602 raft_consensus.cc:504] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.769615 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e4953bd67e40f5823697e67d029f52" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:12.770294 24300 raft_consensus.cc:2454] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:12.771404 24418 leader_election.cc:302] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:12.772090 24918 raft_consensus.cc:2781] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:12.772411 24602 leader_election.cc:288] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:12.772454 24918 raft_consensus.cc:482] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:12.773562 24918 raft_consensus.cc:3037] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.773453 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e4953bd67e40f5823697e67d029f52" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:12.774089 24377 raft_consensus.cc:2454] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 0.
I20260924 14:28:12.778726 24918 raft_consensus.cc:504] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.759438 24446 raft_consensus.cc:2454] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:12.785298 24268 leader_election.cc:302] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:12.786350 24727 raft_consensus.cc:2781] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:12.788203 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e4953bd67e40f5823697e67d029f52" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:12.788721 24377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e4953bd67e40f5823697e67d029f52" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:12.789309 24377 raft_consensus.cc:3037] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.788743 24300 raft_consensus.cc:3037] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.800055 24300 raft_consensus.cc:2454] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:12.801829 24418 leader_election.cc:302] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:12.803416 24602 raft_consensus.cc:2781] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:12.805765 24602 raft_consensus.cc:686] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:12.806115 24198 catalog_manager.cc:5495] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:12.798768 24727 raft_consensus.cc:686] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:12.808568 24727 consensus_queue.cc:237] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.806881 24602 consensus_queue.cc:237] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.791030 24918 leader_election.cc:288] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 1 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:12.813235 24377 raft_consensus.cc:2454] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:12.816593 24370 raft_consensus.cc:2454] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 1.
I20260924 14:28:12.818307 24493 tablet_bootstrap.cc:492] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa: No bootstrap required, opened a new log
I20260924 14:28:12.819031 24493 ts_tablet_manager.cc:1397] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa: Time spent bootstrapping tablet: real 0.096s	user 0.025s	sys 0.046s
I20260924 14:28:12.822366 24493 raft_consensus.cc:348] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.823307 24493 raft_consensus.cc:374] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:12.823668 24493 raft_consensus.cc:729] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Initialized, Role: FOLLOWER
I20260924 14:28:12.824597 24493 consensus_queue.cc:260] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.827461 24493 ts_tablet_manager.cc:1428] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260924 14:28:12.844580 24370 raft_consensus.cc:3037] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:12.845221 24198 catalog_manager.cc:5495] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:12.847478 24200 catalog_manager.cc:5495] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:12.857599 24916 raft_consensus.cc:482] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:12.858206 24918 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:12.858055 24916 raft_consensus.cc:504] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.863731 24918 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:12.865609 24916 leader_election.cc:288] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:12.866492 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf594a126774d39a57f354d4fc7d338" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:12.867226 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf594a126774d39a57f354d4fc7d338" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144" is_pre_election: true
I20260924 14:28:12.868182 24300 raft_consensus.cc:2454] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:12.869289 24344 leader_election.cc:302] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:12.870064 24872 raft_consensus.cc:2781] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:12.870421 24872 raft_consensus.cc:482] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:12.865074 24602 consensus_queue.cc:1035] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260924 14:28:12.870743 24872 raft_consensus.cc:3037] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:12.905668 24872 raft_consensus.cc:504] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.907625 24872 leader_election.cc:288] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:12.909363 24370 raft_consensus.cc:1226] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Rejecting Update request from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260924 14:28:12.909861 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf594a126774d39a57f354d4fc7d338" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:12.910046 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf594a126774d39a57f354d4fc7d338" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:12.915911 24300 raft_consensus.cc:3037] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:12.921137 24300 raft_consensus.cc:2454] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:12.921342 24918 consensus_queue.cc:1046] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:12.922145 24344 leader_election.cc:302] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:12.923167 24872 raft_consensus.cc:2781] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:12.923242 24297 raft_consensus.cc:1226] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Rejecting Update request from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 for earlier term 1. Current term is 2. Ops: []
I20260924 14:28:12.923976 24872 raft_consensus.cc:686] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:12.925056 24872 consensus_queue.cc:237] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.931840 24919 raft_consensus.cc:3032] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:12.932238 24919 raft_consensus.cc:729] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:12.933058 24919 consensus_queue.cc:260] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:12.934015 24919 raft_consensus.cc:3037] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:12.953918 24200 catalog_manager.cc:5495] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 1 to 2, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:12.987305 24370 raft_consensus.cc:3037] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:13.187415 24918 consensus_queue.cc:1035] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:13.196599 24919 consensus_queue.cc:1035] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:13.230667 24919 consensus_queue.cc:1035] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:13.240509 24918 consensus_queue.cc:1035] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:13.275180 24727 consensus_queue.cc:1035] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:13.288378 24861 consensus_queue.cc:1035] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:13.305832 24446 raft_consensus.cc:1261] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260924 14:28:13.306280 24446 pending_rounds.cc:76] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Aborting all ops after (but not including) 0
I20260924 14:28:13.306529 24446 pending_rounds.cc:97] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260924 14:28:13.306820 24446 raft_consensus.cc:2866] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:28:13.308374 24873 consensus_queue.cc:1035] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:13.317272 24296 raft_consensus.cc:1261] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260924 14:28:13.319399 24872 consensus_queue.cc:1035] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:13.368292 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:28:13.375478 24967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:13.401988 24970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:13.402055 24968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:13.402825 23135 server_base.cc:1034] running on GCE node
I20260924 14:28:13.405205 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:28:13.405467 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:28:13.405649 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260093405634 us; error 0 us; skew 500 ppm
I20260924 14:28:13.406208 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:28:13.409438 23135 webserver.cc:460] Webserver started at http://127.22.151.196:34325/ using document root <none> and password file <none>
I20260924 14:28:13.410117 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:28:13.410447 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:28:13.410904 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:28:13.412683 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-3-root/instance:
uuid: "b8062fa80b5f415e991884a029861641"
format_stamp: "Formatted at 2026-09-24 14:28:13 on dist-test-slave-pm5x"
I20260924 14:28:13.420601 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.006s	user 0.001s	sys 0.004s
I20260924 14:28:13.431111 24975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:13.432356 23135 fs_manager.cc:725] Time spent opening block manager: real 0.010s	user 0.008s	sys 0.002s
I20260924 14:28:13.432739 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-3-root
uuid: "b8062fa80b5f415e991884a029861641"
format_stamp: "Formatted at 2026-09-24 14:28:13 on dist-test-slave-pm5x"
I20260924 14:28:13.433174 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:28:13.449293 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:28:13.450594 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:28:13.452126 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:28:13.454875 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:28:13.455132 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:13.455420 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:28:13.455673 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.001s
I20260924 14:28:13.566717 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.196:41987
I20260924 14:28:13.566814 25037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.196:41987 every 8 connection(s)
I20260924 14:28:13.572553 23135 auto_leader_rebalancer-test.cc:413] add a tserver: b8062fa80b5f415e991884a029861641
I20260924 14:28:13.589052 25038 heartbeater.cc:347] Connected to a master server at 127.22.151.254:36341
I20260924 14:28:13.589540 25038 heartbeater.cc:464] Registering TS with master...
I20260924 14:28:13.590282 25038 heartbeater.cc:511] Master 127.22.151.254:36341 requested a full tablet report, sending...
I20260924 14:28:13.592521 24198 ts_manager.cc:194] Registered new tserver with Master: b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:13.594223 24198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52698
I20260924 14:28:14.108270 24446 consensus_queue.cc:237] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:14.116698 24370 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:14.118275 24919 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:14.119390 24296 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:14.120872 24919 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:14.131386 24602 raft_consensus.cc:2932] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:14.137429 24418 consensus_peers.cc:487] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 568b498fb63e4638b92d175b8d122c0e. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:14.137380 24370 raft_consensus.cc:2932] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.156262 24198 catalog_manager.cc:5495] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.158959 24297 raft_consensus.cc:2932] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.170707 24371 consensus_queue.cc:237] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:14.186810 24444 raft_consensus.cc:1261] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:14.188755 24872 consensus_queue.cc:1035] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:14.190692 24297 raft_consensus.cc:1261] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:14.191977 24872 consensus_queue.cc:1035] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:14.210491 24872 raft_consensus.cc:2932] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.212381 24444 raft_consensus.cc:2932] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.218221 24297 raft_consensus.cc:2932] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:14.219379 24344 consensus_peers.cc:487] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5caac6840188451a8e325a37b8ea0e3c. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:14.232198 24371 consensus_queue.cc:237] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:14.246340 24198 catalog_manager.cc:5495] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.264784 24297 raft_consensus.cc:1261] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:14.265141 24444 raft_consensus.cc:1261] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:14.266324 24344 consensus_peers.cc:487] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2bdd9163fb2d44068b0f93f1bbfd7387. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:14.267014 24872 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:14.269409 24771 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:14.281005 24872 raft_consensus.cc:2932] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.283133 24444 raft_consensus.cc:2932] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.283807 24297 raft_consensus.cc:2932] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.301463 24371 consensus_queue.cc:237] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:14.312461 24198 catalog_manager.cc:5495] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.324049 24444 raft_consensus.cc:1261] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:14.325235 24344 consensus_peers.cc:487] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c775016b57f24851b9c52bd2d708b4ef. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:14.326234 24771 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:14.344224 24297 raft_consensus.cc:1261] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:14.353477 24872 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:14.368952 24872 raft_consensus.cc:2932] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.373963 24297 raft_consensus.cc:2932] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.375844 24444 raft_consensus.cc:2932] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.393628 24198 catalog_manager.cc:5495] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:14.598372 25038 heartbeater.cc:503] Master 127.22.151.254:36341 was elected leader, sending a full tablet report...
I20260924 14:28:14.670166 25067 ts_tablet_manager.cc:927] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:14.672259 25067 tablet_copy_client.cc:316] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:14.692945 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet 568b498fb63e4638b92d175b8d122c0e from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:14.693754 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet 568b498fb63e4638b92d175b8d122c0e from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-568b498fb63e4638b92d175b8d122c0e
I20260924 14:28:14.702150 24462 tablet_copy_source_session.cc:205] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:14.706825 25067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 568b498fb63e4638b92d175b8d122c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:14.722815 25067 tablet_copy_client.cc:791] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:14.723527 25067 tablet_copy_client.cc:655] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:14.730034 25067 tablet_copy_client.cc:531] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:14.739827 25067 tablet_bootstrap.cc:492] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:14.744874 25071 ts_tablet_manager.cc:927] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:14.759377 25074 ts_tablet_manager.cc:927] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:14.762606 25074 tablet_copy_client.cc:316] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:14.782358 25077 ts_tablet_manager.cc:927] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:14.792258 25067 tablet_bootstrap.cc:492] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:14.793450 25071 tablet_copy_client.cc:316] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:14.793532 24388 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet 5caac6840188451a8e325a37b8ea0e3c from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:14.796797 24388 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet 5caac6840188451a8e325a37b8ea0e3c from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-5caac6840188451a8e325a37b8ea0e3c
I20260924 14:28:14.797598 25067 tablet_bootstrap.cc:492] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:14.798766 25067 ts_tablet_manager.cc:1397] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.059s	user 0.024s	sys 0.027s
I20260924 14:28:14.799810 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet c775016b57f24851b9c52bd2d708b4ef from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:14.800503 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet c775016b57f24851b9c52bd2d708b4ef from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-c775016b57f24851b9c52bd2d708b4ef
I20260924 14:28:14.824170 25077 tablet_copy_client.cc:316] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:14.827270 24386 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet 2bdd9163fb2d44068b0f93f1bbfd7387 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:14.828470 24386 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet 2bdd9163fb2d44068b0f93f1bbfd7387 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-2bdd9163fb2d44068b0f93f1bbfd7387
I20260924 14:28:14.826581 25067 raft_consensus.cc:348] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:14.829981 25067 raft_consensus.cc:729] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:14.838632 24386 tablet_copy_source_session.cc:205] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:14.839191 24388 tablet_copy_source_session.cc:205] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:14.840636 25067 consensus_queue.cc:260] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:14.845114 25077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bdd9163fb2d44068b0f93f1bbfd7387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:14.850659 24387 tablet_copy_source_session.cc:205] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:14.855149 25071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c775016b57f24851b9c52bd2d708b4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:14.860021 25074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5caac6840188451a8e325a37b8ea0e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:14.862602 25067 ts_tablet_manager.cc:1428] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.063s	user 0.005s	sys 0.011s
I20260924 14:28:14.865497 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-568b498fb63e4638b92d175b8d122c0e received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:14.865964 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-568b498fb63e4638b92d175b8d122c0e on tablet 568b498fb63e4638b92d175b8d122c0e with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:14.950507 25074 tablet_copy_client.cc:791] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:14.951107 25074 tablet_copy_client.cc:655] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:14.954977 25077 tablet_copy_client.cc:791] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:14.955605 25077 tablet_copy_client.cc:655] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:14.958141 25071 tablet_copy_client.cc:791] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:14.958691 25071 tablet_copy_client.cc:655] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:14.959707 25074 tablet_copy_client.cc:531] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:14.963594 25071 tablet_copy_client.cc:531] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:14.961858 25077 tablet_copy_client.cc:531] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:14.974082 25074 tablet_bootstrap.cc:492] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:14.975970 25077 tablet_bootstrap.cc:492] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:14.977674 25071 tablet_bootstrap.cc:492] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:15.015842 25074 tablet_bootstrap.cc:492] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:15.016783 25074 tablet_bootstrap.cc:492] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:15.017503 25074 ts_tablet_manager.cc:1397] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.044s	user 0.021s	sys 0.017s
I20260924 14:28:15.019181 25077 tablet_bootstrap.cc:492] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:15.020009 25077 tablet_bootstrap.cc:492] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:15.020738 25077 ts_tablet_manager.cc:1397] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.017s
I20260924 14:28:15.021499 25074 raft_consensus.cc:348] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:15.022485 25074 raft_consensus.cc:729] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:15.023164 25074 consensus_queue.cc:260] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:15.023490 25077 raft_consensus.cc:348] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:15.024662 25077 raft_consensus.cc:729] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:15.025679 25077 consensus_queue.cc:260] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:15.027904 25071 tablet_bootstrap.cc:492] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:15.028625 25077 ts_tablet_manager.cc:1428] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260924 14:28:15.029588 25071 tablet_bootstrap.cc:492] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:15.030292 25071 ts_tablet_manager.cc:1397] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.053s	user 0.018s	sys 0.019s
I20260924 14:28:15.032330 24388 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-2bdd9163fb2d44068b0f93f1bbfd7387 received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:15.032954 24388 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-2bdd9163fb2d44068b0f93f1bbfd7387 on tablet 2bdd9163fb2d44068b0f93f1bbfd7387 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:15.033593 25071 raft_consensus.cc:348] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:15.034814 25071 raft_consensus.cc:729] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:15.035475 25071 consensus_queue.cc:260] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:15.038290 25074 ts_tablet_manager.cc:1428] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.003s
I20260924 14:28:15.039503 25071 ts_tablet_manager.cc:1428] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.006s
I20260924 14:28:15.040889 24388 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-5caac6840188451a8e325a37b8ea0e3c received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:15.041594 24388 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-5caac6840188451a8e325a37b8ea0e3c on tablet 5caac6840188451a8e325a37b8ea0e3c with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:15.042382 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-c775016b57f24851b9c52bd2d708b4ef received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:15.043020 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-c775016b57f24851b9c52bd2d708b4ef on tablet c775016b57f24851b9c52bd2d708b4ef with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:15.203333 25012 raft_consensus.cc:1203] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:15.226253 25012 raft_consensus.cc:1203] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:15.243043 25012 raft_consensus.cc:1203] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:15.335547 25012 raft_consensus.cc:1203] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:15.626286 24918 raft_consensus.cc:1050] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:15.628695 24918 consensus_queue.cc:237] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:15.632261 24872 raft_consensus.cc:1050] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:15.634605 24872 consensus_queue.cc:237] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:15.637517 25012 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.638363 24371 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.639011 24300 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.640215 24602 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:15.642127 24602 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:15.644224 25083 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:15.658893 25013 raft_consensus.cc:1261] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.659030 24444 raft_consensus.cc:1261] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.660475 24771 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:15.664060 24771 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:15.689114 24300 raft_consensus.cc:1261] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.693164 24602 raft_consensus.cc:2932] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.694860 24297 raft_consensus.cc:2932] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.695201 24444 raft_consensus.cc:2932] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.697505 24771 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:15.699293 25042 raft_consensus.cc:2932] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.713408 25012 raft_consensus.cc:2932] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.716696 24376 raft_consensus.cc:2932] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.714394 24201 catalog_manager.cc:5495] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.727141 24771 raft_consensus.cc:1050] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:15.729580 24771 consensus_queue.cc:237] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:15.734112 24200 catalog_manager.cc:5495] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.746251 25012 raft_consensus.cc:2932] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.790230 25012 raft_consensus.cc:1261] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.790958 24297 raft_consensus.cc:1261] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.771482 24300 raft_consensus.cc:2932] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.792641 25097 consensus_queue.cc:1035] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:15.795876 24446 raft_consensus.cc:1261] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.815881 24771 consensus_queue.cc:1035] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:15.820216 25097 consensus_queue.cc:1035] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260924 14:28:15.873878 25012 raft_consensus.cc:2932] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.872565 25090 raft_consensus.cc:2932] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.881043 24449 consensus_queue.cc:237] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:15.892828 25013 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:15.894863 24444 raft_consensus.cc:2932] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.899904 25087 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:15.900959 24300 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:15.903249 25087 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:15.912664 24297 raft_consensus.cc:2932] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.913090 25086 raft_consensus.cc:2932] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.920537 25042 raft_consensus.cc:1050] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:15.924160 25042 consensus_queue.cc:237] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:15.923527 25013 raft_consensus.cc:2932] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.929951 24199 catalog_manager.cc:5495] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.942920 24297 raft_consensus.cc:2932] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:15.944483 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 568b498fb63e4638b92d175b8d122c0e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:15.978040 25013 raft_consensus.cc:1261] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.980473 24297 raft_consensus.cc:1261] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.983839 24446 raft_consensus.cc:1261] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:15.985002 24771 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:15.985924 25090 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:15.988480 25042 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:16.009089 24371 consensus_queue.cc:237] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:16.006139 24200 catalog_manager.cc:5495] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.033663 25090 raft_consensus.cc:2932] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.035295 24444 raft_consensus.cc:2932] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.041021 24449 raft_consensus.cc:1261] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:16.044607 24872 consensus_queue.cc:1035] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:16.057152 24300 raft_consensus.cc:2932] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.057817 25106 raft_consensus.cc:2932] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.059087 25013 raft_consensus.cc:1261] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:16.060115 25012 raft_consensus.cc:2932] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.075695 24447 raft_consensus.cc:2932] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.091025 24200 catalog_manager.cc:5495] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.091791 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet 568b498fb63e4638b92d175b8d122c0e with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:16.107664 25097 consensus_queue.cc:1035] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:16.110481 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 5caac6840188451a8e325a37b8ea0e3c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:16.139936 25111 tablet_replica.cc:330] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:16.168198 25012 raft_consensus.cc:2932] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.173689 25111 raft_consensus.cc:2229] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:16.174412 25111 raft_consensus.cc:2258] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:16.175495 24370 consensus_queue.cc:237] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:16.180781 25111 ts_tablet_manager.cc:1905] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:16.195526 25013 raft_consensus.cc:1261] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:16.195475 24198 catalog_manager.cc:5495] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.200440 25116 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:16.201917 24296 raft_consensus.cc:1261] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:16.208060 25116 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:16.215031 25111 ts_tablet_manager.cc:1918] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:16.216095 25111 log.cc:1196] T 568b498fb63e4638b92d175b8d122c0e P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/568b498fb63e4638b92d175b8d122c0e
I20260924 14:28:16.218412 25112 raft_consensus.cc:2932] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.221772 25013 raft_consensus.cc:2932] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.224145 24296 raft_consensus.cc:2932] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.220758 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet 568b498fb63e4638b92d175b8d122c0e (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:16.229959 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet 5caac6840188451a8e325a37b8ea0e3c with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:16.232731 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 2bdd9163fb2d44068b0f93f1bbfd7387 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:16.240509 24370 consensus_queue.cc:237] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:16.246032 24198 catalog_manager.cc:5495] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.249871 25013 raft_consensus.cc:1261] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:16.250191 24300 raft_consensus.cc:1261] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:16.254304 25107 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:16.261574 25115 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260924 14:28:16.263075 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bdd9163fb2d44068b0f93f1bbfd7387 with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:16.283798 25114 raft_consensus.cc:2932] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.286284 25013 raft_consensus.cc:2932] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.289053 25118 tablet_replica.cc:330] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:16.287285 25117 tablet_replica.cc:330] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:16.290999 25117 raft_consensus.cc:2229] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:16.292541 25118 raft_consensus.cc:2229] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:16.293182 25118 raft_consensus.cc:2258] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:16.293889 25117 raft_consensus.cc:2258] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:16.297163 25118 ts_tablet_manager.cc:1905] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:16.301316 25117 ts_tablet_manager.cc:1905] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:16.306682 24296 raft_consensus.cc:2932] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.308539 24199 catalog_manager.cc:5495] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:16.314424 25118 ts_tablet_manager.cc:1918] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:16.314911 25118 log.cc:1196] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/2bdd9163fb2d44068b0f93f1bbfd7387
I20260924 14:28:16.316085 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet c775016b57f24851b9c52bd2d708b4ef with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:16.316787 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet 2bdd9163fb2d44068b0f93f1bbfd7387 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:16.324656 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet c775016b57f24851b9c52bd2d708b4ef with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:16.326036 25118 tablet_replica.cc:330] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:16.327010 25118 raft_consensus.cc:2229] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:16.327728 25118 raft_consensus.cc:2258] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:16.330986 25118 ts_tablet_manager.cc:1905] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:16.355190 25118 ts_tablet_manager.cc:1918] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:16.355650 25118 log.cc:1196] T c775016b57f24851b9c52bd2d708b4ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/c775016b57f24851b9c52bd2d708b4ef
I20260924 14:28:16.357223 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet c775016b57f24851b9c52bd2d708b4ef (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:16.361752 25117 ts_tablet_manager.cc:1918] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:16.362186 25117 log.cc:1196] T 5caac6840188451a8e325a37b8ea0e3c P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/5caac6840188451a8e325a37b8ea0e3c
I20260924 14:28:16.363965 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet 5caac6840188451a8e325a37b8ea0e3c (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:17.402043 24444 consensus_queue.cc:237] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:17.408025 24297 raft_consensus.cc:1261] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:17.408305 24418 consensus_peers.cc:487] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c4bc71d12c14157982c98b4c7ca505c. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:17.408125 24371 raft_consensus.cc:1261] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:17.409451 25086 consensus_queue.cc:1035] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:17.410382 25122 consensus_queue.cc:1035] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:17.418743 24602 raft_consensus.cc:2932] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.420841 24297 raft_consensus.cc:2932] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.421063 24370 raft_consensus.cc:2932] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.439512 24198 catalog_manager.cc:5495] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.457944 24371 consensus_queue.cc:237] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:17.498116 24344 consensus_peers.cc:487] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37e78577921b4cf18418d98b6f6a30a1. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:17.502658 24297 raft_consensus.cc:1261] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:17.502725 24444 raft_consensus.cc:1261] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:17.504556 25116 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:17.505816 25121 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:17.531793 25116 raft_consensus.cc:2932] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.533488 24444 raft_consensus.cc:2932] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.535524 24297 raft_consensus.cc:2932] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.549571 24444 consensus_queue.cc:237] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:17.551478 24198 catalog_manager.cc:5495] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.560319 24370 raft_consensus.cc:1261] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:17.560766 24418 consensus_peers.cc:487] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce753b9bac814f17a869f4f8d66deca6. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:17.561501 24297 raft_consensus.cc:1261] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:17.561806 25128 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:17.562973 25127 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:17.572620 25129 raft_consensus.cc:2932] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.574707 24370 raft_consensus.cc:2932] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.578387 24297 raft_consensus.cc:2932] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.588557 24371 consensus_queue.cc:237] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:17.594262 24199 catalog_manager.cc:5495] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.607745 24444 raft_consensus.cc:1261] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:17.607972 24344 consensus_peers.cc:487] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e79eded71d354d4095b73f2f7f7153e5. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:17.609069 25107 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:17.610786 24297 raft_consensus.cc:1261] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:17.611986 25107 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:17.618609 25107 raft_consensus.cc:2932] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.620779 24444 raft_consensus.cc:2932] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.636351 24198 catalog_manager.cc:5495] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.639424 24300 raft_consensus.cc:2932] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:17.918339 25143 ts_tablet_manager.cc:927] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:17.920388 25143 tablet_copy_client.cc:316] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:17.921826 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet 2c4bc71d12c14157982c98b4c7ca505c from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:17.922303 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet 2c4bc71d12c14157982c98b4c7ca505c from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-2c4bc71d12c14157982c98b4c7ca505c
I20260924 14:28:17.928421 24462 tablet_copy_source_session.cc:205] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:17.932485 25143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c4bc71d12c14157982c98b4c7ca505c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:17.947757 25143 tablet_copy_client.cc:791] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:17.948406 25143 tablet_copy_client.cc:655] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:17.952620 25143 tablet_copy_client.cc:531] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:17.963083 25143 tablet_bootstrap.cc:492] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:17.981163 25143 tablet_bootstrap.cc:492] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:17.982065 25143 tablet_bootstrap.cc:492] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:17.982656 25143 ts_tablet_manager.cc:1397] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260924 14:28:17.985074 25143 raft_consensus.cc:348] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:17.985949 25143 raft_consensus.cc:729] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:17.986151 25146 ts_tablet_manager.cc:927] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:17.986609 25143 consensus_queue.cc:260] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:17.987860 25146 tablet_copy_client.cc:316] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:17.989512 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet ce753b9bac814f17a869f4f8d66deca6 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:17.990036 25143 ts_tablet_manager.cc:1428] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260924 14:28:17.990016 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet ce753b9bac814f17a869f4f8d66deca6 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-ce753b9bac814f17a869f4f8d66deca6
I20260924 14:28:17.997563 24461 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-2c4bc71d12c14157982c98b4c7ca505c received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:17.997818 24462 tablet_copy_source_session.cc:205] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:17.998006 24461 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-2c4bc71d12c14157982c98b4c7ca505c on tablet 2c4bc71d12c14157982c98b4c7ca505c with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:18.001437 25146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce753b9bac814f17a869f4f8d66deca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:18.014652 25146 tablet_copy_client.cc:791] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:18.015189 25146 tablet_copy_client.cc:655] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:18.018792 25146 tablet_copy_client.cc:531] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:18.031298 25146 tablet_bootstrap.cc:492] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:18.050608 25146 tablet_bootstrap.cc:492] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:18.051491 25146 tablet_bootstrap.cc:492] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:18.052150 25146 ts_tablet_manager.cc:1397] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.011s
I20260924 14:28:18.054755 25146 raft_consensus.cc:348] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:18.055500 25146 raft_consensus.cc:729] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:18.056182 25146 consensus_queue.cc:260] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:18.058203 25146 ts_tablet_manager.cc:1428] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:28:18.059643 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-ce753b9bac814f17a869f4f8d66deca6 received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:18.059993 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-ce753b9bac814f17a869f4f8d66deca6 on tablet ce753b9bac814f17a869f4f8d66deca6 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:18.066155 25146 ts_tablet_manager.cc:927] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:18.068782 25146 tablet_copy_client.cc:316] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:18.070568 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet 37e78577921b4cf18418d98b6f6a30a1 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:18.071034 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet 37e78577921b4cf18418d98b6f6a30a1 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-37e78577921b4cf18418d98b6f6a30a1
I20260924 14:28:18.077710 24387 tablet_copy_source_session.cc:205] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:18.080514 25146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e78577921b4cf18418d98b6f6a30a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:18.094535 25146 tablet_copy_client.cc:791] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:18.095093 25146 tablet_copy_client.cc:655] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:18.098343 25146 tablet_copy_client.cc:531] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:18.109447 25146 tablet_bootstrap.cc:492] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:18.126582 25146 tablet_bootstrap.cc:492] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:18.127409 25146 tablet_bootstrap.cc:492] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:18.128173 25146 ts_tablet_manager.cc:1397] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.004s
I20260924 14:28:18.130299 25146 raft_consensus.cc:348] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:18.130887 25146 raft_consensus.cc:729] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:18.131498 25146 consensus_queue.cc:260] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:18.133914 25146 ts_tablet_manager.cc:1428] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260924 14:28:18.135517 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-37e78577921b4cf18418d98b6f6a30a1 received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:18.135921 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-37e78577921b4cf18418d98b6f6a30a1 on tablet 37e78577921b4cf18418d98b6f6a30a1 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:18.218842 25146 ts_tablet_manager.cc:927] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:18.221741 25146 tablet_copy_client.cc:316] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:18.223220 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet e79eded71d354d4095b73f2f7f7153e5 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:18.223757 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet e79eded71d354d4095b73f2f7f7153e5 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-e79eded71d354d4095b73f2f7f7153e5
I20260924 14:28:18.231369 24387 tablet_copy_source_session.cc:205] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:18.234742 25146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e79eded71d354d4095b73f2f7f7153e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:18.249408 25146 tablet_copy_client.cc:791] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:18.249994 25146 tablet_copy_client.cc:655] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:18.254305 25146 tablet_copy_client.cc:531] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:18.266253 25146 tablet_bootstrap.cc:492] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:18.282840 25146 tablet_bootstrap.cc:492] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:18.283681 25146 tablet_bootstrap.cc:492] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:18.284247 25146 ts_tablet_manager.cc:1397] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.008s
I20260924 14:28:18.286468 25146 raft_consensus.cc:348] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:18.287252 25146 raft_consensus.cc:729] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:18.287854 25146 consensus_queue.cc:260] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:18.289772 25146 ts_tablet_manager.cc:1428] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260924 14:28:18.291615 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-e79eded71d354d4095b73f2f7f7153e5 received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:18.291980 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-e79eded71d354d4095b73f2f7f7153e5 on tablet e79eded71d354d4095b73f2f7f7153e5 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:18.383430 25013 raft_consensus.cc:1203] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:18.430030 25013 raft_consensus.cc:1203] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:18.602517 25013 raft_consensus.cc:1203] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:18.668751 25013 raft_consensus.cc:1203] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:18.889173 25129 raft_consensus.cc:1050] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:18.891710 25129 consensus_queue.cc:237] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:18.900871 24370 raft_consensus.cc:1261] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:18.903775 25127 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:18.914283 25013 raft_consensus.cc:1261] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:18.916577 24296 raft_consensus.cc:1261] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:18.918763 25129 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:18.920962 25127 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:18.922788 25154 raft_consensus.cc:1050] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:18.925455 25154 consensus_queue.cc:237] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:18.945488 24371 raft_consensus.cc:1261] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:18.947039 25154 consensus_queue.cc:1035] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:18.956173 25013 raft_consensus.cc:1261] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:18.955886 25129 raft_consensus.cc:2932] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:18.957562 24296 raft_consensus.cc:1261] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:18.957242 25012 raft_consensus.cc:2932] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:18.958397 24297 raft_consensus.cc:2932] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:18.967734 25159 consensus_queue.cc:1035] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:18.972206 24371 raft_consensus.cc:2932] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:18.979080 24199 catalog_manager.cc:5495] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:18.980258 25127 consensus_queue.cc:1035] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:19.006392 25158 raft_consensus.cc:2932] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.011570 25012 raft_consensus.cc:2932] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.023483 24297 raft_consensus.cc:2932] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.034870 24371 raft_consensus.cc:2932] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.042698 24198 catalog_manager.cc:5495] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.044296 24444 consensus_queue.cc:237] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:19.056901 25012 raft_consensus.cc:1261] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:19.057364 24296 raft_consensus.cc:1261] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:19.061584 25154 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:19.065455 25158 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:19.077342 25163 raft_consensus.cc:2932] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.085189 24449 consensus_queue.cc:237] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:19.086984 25121 raft_consensus.cc:1050] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:19.087416 25012 raft_consensus.cc:2932] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.089541 25121 consensus_queue.cc:237] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:19.093969 25013 raft_consensus.cc:1261] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:19.095801 24296 raft_consensus.cc:1261] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:19.095635 25127 consensus_queue.cc:1035] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:19.096926 24297 raft_consensus.cc:2932] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.102624 25127 consensus_queue.cc:1035] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:19.118261 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet ce753b9bac814f17a869f4f8d66deca6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:19.125856 25159 raft_consensus.cc:2932] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.138235 25013 raft_consensus.cc:2932] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.147300 24297 raft_consensus.cc:2932] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.144550 24198 catalog_manager.cc:5495] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:19.156935 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 2c4bc71d12c14157982c98b4c7ca505c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:19.161166 25013 raft_consensus.cc:1261] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:19.161605 24300 raft_consensus.cc:1261] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:19.162729 25155 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:19.163856 24444 raft_consensus.cc:1261] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:19.164806 25121 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:19.167254 25155 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:19.186388 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet ce753b9bac814f17a869f4f8d66deca6 with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:19.194021 24199 catalog_manager.cc:5495] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.215794 25176 tablet_replica.cc:330] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:19.220532 25155 raft_consensus.cc:2932] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.219147 25176 raft_consensus.cc:2229] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:19.224860 25176 raft_consensus.cc:2258] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:19.222841 24444 raft_consensus.cc:2932] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.229278 25176 ts_tablet_manager.cc:1905] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:19.239392 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c4bc71d12c14157982c98b4c7ca505c with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:19.245182 25013 raft_consensus.cc:2932] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.247502 24200 catalog_manager.cc:5495] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.251472 25176 ts_tablet_manager.cc:1918] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:19.252013 25176 log.cc:1196] T ce753b9bac814f17a869f4f8d66deca6 P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/ce753b9bac814f17a869f4f8d66deca6
I20260924 14:28:19.254383 25176 tablet_replica.cc:330] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:19.255491 25176 raft_consensus.cc:2229] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:19.256345 25176 raft_consensus.cc:2258] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:19.257566 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet ce753b9bac814f17a869f4f8d66deca6 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:19.260334 24296 raft_consensus.cc:2932] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.265972 25176 ts_tablet_manager.cc:1905] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:19.281841 25176 ts_tablet_manager.cc:1918] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:19.282227 25176 log.cc:1196] T 2c4bc71d12c14157982c98b4c7ca505c P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/2c4bc71d12c14157982c98b4c7ca505c
I20260924 14:28:19.284003 25178 raft_consensus.cc:1050] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:19.288739 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet 2c4bc71d12c14157982c98b4c7ca505c (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:19.289110 25178 consensus_queue.cc:237] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:19.293529 24376 consensus_queue.cc:237] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:19.298478 24446 raft_consensus.cc:1261] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:19.299750 24297 raft_consensus.cc:1261] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:19.299857 25013 raft_consensus.cc:1261] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:19.302189 25013 raft_consensus.cc:1261] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:19.302150 25107 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:19.304214 25107 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:19.306975 24446 raft_consensus.cc:1261] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:19.306965 25121 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260924 14:28:19.307109 25172 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260924 14:28:19.316797 25121 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:19.327365 24446 raft_consensus.cc:2932] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.325948 25107 raft_consensus.cc:2932] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.350059 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 37e78577921b4cf18418d98b6f6a30a1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:19.352100 25012 raft_consensus.cc:2932] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.353462 25107 raft_consensus.cc:2932] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.358557 24198 catalog_manager.cc:5495] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.367383 24297 raft_consensus.cc:2932] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.370021 24444 raft_consensus.cc:2932] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.372480 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet 37e78577921b4cf18418d98b6f6a30a1 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:19.385635 25013 raft_consensus.cc:2932] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.408373 24199 catalog_manager.cc:5495] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.427336 25184 tablet_replica.cc:330] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:19.441286 25184 raft_consensus.cc:2229] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:19.442162 25184 raft_consensus.cc:2258] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:19.441919 24376 consensus_queue.cc:237] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:19.446115 25184 ts_tablet_manager.cc:1905] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:19.447193 24444 raft_consensus.cc:1261] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:19.448575 25182 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:19.457886 25012 raft_consensus.cc:1261] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:19.467571 25172 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:19.473336 25107 raft_consensus.cc:2932] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.477720 25012 raft_consensus.cc:2932] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.482474 25184 ts_tablet_manager.cc:1918] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:19.482972 25184 log.cc:1196] T 37e78577921b4cf18418d98b6f6a30a1 P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/37e78577921b4cf18418d98b6f6a30a1
I20260924 14:28:19.484679 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet 37e78577921b4cf18418d98b6f6a30a1 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:19.488253 24447 raft_consensus.cc:2932] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.489231 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet e79eded71d354d4095b73f2f7f7153e5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:19.490160 24198 catalog_manager.cc:5495] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:19.500416 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet e79eded71d354d4095b73f2f7f7153e5 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:19.501397 25184 tablet_replica.cc:330] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:19.507931 25184 raft_consensus.cc:2229] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:19.508628 25184 raft_consensus.cc:2258] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:19.513622 25184 ts_tablet_manager.cc:1905] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:19.529141 25184 ts_tablet_manager.cc:1918] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:19.529436 25184 log.cc:1196] T e79eded71d354d4095b73f2f7f7153e5 P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/e79eded71d354d4095b73f2f7f7153e5
I20260924 14:28:19.530763 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet e79eded71d354d4095b73f2f7f7153e5 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:20.759565 24297 consensus_queue.cc:237] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:20.770042 24376 raft_consensus.cc:1261] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:20.770264 24446 raft_consensus.cc:1261] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:20.771595 24861 consensus_queue.cc:1035] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:20.772701 25188 consensus_queue.cc:1035] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:20.782711 25188 raft_consensus.cc:2932] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:20.788090 24376 raft_consensus.cc:2932] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:20.789783 24269 consensus_peers.cc:487] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b18bcddd405d4bbd9d39081d45429923. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:20.810859 24446 consensus_queue.cc:237] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:20.812306 24447 raft_consensus.cc:2932] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:20.812593 24198 catalog_manager.cc:5495] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:20.821086 24418 consensus_peers.cc:487] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9aa9c52e97d0444b8ec17719ea5a4ac6. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:20.823009 24376 raft_consensus.cc:1261] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:20.838028 25193 consensus_queue.cc:1035] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:20.848716 24296 raft_consensus.cc:1261] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:20.850353 25190 consensus_queue.cc:1035] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:20.852500 25193 raft_consensus.cc:2932] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:20.854106 24376 raft_consensus.cc:2932] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:20.864043 24296 raft_consensus.cc:2932] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:20.877274 24199 catalog_manager.cc:5495] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:20.898705 24447 consensus_queue.cc:237] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:20.924857 24418 consensus_peers.cc:487] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3822638d266742c4908907a61efe8231. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:20.925916 24376 raft_consensus.cc:1261] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:20.927568 25193 consensus_queue.cc:1035] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:20.928112 24296 raft_consensus.cc:1261] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:20.929906 25159 consensus_queue.cc:1035] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:20.940989 25193 raft_consensus.cc:2932] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:20.942646 24296 raft_consensus.cc:2932] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:20.945595 24376 raft_consensus.cc:2932] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:20.971138 24296 consensus_queue.cc:237] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:20.973399 24201 catalog_manager.cc:5495] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:20.993294 24376 raft_consensus.cc:1261] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:20.993633 24269 consensus_peers.cc:487] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4bf23ec7c761435d8352db2e38996e74. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:20.995173 24447 raft_consensus.cc:1261] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:20.995545 25188 consensus_queue.cc:1035] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:20.999348 25188 consensus_queue.cc:1035] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:21.011866 25156 raft_consensus.cc:2932] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:21.018147 24376 raft_consensus.cc:2932] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:21.018849 24447 raft_consensus.cc:2932] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:21.039199 24198 catalog_manager.cc:5495] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:21.221675 25214 ts_tablet_manager.cc:927] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:21.223640 25214 tablet_copy_client.cc:316] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:21.244565 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet b18bcddd405d4bbd9d39081d45429923 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:21.245217 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet b18bcddd405d4bbd9d39081d45429923 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-b18bcddd405d4bbd9d39081d45429923
I20260924 14:28:21.253809 24313 tablet_copy_source_session.cc:205] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:21.257467 25214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b18bcddd405d4bbd9d39081d45429923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:21.272290 25214 tablet_copy_client.cc:791] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:21.272951 25214 tablet_copy_client.cc:655] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:21.276530 25214 tablet_copy_client.cc:531] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:21.285596 25214 tablet_bootstrap.cc:492] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:21.308071 25214 tablet_bootstrap.cc:492] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:21.308920 25214 tablet_bootstrap.cc:492] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:21.309615 25214 ts_tablet_manager.cc:1397] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.004s
I20260924 14:28:21.312590 25214 raft_consensus.cc:348] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:21.313541 25214 raft_consensus.cc:729] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:21.314366 25214 consensus_queue.cc:260] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:21.317689 25214 ts_tablet_manager.cc:1428] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:28:21.319746 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-b18bcddd405d4bbd9d39081d45429923 received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:21.320153 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-b18bcddd405d4bbd9d39081d45429923 on tablet b18bcddd405d4bbd9d39081d45429923 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:21.338620 25214 ts_tablet_manager.cc:927] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:21.340287 25214 tablet_copy_client.cc:316] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:21.342267 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet 9aa9c52e97d0444b8ec17719ea5a4ac6 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:21.342893 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet 9aa9c52e97d0444b8ec17719ea5a4ac6 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-9aa9c52e97d0444b8ec17719ea5a4ac6
I20260924 14:28:21.352679 24462 tablet_copy_source_session.cc:205] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:21.356472 25214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa9c52e97d0444b8ec17719ea5a4ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:21.370852 25214 tablet_copy_client.cc:791] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:21.371423 25214 tablet_copy_client.cc:655] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:21.376415 25214 tablet_copy_client.cc:531] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:21.383699 25214 tablet_bootstrap.cc:492] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:21.401695 25214 tablet_bootstrap.cc:492] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:21.402582 25214 tablet_bootstrap.cc:492] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:21.403314 25214 ts_tablet_manager.cc:1397] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260924 14:28:21.406091 25214 raft_consensus.cc:348] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:21.406838 25214 raft_consensus.cc:729] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:21.407536 25214 consensus_queue.cc:260] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:21.411335 25214 ts_tablet_manager.cc:1428] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260924 14:28:21.413041 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-9aa9c52e97d0444b8ec17719ea5a4ac6 received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:21.413412 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-9aa9c52e97d0444b8ec17719ea5a4ac6 on tablet 9aa9c52e97d0444b8ec17719ea5a4ac6 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:21.453048 25214 ts_tablet_manager.cc:927] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:21.454864 25214 tablet_copy_client.cc:316] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:21.457036 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet 3822638d266742c4908907a61efe8231 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:21.457556 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet 3822638d266742c4908907a61efe8231 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-3822638d266742c4908907a61efe8231
I20260924 14:28:21.463924 24462 tablet_copy_source_session.cc:205] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:21.467489 25214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3822638d266742c4908907a61efe8231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:21.482751 25214 tablet_copy_client.cc:791] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:21.483348 25214 tablet_copy_client.cc:655] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:21.486708 25214 tablet_copy_client.cc:531] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:21.489789 25222 ts_tablet_manager.cc:927] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:21.497540 25214 tablet_bootstrap.cc:492] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:21.509564 25222 tablet_copy_client.cc:316] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:21.512784 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet 4bf23ec7c761435d8352db2e38996e74 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:21.513316 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet 4bf23ec7c761435d8352db2e38996e74 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-4bf23ec7c761435d8352db2e38996e74
I20260924 14:28:21.521909 25214 tablet_bootstrap.cc:492] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:21.522907 24313 tablet_copy_source_session.cc:205] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:21.522909 25214 tablet_bootstrap.cc:492] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:21.523932 25214 ts_tablet_manager.cc:1397] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.004s
I20260924 14:28:21.527166 25214 raft_consensus.cc:348] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:21.528223 25214 raft_consensus.cc:729] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:21.529657 25214 consensus_queue.cc:260] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:21.531847 25214 ts_tablet_manager.cc:1428] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260924 14:28:21.534099 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-3822638d266742c4908907a61efe8231 received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:21.534502 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-3822638d266742c4908907a61efe8231 on tablet 3822638d266742c4908907a61efe8231 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:21.539482 25222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf23ec7c761435d8352db2e38996e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:21.555573 25222 tablet_copy_client.cc:791] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:21.556223 25222 tablet_copy_client.cc:655] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:21.559324 25222 tablet_copy_client.cc:531] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:21.566802 25222 tablet_bootstrap.cc:492] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:21.585794 25222 tablet_bootstrap.cc:492] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:21.586681 25222 tablet_bootstrap.cc:492] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:21.587436 25222 ts_tablet_manager.cc:1397] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260924 14:28:21.590112 25222 raft_consensus.cc:348] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:21.590909 25222 raft_consensus.cc:729] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:21.591501 25222 consensus_queue.cc:260] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:21.593971 25222 ts_tablet_manager.cc:1428] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:28:21.595597 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-4bf23ec7c761435d8352db2e38996e74 received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:21.595978 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-4bf23ec7c761435d8352db2e38996e74 on tablet 4bf23ec7c761435d8352db2e38996e74 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:21.746434 25013 raft_consensus.cc:1203] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:21.817306 25012 raft_consensus.cc:1203] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:21.966315 25012 raft_consensus.cc:1203] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:21.994894 25012 raft_consensus.cc:1203] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:22.293452 25193 raft_consensus.cc:1050] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:22.296543 25193 consensus_queue.cc:237] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:22.305501 25012 raft_consensus.cc:1261] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.307193 25193 consensus_queue.cc:1035] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:22.341868 24297 raft_consensus.cc:1261] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.343554 25193 consensus_queue.cc:1035] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:22.362016 24371 raft_consensus.cc:1261] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.364063 25193 consensus_queue.cc:1035] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:22.497447 25154 raft_consensus.cc:2932] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:22.522912 25012 raft_consensus.cc:2932] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:22.530495 24861 raft_consensus.cc:1050] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:22.531569 25232 raft_consensus.cc:1050] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:22.533041 24861 consensus_queue.cc:237] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:22.534222 25232 consensus_queue.cc:237] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:22.550894 24303 raft_consensus.cc:2932] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:22.587919 25013 raft_consensus.cc:1261] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.616031 25188 consensus_queue.cc:1035] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260924 14:28:22.628672 25011 raft_consensus.cc:1261] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.626785 24198 catalog_manager.cc:5495] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:22.634677 25237 consensus_queue.cc:1035] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260924 14:28:22.652472 24417 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 96444.1us
W20260924 14:28:22.662710 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 73586.9us
I20260924 14:28:22.706753 24376 raft_consensus.cc:2932] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
W20260924 14:28:22.739265 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 113357us
I20260924 14:28:22.762972 24446 raft_consensus.cc:1261] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.766093 24371 raft_consensus.cc:1261] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.768057 24452 raft_consensus.cc:1261] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.773267 25242 consensus_queue.cc:1035] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:22.773839 24376 raft_consensus.cc:1261] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260924 14:28:22.775013 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 77142.5us
I20260924 14:28:22.776918 25242 consensus_queue.cc:1035] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:22.783159 25241 consensus_queue.cc:1035] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:22.785791 25232 consensus_queue.cc:1035] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:22.805447 25193 raft_consensus.cc:1050] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:22.808823 25193 consensus_queue.cc:237] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:22.841041 25248 raft_consensus.cc:2932] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:22.853034 24374 raft_consensus.cc:2932] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:22.858909 25013 raft_consensus.cc:1261] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.861241 24300 raft_consensus.cc:1261] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.864210 25159 consensus_queue.cc:1035] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:22.867817 25193 consensus_queue.cc:1035] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:22.880656 25013 raft_consensus.cc:2932] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:22.894703 24371 raft_consensus.cc:1261] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:22.900499 25250 consensus_queue.cc:1035] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260924 14:28:22.911175 25232 raft_consensus.cc:2932] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:22.920873 24377 raft_consensus.cc:2932] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:22.929576 24445 raft_consensus.cc:2932] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:22.940302 24452 raft_consensus.cc:2932] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:22.975533 24198 catalog_manager.cc:5495] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.002717 25013 raft_consensus.cc:2932] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.013388 25159 raft_consensus.cc:2932] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.020689 25011 raft_consensus.cc:2932] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.023298 24452 consensus_queue.cc:237] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:23.033598 24298 consensus_queue.cc:237] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:23.031371 24199 catalog_manager.cc:5495] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.041692 25012 raft_consensus.cc:1261] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:23.043612 24377 raft_consensus.cc:1261] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:23.057408 25012 raft_consensus.cc:1261] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:23.043221 25250 consensus_queue.cc:1035] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260924 14:28:23.058837 25252 consensus_queue.cc:1035] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:23.059538 25242 consensus_queue.cc:1035] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:23.059885 24377 raft_consensus.cc:2932] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.067715 24374 raft_consensus.cc:1261] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:23.074527 25188 consensus_queue.cc:1035] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:23.076766 25188 raft_consensus.cc:2932] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.075294 24303 raft_consensus.cc:2932] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.085386 25010 raft_consensus.cc:2932] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.089831 24298 consensus_queue.cc:237] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:23.091975 25011 raft_consensus.cc:2932] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.093743 24201 catalog_manager.cc:5495] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.089051 25250 raft_consensus.cc:2932] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.112602 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet b18bcddd405d4bbd9d39081d45429923 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:23.112951 24374 raft_consensus.cc:2932] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.123921 24377 raft_consensus.cc:1261] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:23.125641 25188 consensus_queue.cc:1035] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:23.132614 25012 raft_consensus.cc:1261] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:23.134104 25242 consensus_queue.cc:1035] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:23.136016 24377 raft_consensus.cc:2932] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.139176 24200 catalog_manager.cc:5495] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:23.143066 25248 raft_consensus.cc:2932] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.164652 25012 raft_consensus.cc:2932] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.177325 24199 catalog_manager.cc:5495] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.169899 24371 raft_consensus.cc:2932] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.179590 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 4bf23ec7c761435d8352db2e38996e74 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:23.182914 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet b18bcddd405d4bbd9d39081d45429923 with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:23.191788 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 9aa9c52e97d0444b8ec17719ea5a4ac6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:23.213949 24445 consensus_queue.cc:237] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:23.221313 25264 tablet_replica.cc:330] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:23.249621 25264 raft_consensus.cc:2229] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:23.251168 25264 raft_consensus.cc:2258] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:23.254685 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aa9c52e97d0444b8ec17719ea5a4ac6 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:23.261723 24297 raft_consensus.cc:1261] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:23.271699 25264 ts_tablet_manager.cc:1905] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:23.272557 25193 consensus_queue.cc:1035] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:23.277881 24199 catalog_manager.cc:5495] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.284825 25012 raft_consensus.cc:1261] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:23.290030 25264 ts_tablet_manager.cc:1918] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:23.290822 25264 log.cc:1196] T b18bcddd405d4bbd9d39081d45429923 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/b18bcddd405d4bbd9d39081d45429923
I20260924 14:28:23.288519 25193 consensus_queue.cc:1035] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260924 14:28:23.294137 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet b18bcddd405d4bbd9d39081d45429923 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:23.299707 25265 tablet_replica.cc:330] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:23.300868 25265 raft_consensus.cc:2229] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:23.307873 25265 raft_consensus.cc:2258] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:23.311074 25252 raft_consensus.cc:2932] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.313973 25010 raft_consensus.cc:2932] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.317557 24298 raft_consensus.cc:2932] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.325759 25265 ts_tablet_manager.cc:1905] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:23.345103 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 3822638d266742c4908907a61efe8231 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:23.347740 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bf23ec7c761435d8352db2e38996e74 with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:23.348734 25264 tablet_replica.cc:330] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:23.349757 25264 raft_consensus.cc:2229] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:23.350458 25264 raft_consensus.cc:2258] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:23.362883 24197 catalog_manager.cc:5495] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:23.368417 25265 ts_tablet_manager.cc:1918] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:23.369907 25264 ts_tablet_manager.cc:1905] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:23.370527 25265 log.cc:1196] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/9aa9c52e97d0444b8ec17719ea5a4ac6
I20260924 14:28:23.381831 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet 3822638d266742c4908907a61efe8231 with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:23.393127 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet 9aa9c52e97d0444b8ec17719ea5a4ac6 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:23.404227 25264 ts_tablet_manager.cc:1918] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:23.404924 25264 log.cc:1196] T 4bf23ec7c761435d8352db2e38996e74 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/4bf23ec7c761435d8352db2e38996e74
I20260924 14:28:23.406888 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet 4bf23ec7c761435d8352db2e38996e74 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:23.409715 25266 tablet_replica.cc:330] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:23.411156 25266 raft_consensus.cc:2229] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:23.412141 25266 raft_consensus.cc:2258] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:23.416967 25266 ts_tablet_manager.cc:1905] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260924 14:28:23.432926 24399 debug-util.cc:398] Leaking SignalData structure 0x7b080016efc0 after lost signal to thread 23138
W20260924 14:28:23.434069 24399 debug-util.cc:398] Leaking SignalData structure 0x7b080016b6e0 after lost signal to thread 24235
I20260924 14:28:23.435105 25266 ts_tablet_manager.cc:1918] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
W20260924 14:28:23.435287 24399 debug-util.cc:398] Leaking SignalData structure 0x7b0800043420 after lost signal to thread 24327
I20260924 14:28:23.435492 25266 log.cc:1196] T 3822638d266742c4908907a61efe8231 P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/3822638d266742c4908907a61efe8231
W20260924 14:28:23.436528 24399 debug-util.cc:398] Leaking SignalData structure 0x7b080024bf60 after lost signal to thread 24402
I20260924 14:28:23.437137 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet 3822638d266742c4908907a61efe8231 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
W20260924 14:28:23.438438 24399 debug-util.cc:398] Leaking SignalData structure 0x7b080025c8c0 after lost signal to thread 24476
W20260924 14:28:23.439677 24399 debug-util.cc:398] Leaking SignalData structure 0x7b080031ad40 after lost signal to thread 25037
I20260924 14:28:24.434123 24452 consensus_queue.cc:237] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:24.441387 24297 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:24.444080 25252 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:28:24.447286 24418 consensus_peers.cc:487] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4995099d467c4203bc2872f519b99950. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:24.450034 24371 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:24.451268 25159 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:24.454594 25252 raft_consensus.cc:2932] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.461359 24297 raft_consensus.cc:2932] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.460506 24371 raft_consensus.cc:2932] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.473001 24298 consensus_queue.cc:237] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:24.488897 24199 catalog_manager.cc:5495] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:24.528338 24269 consensus_peers.cc:487] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd02146c57f9474fba0afbcc145a7a2c. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:24.539017 24452 raft_consensus.cc:1261] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:24.544152 25267 consensus_queue.cc:1035] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:24.550575 24371 raft_consensus.cc:1261] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:24.552160 25267 consensus_queue.cc:1035] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:24.562794 25248 raft_consensus.cc:2932] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.567873 24445 raft_consensus.cc:2932] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.595772 24197 catalog_manager.cc:5495] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.597793 24298 consensus_queue.cc:237] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:24.608382 24371 raft_consensus.cc:2932] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:24.636793 24269 consensus_peers.cc:487] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b53acab36f9a42838efe3526f4aed03a. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:24.638638 24445 raft_consensus.cc:1261] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:24.638988 24374 raft_consensus.cc:1261] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:24.641489 25281 consensus_queue.cc:1035] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260924 14:28:24.643136 25248 consensus_queue.cc:1035] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:24.656364 25248 raft_consensus.cc:2932] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.657287 24445 raft_consensus.cc:2932] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.667140 24374 raft_consensus.cc:2932] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.680918 24371 consensus_queue.cc:237] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:24.701444 24197 catalog_manager.cc:5495] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260924 14:28:24.719043 24344 consensus_peers.cc:487] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9aebcc9d3ed438daf87d68bf2b8f6a9. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:24.719161 24445 raft_consensus.cc:1261] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:24.722747 25191 consensus_queue.cc:1035] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:24.723034 24298 raft_consensus.cc:1261] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:24.731932 25107 consensus_queue.cc:1035] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:24.764127 25107 raft_consensus.cc:2932] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.773115 24445 raft_consensus.cc:2932] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.783758 24197 catalog_manager.cc:5495] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:24.775198 24298 raft_consensus.cc:2932] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:24.949782 25287 ts_tablet_manager.cc:927] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:24.953783 25287 tablet_copy_client.cc:316] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:24.955516 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet dd02146c57f9474fba0afbcc145a7a2c from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:24.956067 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet dd02146c57f9474fba0afbcc145a7a2c from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-dd02146c57f9474fba0afbcc145a7a2c
I20260924 14:28:24.965127 24313 tablet_copy_source_session.cc:205] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:24.967857 25289 ts_tablet_manager.cc:927] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:24.971184 25289 tablet_copy_client.cc:316] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:24.972738 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet 4995099d467c4203bc2872f519b99950 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:24.973415 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet 4995099d467c4203bc2872f519b99950 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-4995099d467c4203bc2872f519b99950
I20260924 14:28:24.973655 25287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd02146c57f9474fba0afbcc145a7a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:24.984376 24462 tablet_copy_source_session.cc:205] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:24.993173 25287 tablet_copy_client.cc:791] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:24.993764 25287 tablet_copy_client.cc:655] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:24.995322 25289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4995099d467c4203bc2872f519b99950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:24.997983 25287 tablet_copy_client.cc:531] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:25.013357 25287 tablet_bootstrap.cc:492] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:25.040483 25291 ts_tablet_manager.cc:927] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:25.051782 25291 tablet_copy_client.cc:316] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:25.051963 25289 tablet_copy_client.cc:791] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:25.052712 25289 tablet_copy_client.cc:655] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:25.053601 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet b53acab36f9a42838efe3526f4aed03a from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:25.054188 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet b53acab36f9a42838efe3526f4aed03a from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-b53acab36f9a42838efe3526f4aed03a
I20260924 14:28:25.055152 25287 tablet_bootstrap.cc:492] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:25.056044 25287 tablet_bootstrap.cc:492] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:25.056674 25287 ts_tablet_manager.cc:1397] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.044s	user 0.019s	sys 0.010s
I20260924 14:28:25.059999 25287 raft_consensus.cc:348] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:25.060864 25287 raft_consensus.cc:729] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:25.062808 25289 tablet_copy_client.cc:531] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:25.063601 25287 consensus_queue.cc:260] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:25.083925 24313 tablet_copy_source_session.cc:205] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:25.087644 25291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b53acab36f9a42838efe3526f4aed03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:25.091368 25289 tablet_bootstrap.cc:492] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:25.119620 25291 tablet_copy_client.cc:791] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:25.120565 25291 tablet_copy_client.cc:655] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:25.137027 25291 tablet_copy_client.cc:531] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:25.152119 25287 ts_tablet_manager.cc:1428] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.095s	user 0.021s	sys 0.023s
I20260924 14:28:25.153795 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-dd02146c57f9474fba0afbcc145a7a2c received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:25.154317 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-dd02146c57f9474fba0afbcc145a7a2c on tablet dd02146c57f9474fba0afbcc145a7a2c with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:25.178179 25291 tablet_bootstrap.cc:492] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:25.191198 25289 tablet_bootstrap.cc:492] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:25.192317 25289 tablet_bootstrap.cc:492] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:25.192999 25289 ts_tablet_manager.cc:1397] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.102s	user 0.023s	sys 0.029s
I20260924 14:28:25.197132 25289 raft_consensus.cc:348] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:25.199081 25289 raft_consensus.cc:729] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:25.200178 25289 consensus_queue.cc:260] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:25.202773 25289 ts_tablet_manager.cc:1428] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260924 14:28:25.205689 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-4995099d467c4203bc2872f519b99950 received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:25.206107 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-4995099d467c4203bc2872f519b99950 on tablet 4995099d467c4203bc2872f519b99950 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:25.240094 25291 tablet_bootstrap.cc:492] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:25.241473 25291 tablet_bootstrap.cc:492] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:25.242712 25291 ts_tablet_manager.cc:1397] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.015s
I20260924 14:28:25.246646 25291 raft_consensus.cc:348] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:25.247859 25291 raft_consensus.cc:729] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:25.249065 25291 consensus_queue.cc:260] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:25.251811 25291 ts_tablet_manager.cc:1428] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260924 14:28:25.253341 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-b53acab36f9a42838efe3526f4aed03a received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:25.253780 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-b53acab36f9a42838efe3526f4aed03a on tablet b53acab36f9a42838efe3526f4aed03a with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:25.331175 25291 ts_tablet_manager.cc:927] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:25.344245 25291 tablet_copy_client.cc:316] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:25.346159 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet c9aebcc9d3ed438daf87d68bf2b8f6a9 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:25.346694 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet c9aebcc9d3ed438daf87d68bf2b8f6a9 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-c9aebcc9d3ed438daf87d68bf2b8f6a9
I20260924 14:28:25.355655 24387 tablet_copy_source_session.cc:205] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:25.358280 25291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9aebcc9d3ed438daf87d68bf2b8f6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:25.373102 25291 tablet_copy_client.cc:791] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:25.373698 25291 tablet_copy_client.cc:655] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:25.407960 25291 tablet_copy_client.cc:531] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:25.420888 25291 tablet_bootstrap.cc:492] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:25.456780 25010 raft_consensus.cc:1203] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:25.463410 25291 tablet_bootstrap.cc:492] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:25.464414 25291 tablet_bootstrap.cc:492] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:25.465055 25291 ts_tablet_manager.cc:1397] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.012s
I20260924 14:28:25.468152 25291 raft_consensus.cc:348] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:25.468881 25291 raft_consensus.cc:729] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:25.469439 25291 consensus_queue.cc:260] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:25.471760 25291 ts_tablet_manager.cc:1428] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260924 14:28:25.475309 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-c9aebcc9d3ed438daf87d68bf2b8f6a9 received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:25.475756 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-c9aebcc9d3ed438daf87d68bf2b8f6a9 on tablet c9aebcc9d3ed438daf87d68bf2b8f6a9 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:25.487118 25012 raft_consensus.cc:1203] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:25.635761 25012 raft_consensus.cc:1203] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:25.754827 25012 raft_consensus.cc:1203] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:25.854643 25252 raft_consensus.cc:1050] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:25.857209 25252 consensus_queue.cc:237] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:25.870078 24302 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:25.871146 25012 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:25.871397 25252 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:25.872746 25299 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:25.873032 24377 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:25.878232 25299 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:25.900219 25193 raft_consensus.cc:2932] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:25.915478 24297 raft_consensus.cc:2932] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:25.928525 25012 raft_consensus.cc:2932] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:25.933079 24200 catalog_manager.cc:5495] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:25.938128 24370 raft_consensus.cc:2932] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:25.981582 25267 raft_consensus.cc:1050] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:25.983768 25267 consensus_queue.cc:237] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:26.069732 24452 raft_consensus.cc:1261] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:26.069732 25010 raft_consensus.cc:1261] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:26.083391 25281 consensus_queue.cc:1035] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:26.086062 25267 consensus_queue.cc:1035] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:26.103839 24370 raft_consensus.cc:1261] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:26.108596 25267 consensus_queue.cc:1035] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:26.162545 24448 consensus_queue.cc:237] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:26.172030 25010 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:26.173394 24370 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:26.173411 25299 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:26.174898 25159 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:26.182426 25159 raft_consensus.cc:2932] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.184525 25010 raft_consensus.cc:2932] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.192574 25267 raft_consensus.cc:2932] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.201035 24377 raft_consensus.cc:2932] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.209041 24199 catalog_manager.cc:5495] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.214905 25012 raft_consensus.cc:2932] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.228087 24370 raft_consensus.cc:2932] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.236814 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet 4995099d467c4203bc2872f519b99950 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:26.241680 25314 raft_consensus.cc:1050] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:26.241425 24197 catalog_manager.cc:5495] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:26.244544 25314 consensus_queue.cc:237] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:26.248123 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 4995099d467c4203bc2872f519b99950 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:26.279660 25268 raft_consensus.cc:1050] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:26.295089 24376 raft_consensus.cc:1261] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:26.300211 25320 tablet_replica.cc:330] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:26.305564 25012 raft_consensus.cc:1261] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:26.312084 25267 consensus_queue.cc:1035] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:26.314112 25248 consensus_queue.cc:1035] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:26.304935 25268 consensus_queue.cc:237] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:26.321183 24447 raft_consensus.cc:1261] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:26.342631 25320 raft_consensus.cc:2229] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:26.369066 25320 raft_consensus.cc:2258] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:26.379913 25314 consensus_queue.cc:1035] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:26.380600 25010 raft_consensus.cc:1261] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:26.385210 24452 raft_consensus.cc:1261] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:26.389816 25301 consensus_queue.cc:1035] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:26.392083 25322 consensus_queue.cc:1035] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:26.395027 24297 raft_consensus.cc:1261] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:26.416491 25268 consensus_queue.cc:1035] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:26.419345 25320 ts_tablet_manager.cc:1905] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:26.418668 24302 consensus_queue.cc:237] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
W20260924 14:28:26.442799 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 44773.8us
I20260924 14:28:26.444597 24452 raft_consensus.cc:2932] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.445705 25010 raft_consensus.cc:1261] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:26.449512 25315 raft_consensus.cc:2932] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.455107 25302 raft_consensus.cc:2932] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.462523 25267 consensus_queue.cc:1035] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:26.462538 24374 raft_consensus.cc:2932] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.465056 25012 raft_consensus.cc:2932] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.468736 24447 raft_consensus.cc:2932] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.484808 25248 raft_consensus.cc:2932] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.486454 25012 raft_consensus.cc:2932] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.487270 24201 catalog_manager.cc:5495] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.518635 25320 ts_tablet_manager.cc:1918] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:26.519114 25320 log.cc:1196] T 4995099d467c4203bc2872f519b99950 P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/4995099d467c4203bc2872f519b99950
I20260924 14:28:26.519061 24201 catalog_manager.cc:5495] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.520793 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet 4995099d467c4203bc2872f519b99950 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:26.529390 24197 catalog_manager.cc:5495] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.540838 24297 raft_consensus.cc:2932] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.544378 24447 raft_consensus.cc:1261] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:26.545082 24448 raft_consensus.cc:2932] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.547303 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet dd02146c57f9474fba0afbcc145a7a2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:26.553640 25248 consensus_queue.cc:1035] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:26.551260 25012 raft_consensus.cc:2932] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.562191 24452 raft_consensus.cc:2932] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.571393 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet dd02146c57f9474fba0afbcc145a7a2c with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:26.587189 24302 consensus_queue.cc:237] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:26.607225 24370 raft_consensus.cc:1261] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:26.609942 24374 consensus_queue.cc:237] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:26.613219 25248 consensus_queue.cc:1035] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:26.619817 25011 raft_consensus.cc:1261] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:26.621433 24447 raft_consensus.cc:1261] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:26.621475 25302 consensus_queue.cc:1035] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:26.622399 25332 tablet_replica.cc:330] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:26.623145 25322 consensus_queue.cc:1035] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:26.624431 25332 raft_consensus.cc:2229] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:26.625790 25010 raft_consensus.cc:1261] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:26.629608 25332 raft_consensus.cc:2258] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:26.637157 25326 consensus_queue.cc:1035] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260924 14:28:26.645035 25332 ts_tablet_manager.cc:1905] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:26.655279 25331 raft_consensus.cc:2932] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.659330 25268 raft_consensus.cc:2932] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.667233 25010 raft_consensus.cc:2932] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.663165 24452 raft_consensus.cc:2932] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.674944 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet b53acab36f9a42838efe3526f4aed03a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:26.675642 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet c9aebcc9d3ed438daf87d68bf2b8f6a9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:26.680168 24370 raft_consensus.cc:2932] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.689018 24197 catalog_manager.cc:5495] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:26.713145 24201 catalog_manager.cc:5495] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:26.717270 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet b53acab36f9a42838efe3526f4aed03a with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:26.693939 25012 raft_consensus.cc:2932] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:26.734643 25333 tablet_replica.cc:330] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:26.736104 25333 raft_consensus.cc:2229] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:26.738046 25332 ts_tablet_manager.cc:1918] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:26.745011 25332 log.cc:1196] T dd02146c57f9474fba0afbcc145a7a2c P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/dd02146c57f9474fba0afbcc145a7a2c
I20260924 14:28:26.745584 25333 raft_consensus.cc:2258] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:26.751526 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet dd02146c57f9474fba0afbcc145a7a2c (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:26.753953 25333 ts_tablet_manager.cc:1905] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:26.760893 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet c9aebcc9d3ed438daf87d68bf2b8f6a9 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:26.761911 25320 tablet_replica.cc:330] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:26.762912 25320 raft_consensus.cc:2229] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:26.763656 25320 raft_consensus.cc:2258] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:26.767969 25320 ts_tablet_manager.cc:1905] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:26.772802 25333 ts_tablet_manager.cc:1918] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:26.773270 25333 log.cc:1196] T b53acab36f9a42838efe3526f4aed03a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/b53acab36f9a42838efe3526f4aed03a
I20260924 14:28:26.774900 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet b53acab36f9a42838efe3526f4aed03a (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:26.784334 25320 ts_tablet_manager.cc:1918] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:26.784780 25320 log.cc:1196] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/c9aebcc9d3ed438daf87d68bf2b8f6a9
I20260924 14:28:26.786082 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet c9aebcc9d3ed438daf87d68bf2b8f6a9 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
W20260924 14:28:27.476593 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 58051.2us
W20260924 14:28:27.497381 24269 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 80363.6us
I20260924 14:28:27.778337 24376 consensus_queue.cc:237] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:27.787815 24344 consensus_peers.cc:487] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 86057fee00d44a4c9607d990f2782813. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:27.788641 24298 raft_consensus.cc:1261] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:27.789767 24448 raft_consensus.cc:1261] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:27.790275 25344 consensus_queue.cc:1035] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:27.792060 25322 consensus_queue.cc:1035] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:27.804965 25322 raft_consensus.cc:2932] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:27.807240 24448 raft_consensus.cc:2932] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:27.809028 24298 raft_consensus.cc:2932] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:27.943727 24197 catalog_manager.cc:5495] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:27.980903 24448 consensus_queue.cc:237] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:28.020211 24298 raft_consensus.cc:1261] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:28.021867 24376 raft_consensus.cc:1261] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:28.022001 24418 consensus_peers.cc:487] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e22099296aa6419a924560e9fbe81cfd. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:28.022715 25193 consensus_queue.cc:1035] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:28.024441 25193 consensus_queue.cc:1035] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:28.048375 25159 raft_consensus.cc:2932] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:28.061380 24370 raft_consensus.cc:2932] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:28.067468 24448 consensus_queue.cc:237] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:28.076120 24201 catalog_manager.cc:5495] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:28.090426 24298 raft_consensus.cc:2932] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:28.110033 24418 consensus_peers.cc:487] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7514385d8e4a45e196a8b870b01c1c76. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:28.110172 24300 raft_consensus.cc:1261] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:28.111290 24370 raft_consensus.cc:1261] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:28.112726 25335 consensus_queue.cc:1035] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:28.115350 25193 consensus_queue.cc:1035] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:28.134267 25159 raft_consensus.cc:2932] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:28.135807 24370 raft_consensus.cc:2932] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:28.154505 24300 raft_consensus.cc:2932] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:28.159587 24200 catalog_manager.cc:5495] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:28.161913 24298 consensus_queue.cc:237] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:28.186614 24269 consensus_peers.cc:487] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7959d21c60de4745bebfd367d2367714. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:28.188037 24448 raft_consensus.cc:1261] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:28.188020 24370 raft_consensus.cc:1261] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:28.190292 25343 consensus_queue.cc:1035] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:28.191233 25341 consensus_queue.cc:1035] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:28.200575 25341 raft_consensus.cc:2932] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:28.202762 24448 raft_consensus.cc:2932] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:28.213692 25364 ts_tablet_manager.cc:927] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:28.231436 25364 tablet_copy_client.cc:316] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:28.233119 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet 86057fee00d44a4c9607d990f2782813 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:28.233649 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet 86057fee00d44a4c9607d990f2782813 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-86057fee00d44a4c9607d990f2782813
I20260924 14:28:28.236248 24370 raft_consensus.cc:2932] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:28.241329 24387 tablet_copy_source_session.cc:205] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:28.244709 25364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86057fee00d44a4c9607d990f2782813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:28.251654 24197 catalog_manager.cc:5495] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:28.287194 25364 tablet_copy_client.cc:791] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:28.287884 25364 tablet_copy_client.cc:655] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:28.295782 25364 tablet_copy_client.cc:531] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:28.305387 25364 tablet_bootstrap.cc:492] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:28.334693 25364 tablet_bootstrap.cc:492] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:28.335832 25364 tablet_bootstrap.cc:492] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:28.338435 25364 ts_tablet_manager.cc:1397] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.018s
I20260924 14:28:28.341950 25364 raft_consensus.cc:348] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:28.343017 25364 raft_consensus.cc:729] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:28.343835 25364 consensus_queue.cc:260] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:28.354638 25364 ts_tablet_manager.cc:1428] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.003s
W20260924 14:28:28.363072 25034 debug-util.cc:398] Leaking SignalData structure 0x7b0800301060 after lost signal to thread 23138
W20260924 14:28:28.364143 25034 debug-util.cc:398] Leaking SignalData structure 0x7b0800301740 after lost signal to thread 24235
W20260924 14:28:28.365396 25034 debug-util.cc:398] Leaking SignalData structure 0x7b0800337cc0 after lost signal to thread 24327
W20260924 14:28:28.366490 25034 debug-util.cc:398] Leaking SignalData structure 0x7b08001596e0 after lost signal to thread 24402
W20260924 14:28:28.367621 25034 debug-util.cc:398] Leaking SignalData structure 0x7b0800286160 after lost signal to thread 24476
W20260924 14:28:28.368759 25034 debug-util.cc:398] Leaking SignalData structure 0x7b080025d0c0 after lost signal to thread 25037
I20260924 14:28:28.374265 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-86057fee00d44a4c9607d990f2782813 received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:28.374747 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-86057fee00d44a4c9607d990f2782813 on tablet 86057fee00d44a4c9607d990f2782813 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:28.412071 25364 ts_tablet_manager.cc:927] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:28.444617 25364 tablet_copy_client.cc:316] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:28.450837 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet e22099296aa6419a924560e9fbe81cfd from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:28.451383 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet e22099296aa6419a924560e9fbe81cfd from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-e22099296aa6419a924560e9fbe81cfd
I20260924 14:28:28.460512 24462 tablet_copy_source_session.cc:205] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:28.464170 25364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e22099296aa6419a924560e9fbe81cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:28.481215 25364 tablet_copy_client.cc:791] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:28.481877 25364 tablet_copy_client.cc:655] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:28.488521 25364 tablet_copy_client.cc:531] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:28.499778 25364 tablet_bootstrap.cc:492] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:28.530730 25364 tablet_bootstrap.cc:492] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:28.531813 25364 tablet_bootstrap.cc:492] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:28.532567 25364 ts_tablet_manager.cc:1397] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.010s
I20260924 14:28:28.535437 25364 raft_consensus.cc:348] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:28.536401 25364 raft_consensus.cc:729] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:28.537102 25364 consensus_queue.cc:260] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:28.539767 25364 ts_tablet_manager.cc:1428] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:28:28.545657 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-e22099296aa6419a924560e9fbe81cfd received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:28.546061 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-e22099296aa6419a924560e9fbe81cfd on tablet e22099296aa6419a924560e9fbe81cfd with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:28.557703 25364 ts_tablet_manager.cc:927] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:28.561582 25364 tablet_copy_client.cc:316] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:28.564013 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet 7514385d8e4a45e196a8b870b01c1c76 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:28.564591 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet 7514385d8e4a45e196a8b870b01c1c76 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-7514385d8e4a45e196a8b870b01c1c76
I20260924 14:28:28.574846 24462 tablet_copy_source_session.cc:205] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:28.578560 25364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7514385d8e4a45e196a8b870b01c1c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:28.599826 25364 tablet_copy_client.cc:791] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:28.600410 25364 tablet_copy_client.cc:655] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:28.608088 25010 raft_consensus.cc:1203] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:28.618225 25364 tablet_copy_client.cc:531] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:28.633850 25364 tablet_bootstrap.cc:492] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:28.671953 25364 tablet_bootstrap.cc:492] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:28.672945 25364 tablet_bootstrap.cc:492] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:28.673619 25364 ts_tablet_manager.cc:1397] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.040s	user 0.025s	sys 0.004s
I20260924 14:28:28.677268 25364 raft_consensus.cc:348] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:28.678030 25364 raft_consensus.cc:729] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:28.678598 25364 consensus_queue.cc:260] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:28.681144 25364 ts_tablet_manager.cc:1428] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.005s
I20260924 14:28:28.687606 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-7514385d8e4a45e196a8b870b01c1c76 received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:28.688125 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-7514385d8e4a45e196a8b870b01c1c76 on tablet 7514385d8e4a45e196a8b870b01c1c76 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:28.730226 25364 ts_tablet_manager.cc:927] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:28.736919 25364 tablet_copy_client.cc:316] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:28.738440 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet 7959d21c60de4745bebfd367d2367714 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:28.738991 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet 7959d21c60de4745bebfd367d2367714 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-7959d21c60de4745bebfd367d2367714
I20260924 14:28:28.781646 24313 tablet_copy_source_session.cc:205] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:28.784716 25364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7959d21c60de4745bebfd367d2367714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:28.796507 25364 tablet_copy_client.cc:791] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:28.797032 25364 tablet_copy_client.cc:655] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:28.802552 25364 tablet_copy_client.cc:531] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:28.818498 25364 tablet_bootstrap.cc:492] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:28.819774 25010 raft_consensus.cc:1203] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:28.889750 25364 tablet_bootstrap.cc:492] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:28.890674 25364 tablet_bootstrap.cc:492] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:28.892236 25364 ts_tablet_manager.cc:1397] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.074s	user 0.022s	sys 0.029s
I20260924 14:28:28.894860 25364 raft_consensus.cc:348] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:28.896968 25364 raft_consensus.cc:729] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:28.897549 25364 consensus_queue.cc:260] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:28.899523 25364 ts_tablet_manager.cc:1428] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260924 14:28:28.900890 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-7959d21c60de4745bebfd367d2367714 received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:28.901247 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-7959d21c60de4745bebfd367d2367714 on tablet 7959d21c60de4745bebfd367d2367714 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:28.982404 25010 raft_consensus.cc:1203] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:29.004791 25338 raft_consensus.cc:1050] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:29.007570 25338 consensus_queue.cc:237] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:29.020109 25010 raft_consensus.cc:1261] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.022219 24298 raft_consensus.cc:1261] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.024358 24451 raft_consensus.cc:1261] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.025974 25302 consensus_queue.cc:1035] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:29.036309 25322 consensus_queue.cc:1035] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:29.040449 25338 consensus_queue.cc:1035] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260924 14:28:29.070420 25338 raft_consensus.cc:2932] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.075569 24451 raft_consensus.cc:2932] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.084818 25010 raft_consensus.cc:2932] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.107100 24300 raft_consensus.cc:2932] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.105649 24200 catalog_manager.cc:5495] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.139191 25010 raft_consensus.cc:1203] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:29.182338 24370 consensus_queue.cc:237] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:29.189904 24302 raft_consensus.cc:1261] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:29.190569 25010 raft_consensus.cc:1261] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:29.191494 25378 consensus_queue.cc:1035] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:29.199183 25338 consensus_queue.cc:1035] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:29.201584 25302 raft_consensus.cc:2932] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.207216 24302 raft_consensus.cc:2932] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.209281 25010 raft_consensus.cc:2932] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.223620 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 86057fee00d44a4c9607d990f2782813 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:29.224673 25335 raft_consensus.cc:1050] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:29.226764 24197 catalog_manager.cc:5495] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.230479 25335 consensus_queue.cc:237] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:29.238893 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet 86057fee00d44a4c9607d990f2782813 with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:29.261171 24374 raft_consensus.cc:1261] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.268451 24300 raft_consensus.cc:1261] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.270551 25159 consensus_queue.cc:1035] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:29.273962 25159 consensus_queue.cc:1035] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:29.289062 25385 tablet_replica.cc:330] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:29.290675 25385 raft_consensus.cc:2229] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:29.291416 25385 raft_consensus.cc:2258] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:29.299108 25385 ts_tablet_manager.cc:1905] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:29.307986 25010 raft_consensus.cc:1261] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.323745 25159 consensus_queue.cc:1035] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:29.343293 25387 raft_consensus.cc:2932] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.357025 24300 raft_consensus.cc:2932] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.362433 25385 ts_tablet_manager.cc:1918] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:29.362908 25385 log.cc:1196] T 86057fee00d44a4c9607d990f2782813 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/86057fee00d44a4c9607d990f2782813
I20260924 14:28:29.364524 24370 raft_consensus.cc:2932] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.366761 24197 catalog_manager.cc:5495] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:29.372540 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet 86057fee00d44a4c9607d990f2782813 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:29.422199 25010 raft_consensus.cc:2932] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.456184 24446 consensus_queue.cc:237] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:29.461521 25010 raft_consensus.cc:1261] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:29.461524 24300 raft_consensus.cc:1261] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:29.463047 25387 consensus_queue.cc:1035] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:29.464092 25388 consensus_queue.cc:1035] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:29.471304 25391 raft_consensus.cc:2932] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.479271 25011 raft_consensus.cc:2932] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.478009 24300 raft_consensus.cc:2932] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.499521 24197 catalog_manager.cc:5495] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.510547 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet e22099296aa6419a924560e9fbe81cfd with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:29.514868 25393 tablet_replica.cc:330] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:29.516914 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet e22099296aa6419a924560e9fbe81cfd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:29.517143 25393 raft_consensus.cc:2229] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:29.518024 25393 raft_consensus.cc:2258] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:29.523568 25393 ts_tablet_manager.cc:1905] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:29.540112 25393 ts_tablet_manager.cc:1918] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:29.540591 25393 log.cc:1196] T e22099296aa6419a924560e9fbe81cfd P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/e22099296aa6419a924560e9fbe81cfd
I20260924 14:28:29.545919 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet e22099296aa6419a924560e9fbe81cfd (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:29.582656 25388 raft_consensus.cc:1050] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:29.585785 25388 consensus_queue.cc:237] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:29.613169 25012 raft_consensus.cc:1261] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.616312 25375 raft_consensus.cc:1050] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:29.619762 24300 raft_consensus.cc:1261] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.620186 24370 raft_consensus.cc:1261] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.618731 25375 consensus_queue.cc:237] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:29.621417 25387 consensus_queue.cc:1035] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:29.623416 25387 consensus_queue.cc:1035] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:29.633936 25387 consensus_queue.cc:1035] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:29.686246 25012 raft_consensus.cc:1261] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.687670 25343 consensus_queue.cc:1035] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:29.692945 25011 raft_consensus.cc:2932] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.704666 24451 raft_consensus.cc:1261] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.710753 24370 raft_consensus.cc:1261] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:29.712165 25335 raft_consensus.cc:2932] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.712893 24302 raft_consensus.cc:2932] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.729242 25399 consensus_queue.cc:1035] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:29.742883 24197 catalog_manager.cc:5495] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.751822 25343 consensus_queue.cc:1035] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260924 14:28:29.855132 24451 consensus_queue.cc:237] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:29.858405 24376 raft_consensus.cc:2932] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.864131 25343 raft_consensus.cc:2932] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.877588 24302 raft_consensus.cc:1261] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:29.879113 25335 consensus_queue.cc:1035] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:29.879142 25011 raft_consensus.cc:1261] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:29.885135 25335 consensus_queue.cc:1035] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:29.896261 25373 raft_consensus.cc:2932] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.900240 24302 raft_consensus.cc:2932] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.918926 24201 catalog_manager.cc:5495] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.931020 24376 raft_consensus.cc:2932] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.918926 25011 raft_consensus.cc:2932] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.941104 24449 raft_consensus.cc:2932] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.943871 25012 raft_consensus.cc:2932] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:29.966073 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 7514385d8e4a45e196a8b870b01c1c76 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:29.969133 24201 catalog_manager.cc:5495] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:29.978127 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet 7514385d8e4a45e196a8b870b01c1c76 with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:29.979213 25393 tablet_replica.cc:330] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:29.980522 25393 raft_consensus.cc:2229] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:29.981362 25393 raft_consensus.cc:2258] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:30.003661 25393 ts_tablet_manager.cc:1905] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:30.008548 24302 consensus_queue.cc:237] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:30.027789 25011 raft_consensus.cc:1261] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:30.030001 25401 consensus_queue.cc:1035] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:30.043282 24376 raft_consensus.cc:1261] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:30.054242 25401 consensus_queue.cc:1035] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:30.057271 25343 raft_consensus.cc:2932] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:30.069590 25012 raft_consensus.cc:2932] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:30.070302 24376 raft_consensus.cc:2932] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:30.090324 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 7959d21c60de4745bebfd367d2367714 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:30.096442 25393 ts_tablet_manager.cc:1918] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:30.097141 25393 log.cc:1196] T 7514385d8e4a45e196a8b870b01c1c76 P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/7514385d8e4a45e196a8b870b01c1c76
I20260924 14:28:30.102401 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet 7514385d8e4a45e196a8b870b01c1c76 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:30.103801 24201 catalog_manager.cc:5495] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:30.118654 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet 7959d21c60de4745bebfd367d2367714 with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:30.132027 25406 tablet_replica.cc:330] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:30.133402 25406 raft_consensus.cc:2229] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:30.134230 25406 raft_consensus.cc:2258] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:30.137961 25406 ts_tablet_manager.cc:1905] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:30.159729 25406 ts_tablet_manager.cc:1918] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:30.160334 25406 log.cc:1196] T 7959d21c60de4745bebfd367d2367714 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/7959d21c60de4745bebfd367d2367714
I20260924 14:28:30.162567 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet 7959d21c60de4745bebfd367d2367714 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:31.231638 24376 consensus_queue.cc:237] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:31.243230 24344 consensus_peers.cc:487] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e065e0c774647b1a9b127accaebd70a. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:31.244990 24297 raft_consensus.cc:1261] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:31.245342 24449 raft_consensus.cc:1261] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:31.246729 25409 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:31.253046 25408 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:31.276533 25403 raft_consensus.cc:2932] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.311597 24449 raft_consensus.cc:2932] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.346081 24297 consensus_queue.cc:237] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:31.364192 24300 raft_consensus.cc:2932] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.370369 24201 catalog_manager.cc:5495] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.413240 24370 raft_consensus.cc:1261] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:31.414579 24269 consensus_peers.cc:487] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dfe06d46f9904520a8709a2b3824f11c. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:31.420158 24451 raft_consensus.cc:1261] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:31.441246 25397 consensus_queue.cc:1035] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:31.445117 25343 consensus_queue.cc:1035] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:31.457370 25343 raft_consensus.cc:2932] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.462235 24451 raft_consensus.cc:2932] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.488940 24370 consensus_queue.cc:237] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:31.492098 24374 raft_consensus.cc:2932] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.499620 24197 catalog_manager.cc:5495] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:31.526162 24344 consensus_peers.cc:487] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ca317d21c5f40f69bc0c4ad009f6e94. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:31.528568 24451 raft_consensus.cc:1261] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:31.531127 25403 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:31.532799 24297 raft_consensus.cc:1261] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:31.534439 25338 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:31.542806 25409 raft_consensus.cc:2932] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.545575 24451 raft_consensus.cc:2932] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.558229 24297 raft_consensus.cc:2932] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.570814 24370 consensus_queue.cc:237] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:31.572503 24199 catalog_manager.cc:5495] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:31.620289 24344 consensus_peers.cc:487] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bec046486fdc464290fafd6e4978632e. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:31.620679 24449 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:31.622984 25338 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:31.630303 25427 ts_tablet_manager.cc:927] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:31.630367 24297 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:31.640594 25338 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:31.652040 25427 tablet_copy_client.cc:316] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:31.654254 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet 0e065e0c774647b1a9b127accaebd70a from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:31.654925 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet 0e065e0c774647b1a9b127accaebd70a from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-0e065e0c774647b1a9b127accaebd70a
I20260924 14:28:31.654349 25403 raft_consensus.cc:2932] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.658407 24451 raft_consensus.cc:2932] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.673686 24387 tablet_copy_source_session.cc:205] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:31.676499 24302 raft_consensus.cc:2932] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:31.680832 25427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e065e0c774647b1a9b127accaebd70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:31.700799 24197 catalog_manager.cc:5495] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:31.724115 25427 tablet_copy_client.cc:791] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:31.724983 25427 tablet_copy_client.cc:655] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:31.736295 25427 tablet_copy_client.cc:531] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:31.751513 25427 tablet_bootstrap.cc:492] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:31.775465 25427 tablet_bootstrap.cc:492] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:31.776490 25427 tablet_bootstrap.cc:492] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:31.777328 25427 ts_tablet_manager.cc:1397] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.014s
I20260924 14:28:31.780642 25427 raft_consensus.cc:348] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:31.781340 25427 raft_consensus.cc:729] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:31.782063 25427 consensus_queue.cc:260] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:31.787230 25427 ts_tablet_manager.cc:1428] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260924 14:28:31.791659 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-0e065e0c774647b1a9b127accaebd70a received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:31.792101 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-0e065e0c774647b1a9b127accaebd70a on tablet 0e065e0c774647b1a9b127accaebd70a with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:31.990149 25427 ts_tablet_manager.cc:927] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:31.993804 25434 ts_tablet_manager.cc:927] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:32.006870 25434 tablet_copy_client.cc:316] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:32.009377 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet dfe06d46f9904520a8709a2b3824f11c from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:32.009965 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet dfe06d46f9904520a8709a2b3824f11c from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-dfe06d46f9904520a8709a2b3824f11c
I20260924 14:28:32.018988 25427 tablet_copy_client.cc:316] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:32.027180 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet 3ca317d21c5f40f69bc0c4ad009f6e94 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:32.027769 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet 3ca317d21c5f40f69bc0c4ad009f6e94 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-3ca317d21c5f40f69bc0c4ad009f6e94
I20260924 14:28:32.032130 24313 tablet_copy_source_session.cc:205] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:32.035565 25434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfe06d46f9904520a8709a2b3824f11c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:32.036348 24387 tablet_copy_source_session.cc:205] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:32.039693 25427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca317d21c5f40f69bc0c4ad009f6e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:32.056161 25427 tablet_copy_client.cc:791] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:32.056769 25427 tablet_copy_client.cc:655] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:32.061820 25427 tablet_copy_client.cc:531] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:32.065191 25434 tablet_copy_client.cc:791] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:32.065905 25434 tablet_copy_client.cc:655] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:32.071132 25427 tablet_bootstrap.cc:492] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:32.073087 25434 tablet_copy_client.cc:531] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:32.080792 25012 raft_consensus.cc:1203] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:32.115631 25427 tablet_bootstrap.cc:492] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:32.116741 25427 tablet_bootstrap.cc:492] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:32.117508 25427 ts_tablet_manager.cc:1397] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.047s	user 0.027s	sys 0.014s
I20260924 14:28:32.120276 25427 raft_consensus.cc:348] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:32.121277 25427 raft_consensus.cc:729] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:32.121999 25427 consensus_queue.cc:260] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:32.125360 25427 ts_tablet_manager.cc:1428] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260924 14:28:32.127285 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-3ca317d21c5f40f69bc0c4ad009f6e94 received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:32.127738 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-3ca317d21c5f40f69bc0c4ad009f6e94 on tablet 3ca317d21c5f40f69bc0c4ad009f6e94 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:32.155076 25427 ts_tablet_manager.cc:927] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:32.162467 25434 tablet_bootstrap.cc:492] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:32.174558 25427 tablet_copy_client.cc:316] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:32.178294 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet bec046486fdc464290fafd6e4978632e from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:32.178879 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet bec046486fdc464290fafd6e4978632e from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-bec046486fdc464290fafd6e4978632e
I20260924 14:28:32.193284 24387 tablet_copy_source_session.cc:205] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:32.207945 25434 tablet_bootstrap.cc:492] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:32.208011 25427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bec046486fdc464290fafd6e4978632e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:32.224203 25434 tablet_bootstrap.cc:492] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:32.225032 25434 ts_tablet_manager.cc:1397] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.010s
I20260924 14:28:32.228166 25434 raft_consensus.cc:348] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:32.229143 25434 raft_consensus.cc:729] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:32.229833 25434 consensus_queue.cc:260] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:32.240701 25427 tablet_copy_client.cc:791] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:32.241003 25434 ts_tablet_manager.cc:1428] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260924 14:28:32.241571 25427 tablet_copy_client.cc:655] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:32.251434 25427 tablet_copy_client.cc:531] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:32.252054 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-dfe06d46f9904520a8709a2b3824f11c received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:32.252468 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-dfe06d46f9904520a8709a2b3824f11c on tablet dfe06d46f9904520a8709a2b3824f11c with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:32.264612 25427 tablet_bootstrap.cc:492] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:32.299009 25427 tablet_bootstrap.cc:492] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:32.300099 25427 tablet_bootstrap.cc:492] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:32.300828 25427 ts_tablet_manager.cc:1397] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.036s	user 0.023s	sys 0.006s
I20260924 14:28:32.303813 25427 raft_consensus.cc:348] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:32.304801 25427 raft_consensus.cc:729] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:32.305492 25427 consensus_queue.cc:260] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:32.307826 25427 ts_tablet_manager.cc:1428] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260924 14:28:32.310603 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-bec046486fdc464290fafd6e4978632e received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:32.310998 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-bec046486fdc464290fafd6e4978632e on tablet bec046486fdc464290fafd6e4978632e with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:32.541548 25011 raft_consensus.cc:1203] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:32.576895 25011 raft_consensus.cc:1203] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:32.585930 25011 raft_consensus.cc:1203] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:32.686910 25431 raft_consensus.cc:1050] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:32.692291 25431 consensus_queue.cc:237] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:32.700184 25012 raft_consensus.cc:1261] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:32.700868 24451 raft_consensus.cc:1261] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:32.704419 24297 raft_consensus.cc:1261] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:32.706023 25403 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:32.707767 25403 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:32.711234 25409 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:32.752210 25403 raft_consensus.cc:2932] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:32.773401 25012 raft_consensus.cc:2932] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:32.801304 24451 raft_consensus.cc:2932] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:32.810195 24201 catalog_manager.cc:5495] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:32.825474 24297 raft_consensus.cc:2932] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:32.881603 24372 consensus_queue.cc:237] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:32.896035 24451 raft_consensus.cc:1261] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:32.896355 25011 raft_consensus.cc:1261] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:32.898155 25446 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:32.899072 25445 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:32.910131 25445 raft_consensus.cc:2932] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:32.917584 25011 raft_consensus.cc:2932] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:32.912784 24451 raft_consensus.cc:2932] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:32.938019 24200 catalog_manager.cc:5495] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:32.939692 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 0e065e0c774647b1a9b127accaebd70a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:32.963421 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e065e0c774647b1a9b127accaebd70a with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:32.968082 25451 tablet_replica.cc:330] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:32.969836 25451 raft_consensus.cc:2229] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:32.970682 25451 raft_consensus.cc:2258] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:32.973604 25451 ts_tablet_manager.cc:1905] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:32.995108 25451 ts_tablet_manager.cc:1918] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:32.995528 25451 log.cc:1196] T 0e065e0c774647b1a9b127accaebd70a P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/0e065e0c774647b1a9b127accaebd70a
I20260924 14:28:32.997123 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet 0e065e0c774647b1a9b127accaebd70a (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:33.028577 25343 raft_consensus.cc:1050] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:33.031049 25343 consensus_queue.cc:237] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:33.040979 25011 raft_consensus.cc:1261] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:33.042908 24451 raft_consensus.cc:1261] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:33.044371 24372 raft_consensus.cc:1261] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:33.045079 25410 consensus_queue.cc:1035] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:33.046604 25415 consensus_queue.cc:1035] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:33.048633 25415 consensus_queue.cc:1035] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:33.084342 25410 raft_consensus.cc:2932] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.087615 24376 raft_consensus.cc:2932] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.103884 24451 raft_consensus.cc:2932] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.094357 25012 raft_consensus.cc:2932] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.111781 24200 catalog_manager.cc:5495] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.113235 25446 raft_consensus.cc:1050] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:33.116609 25446 consensus_queue.cc:237] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:33.134452 24297 consensus_queue.cc:237] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:33.145900 25012 raft_consensus.cc:1261] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:33.165060 25410 consensus_queue.cc:1035] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:33.179546 24449 raft_consensus.cc:1261] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:33.181142 25457 consensus_queue.cc:1035] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:33.183148 25011 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:33.184217 25431 raft_consensus.cc:1050] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:33.185492 25445 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:33.186757 25431 consensus_queue.cc:237] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:33.191825 24451 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:33.195546 24297 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:33.196046 25343 raft_consensus.cc:2932] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.197703 25445 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260924 14:28:33.199525 24269 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 52417.9us
I20260924 14:28:33.198611 24451 raft_consensus.cc:2932] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.207805 25403 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:33.222259 25011 raft_consensus.cc:2932] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.229943 24446 raft_consensus.cc:1261] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:33.232810 25012 raft_consensus.cc:1261] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:33.234162 25446 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:33.235702 25403 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:33.240926 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet dfe06d46f9904520a8709a2b3824f11c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:33.257787 24199 catalog_manager.cc:5495] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:33.259317 24302 raft_consensus.cc:1261] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:33.252758 25403 raft_consensus.cc:2932] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.258827 24297 raft_consensus.cc:2932] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.275207 24449 raft_consensus.cc:2932] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.284919 25446 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260924 14:28:33.308342 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet dfe06d46f9904520a8709a2b3824f11c with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:33.314585 25409 raft_consensus.cc:2932] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.321264 25012 raft_consensus.cc:2932] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.321738 25011 raft_consensus.cc:2932] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.327152 24200 catalog_manager.cc:5495] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.356403 24372 consensus_queue.cc:237] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:33.361923 24449 raft_consensus.cc:2932] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.362401 24297 raft_consensus.cc:2932] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.378171 24446 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:33.383184 25403 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:33.390383 24200 catalog_manager.cc:5495] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.409190 25446 raft_consensus.cc:2932] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.410832 24446 raft_consensus.cc:2932] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.420337 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet bec046486fdc464290fafd6e4978632e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:33.430863 25013 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:33.431718 25468 tablet_replica.cc:330] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:33.433408 25446 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:33.437809 25012 raft_consensus.cc:2932] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.450908 25468 raft_consensus.cc:2229] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:33.453567 25468 raft_consensus.cc:2258] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:33.452570 24187 catalog_manager.cc:5018] ChangeConfig:REMOVE_PEER RPC for tablet bec046486fdc464290fafd6e4978632e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260924 14:28:33.458570 25468 ts_tablet_manager.cc:1905] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:33.463773 24200 catalog_manager.cc:5495] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.509524 25468 ts_tablet_manager.cc:1918] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:33.510397 25468 log.cc:1196] T dfe06d46f9904520a8709a2b3824f11c P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/dfe06d46f9904520a8709a2b3824f11c
I20260924 14:28:33.512944 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet dfe06d46f9904520a8709a2b3824f11c (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:33.517002 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet bec046486fdc464290fafd6e4978632e with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:33.527127 24370 consensus_queue.cc:237] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:33.527823 25469 tablet_replica.cc:330] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:33.530277 25469 raft_consensus.cc:2229] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:33.533344 25013 raft_consensus.cc:1261] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:33.534725 25446 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:33.535810 24297 raft_consensus.cc:1261] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:33.537249 25469 raft_consensus.cc:2258] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:33.543376 25469 ts_tablet_manager.cc:1905] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:33.542987 25463 raft_consensus.cc:2932] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.544641 25446 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:33.548303 25010 raft_consensus.cc:2932] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.563586 25469 ts_tablet_manager.cc:1918] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:33.564193 25469 log.cc:1196] T bec046486fdc464290fafd6e4978632e P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/bec046486fdc464290fafd6e4978632e
I20260924 14:28:33.567436 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 3ca317d21c5f40f69bc0c4ad009f6e94 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:33.569510 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet bec046486fdc464290fafd6e4978632e (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:33.578464 24199 catalog_manager.cc:5495] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:33.566602 24297 raft_consensus.cc:2932] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:33.597213 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ca317d21c5f40f69bc0c4ad009f6e94 with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:33.616647 25470 tablet_replica.cc:330] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:33.617833 25470 raft_consensus.cc:2229] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:33.618469 25470 raft_consensus.cc:2258] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:33.621177 25470 ts_tablet_manager.cc:1905] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:33.635854 25470 ts_tablet_manager.cc:1918] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:33.636353 25470 log.cc:1196] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/3ca317d21c5f40f69bc0c4ad009f6e94
I20260924 14:28:33.638303 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet 3ca317d21c5f40f69bc0c4ad009f6e94 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:33.882872 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:34.111569 24370 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2bdd9163fb2d44068b0f93f1bbfd7387"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:34.112349 24370 raft_consensus.cc:595] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:34.159511 25409 raft_consensus.cc:979] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:34.161303 25409 raft_consensus.cc:1067] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:34.167495 25008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2bdd9163fb2d44068b0f93f1bbfd7387"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45626
I20260924 14:28:34.168210 25008 raft_consensus.cc:482] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:34.168627 25008 raft_consensus.cc:3037] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.176941 25008 raft_consensus.cc:504] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.177798 24446 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:34.178417 24446 raft_consensus.cc:595] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Received request to transfer leadership to TS 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:34.181152 24376 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:34.181269 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdd9163fb2d44068b0f93f1bbfd7387" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:34.181747 24376 raft_consensus.cc:595] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:34.181994 24297 raft_consensus.cc:3037] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.184247 24372 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:34.184912 24372 raft_consensus.cc:595] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:34.185647 25008 leader_election.cc:288] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:34.190038 24297 raft_consensus.cc:2454] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:34.192122 24979 leader_election.cc:302] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:34.193455 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdd9163fb2d44068b0f93f1bbfd7387" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:34.194372 24372 raft_consensus.cc:3032] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:34.194860 24372 raft_consensus.cc:729] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:34.196136 24372 consensus_queue.cc:260] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.198292 24372 raft_consensus.cc:3037] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.206660 25483 raft_consensus.cc:2781] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:34.207309 24297 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:34.208102 24297 raft_consensus.cc:595] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Received request to transfer leadership to TS 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:34.210624 25483 raft_consensus.cc:686] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:34.211170 24372 raft_consensus.cc:2454] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:34.211967 24376 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:34.211583 25483 consensus_queue.cc:237] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.212572 24376 raft_consensus.cc:595] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:34.213692 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260924 14:28:34.215376 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:34.216156 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:34.226720 24199 catalog_manager.cc:5495] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:34.363574 25481 raft_consensus.cc:979] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:34.364056 25471 raft_consensus.cc:1067] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Signalling peer 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:34.365543 24370 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:49700
I20260924 14:28:34.366021 24370 raft_consensus.cc:482] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:34.366281 24370 raft_consensus.cc:3037] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.372581 24370 raft_consensus.cc:504] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:34.374886 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:34.375324 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317bae4ebac443c8a2fada29c389ab79" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:34.375594 24300 raft_consensus.cc:3037] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.374900 24370 leader_election.cc:288] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:34.376734 24446 raft_consensus.cc:3032] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:34.377045 24446 raft_consensus.cc:729] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:34.377758 24446 consensus_queue.cc:260] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:34.378744 24446 raft_consensus.cc:3037] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.381088 24300 raft_consensus.cc:2454] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:34.382225 24344 leader_election.cc:302] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:34.383086 25463 raft_consensus.cc:2781] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:34.383523 25463 raft_consensus.cc:686] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:34.384161 24446 raft_consensus.cc:2454] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:34.384348 25463 consensus_queue.cc:237] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:34.398901 24199 catalog_manager.cc:5495] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 1 to 2, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:34.399755 25477 raft_consensus.cc:979] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:34.400291 25475 raft_consensus.cc:1067] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:34.402004 25008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45626
I20260924 14:28:34.402598 25008 raft_consensus.cc:482] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:34.402968 25008 raft_consensus.cc:3037] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.412716 25008 raft_consensus.cc:504] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.414440 25008 leader_election.cc:288] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:34.415236 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:34.415531 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79eded71d354d4095b73f2f7f7153e5" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:34.415944 24370 raft_consensus.cc:3032] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:34.416069 24446 raft_consensus.cc:3037] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.416309 24370 raft_consensus.cc:729] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:34.417073 24370 consensus_queue.cc:260] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.418308 24370 raft_consensus.cc:3037] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.423285 24446 raft_consensus.cc:2454] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:34.425014 24370 raft_consensus.cc:2454] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:34.425011 24978 leader_election.cc:302] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:34.426024 25483 raft_consensus.cc:2781] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:34.426550 25483 raft_consensus.cc:686] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:34.427431 25483 consensus_queue.cc:237] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.435424 24199 catalog_manager.cc:5495] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:34.522140 25480 raft_consensus.cc:979] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:34.522692 25480 raft_consensus.cc:1067] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:34.524364 25008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45626
I20260924 14:28:34.524996 25008 raft_consensus.cc:482] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:34.525313 25008 raft_consensus.cc:3037] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.531183 25008 raft_consensus.cc:504] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.532891 25008 leader_election.cc:288] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:34.533980 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:34.534600 24297 raft_consensus.cc:3037] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.534708 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca317d21c5f40f69bc0c4ad009f6e94" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:34.535387 24370 raft_consensus.cc:3032] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:34.535751 24370 raft_consensus.cc:729] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:34.536615 24370 consensus_queue.cc:260] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.537930 24370 raft_consensus.cc:3037] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.541427 24297 raft_consensus.cc:2454] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:34.542438 24979 leader_election.cc:302] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:34.543100 25483 raft_consensus.cc:2781] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:34.543165 24370 raft_consensus.cc:2454] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:34.543813 25483 raft_consensus.cc:686] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:34.544719 25483 consensus_queue.cc:237] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.555593 24199 catalog_manager.cc:5495] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:34.638768 25480 raft_consensus.cc:979] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:34.639393 25463 raft_consensus.cc:1067] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:34.651994 25008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45626
I20260924 14:28:34.653241 25008 raft_consensus.cc:482] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:34.653647 25008 raft_consensus.cc:3037] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.654899 24297 consensus_queue.cc:237] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:34.661361 25008 raft_consensus.cc:504] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.662053 24300 raft_consensus.cc:1261] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:34.663699 25008 leader_election.cc:288] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:34.663623 25483 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:34.664366 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
W20260924 14:28:34.664969 24269 consensus_peers.cc:487] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 095719203855447b9091cc7846d566de. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:34.664981 24376 raft_consensus.cc:3032] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:34.666013 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e78577921b4cf18418d98b6f6a30a1" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:34.666240 24376 raft_consensus.cc:729] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:34.666632 24446 raft_consensus.cc:3037] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.667032 24376 consensus_queue.cc:260] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.668314 24376 raft_consensus.cc:3037] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.672981 24370 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:34.673518 24448 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:34.673961 25458 raft_consensus.cc:979] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:34.674729 25457 raft_consensus.cc:1067] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Signalling peer 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:34.676374 25458 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:34.678570 24370 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:49710
I20260924 14:28:34.678845 25457 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:34.681358 24446 raft_consensus.cc:2454] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:34.682464 24978 leader_election.cc:302] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:34.683276 25483 raft_consensus.cc:2781] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:34.683739 25483 raft_consensus.cc:686] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:34.684541 25483 consensus_queue.cc:237] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:34.685676 24376 raft_consensus.cc:2454] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:34.679277 24370 raft_consensus.cc:482] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:34.692037 24370 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.694825 25459 raft_consensus.cc:2932] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:34.700201 24451 raft_consensus.cc:2932] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:34.700403 24372 raft_consensus.cc:2932] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:34.718906 24370 raft_consensus.cc:504] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:34.722000 24370 leader_election.cc:288] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:34.722170 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:34.722796 24449 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.724390 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:34.724969 24300 raft_consensus.cc:3032] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:34.725224 24300 raft_consensus.cc:729] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:34.725880 24300 consensus_queue.cc:260] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:34.727140 24300 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:34.730741 24199 catalog_manager.cc:5495] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:34.742224 24449 raft_consensus.cc:2454] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:34.748308 24197 catalog_manager.cc:5495] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:34.749111 24343 leader_election.cc:302] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:34.750029 25463 raft_consensus.cc:2781] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:34.750516 25463 raft_consensus.cc:686] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:34.751386 25463 consensus_queue.cc:237] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:34.770195 24300 raft_consensus.cc:2454] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:34.775131 24376 raft_consensus.cc:1261] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:34.778369 24451 consensus_queue.cc:237] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:34.783113 25497 consensus_queue.cc:1035] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260924 14:28:34.798612 24449 raft_consensus.cc:1261] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:34.798904 24300 raft_consensus.cc:1261] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:34.802551 25463 consensus_queue.cc:1035] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260924 14:28:34.804728 25477 consensus_queue.cc:1035] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:28:34.814239 24418 consensus_peers.cc:487] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd5890410f3f4f22a1f26e09cc824bb2. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:34.817173 24372 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:34.818904 24297 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:34.820350 24451 raft_consensus.cc:1261] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:34.820679 25481 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:34.826431 25472 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:34.844516 25497 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:34.870204 24370 raft_consensus.cc:1261] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:34.871870 25483 consensus_queue.cc:1035] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:34.871320 25472 raft_consensus.cc:2932] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:34.866850 24197 catalog_manager.cc:5495] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:34.932785 24298 consensus_queue.cc:237] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:34.938632 24370 raft_consensus.cc:2932] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:34.927104 24300 raft_consensus.cc:2932] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:34.955892 24199 catalog_manager.cc:5495] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260924 14:28:34.970894 24269 consensus_peers.cc:487] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e838ce343bad4d42aa930f68fe4a4b2c. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:34.972153 24372 raft_consensus.cc:1261] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:34.974427 25458 consensus_queue.cc:1035] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:34.970942 24451 raft_consensus.cc:1261] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:34.978412 25459 consensus_queue.cc:1035] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:34.994764 25457 raft_consensus.cc:2932] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:35.000706 24372 raft_consensus.cc:2932] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:35.010986 24451 raft_consensus.cc:2932] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:35.022993 24370 raft_consensus.cc:1261] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:35.022632 24298 consensus_queue.cc:237] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:35.031033 24199 catalog_manager.cc:5495] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:35.024461 25497 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260924 14:28:35.054401 24269 consensus_peers.cc:487] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 93b3f9be3c4846a8b75e5e42099e6dbc. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:35.055568 24446 raft_consensus.cc:1261] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:35.058346 25457 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:35.059161 24300 raft_consensus.cc:1261] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:35.060859 25483 consensus_queue.cc:1035] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:35.064664 24370 raft_consensus.cc:1261] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:35.066165 25457 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:35.087993 25458 raft_consensus.cc:2932] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:35.089610 25521 ts_tablet_manager.cc:927] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:35.093464 24372 raft_consensus.cc:2932] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:35.099308 24446 raft_consensus.cc:2932] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:35.106981 25521 tablet_copy_client.cc:316] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:35.108713 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet 095719203855447b9091cc7846d566de from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:35.109262 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet 095719203855447b9091cc7846d566de from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-095719203855447b9091cc7846d566de
I20260924 14:28:35.112953 24197 catalog_manager.cc:5495] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:35.121742 24313 tablet_copy_source_session.cc:205] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:35.126258 25521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 095719203855447b9091cc7846d566de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:35.141889 25521 tablet_copy_client.cc:791] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:35.142503 25521 tablet_copy_client.cc:655] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:35.146844 25521 tablet_copy_client.cc:531] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:35.155793 25521 tablet_bootstrap.cc:492] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:35.171809 24370 raft_consensus.cc:1261] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:35.175141 25483 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:35.247722 24449 raft_consensus.cc:1261] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:35.251788 25484 consensus_queue.cc:1035] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:28:35.269379 25521 tablet_bootstrap.cc:492] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:35.270390 25521 tablet_bootstrap.cc:492] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:35.271214 25521 ts_tablet_manager.cc:1397] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.116s	user 0.014s	sys 0.022s
I20260924 14:28:35.277323 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:35.277103 25521 raft_consensus.cc:348] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:35.277990 25521 raft_consensus.cc:729] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:35.278558 25521 consensus_queue.cc:260] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:35.280902 25521 ts_tablet_manager.cc:1428] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260924 14:28:35.282874 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-095719203855447b9091cc7846d566de received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:35.283252 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-095719203855447b9091cc7846d566de on tablet 095719203855447b9091cc7846d566de with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:35.289399 24449 raft_consensus.cc:1261] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:35.291231 25463 consensus_queue.cc:1035] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:35.306041 25521 ts_tablet_manager.cc:927] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:35.313673 24300 raft_consensus.cc:1261] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:35.321224 25521 tablet_copy_client.cc:316] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:35.328408 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet bd5890410f3f4f22a1f26e09cc824bb2 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:35.328964 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet bd5890410f3f4f22a1f26e09cc824bb2 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-bd5890410f3f4f22a1f26e09cc824bb2
I20260924 14:28:35.324950 25477 consensus_queue.cc:1035] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260924 14:28:35.340523 24462 tablet_copy_source_session.cc:205] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:35.343945 25521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5890410f3f4f22a1f26e09cc824bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:35.347812 24303 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:35.348474 24303 raft_consensus.cc:595] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Received request to transfer leadership to TS 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:35.368747 24376 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0e065e0c774647b1a9b127accaebd70a"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:35.369345 24376 raft_consensus.cc:595] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:35.371647 25521 tablet_copy_client.cc:791] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:35.372079 24446 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2c4bc71d12c14157982c98b4c7ca505c"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:35.372733 25521 tablet_copy_client.cc:655] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:35.372764 24446 raft_consensus.cc:595] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:35.376297 24449 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4995099d467c4203bc2872f519b99950"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:35.376806 24449 raft_consensus.cc:595] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Received request to transfer leadership to TS 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:35.378466 25521 tablet_copy_client.cc:531] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:35.380555 24370 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:35.381110 24370 raft_consensus.cc:595] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:35.384140 24446 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ce753b9bac814f17a869f4f8d66deca6"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:35.384662 24446 raft_consensus.cc:595] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:35.385641 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260924 14:28:35.387224 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:35.387850 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:35.389686 25521 tablet_bootstrap.cc:492] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:35.427014 25521 tablet_bootstrap.cc:492] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:35.428038 25521 tablet_bootstrap.cc:492] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:35.428757 25521 ts_tablet_manager.cc:1397] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.008s
I20260924 14:28:35.432907 25531 ts_tablet_manager.cc:927] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:35.434958 25521 raft_consensus.cc:348] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:35.435997 25521 raft_consensus.cc:729] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:35.436930 25475 raft_consensus.cc:979] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:35.436860 25521 consensus_queue.cc:260] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:35.437390 25475 raft_consensus.cc:1067] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:35.439060 25521 ts_tablet_manager.cc:1428] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260924 14:28:35.442256 25009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0e065e0c774647b1a9b127accaebd70a"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45626
I20260924 14:28:35.443140 25009 raft_consensus.cc:482] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:35.443411 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-bd5890410f3f4f22a1f26e09cc824bb2 received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:35.443676 25009 raft_consensus.cc:3037] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.443912 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-bd5890410f3f4f22a1f26e09cc824bb2 on tablet bd5890410f3f4f22a1f26e09cc824bb2 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:35.445312 25531 tablet_copy_client.cc:316] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:35.447656 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet e838ce343bad4d42aa930f68fe4a4b2c from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:35.448158 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet e838ce343bad4d42aa930f68fe4a4b2c from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-e838ce343bad4d42aa930f68fe4a4b2c
I20260924 14:28:35.450417 25521 ts_tablet_manager.cc:927] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:35.456386 24313 tablet_copy_source_session.cc:205] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:35.464485 25531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e838ce343bad4d42aa930f68fe4a4b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:35.464519 25009 raft_consensus.cc:504] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.474316 25009 leader_election.cc:288] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:35.475205 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e065e0c774647b1a9b127accaebd70a" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:35.475852 24446 raft_consensus.cc:3037] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.477422 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e065e0c774647b1a9b127accaebd70a" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:35.477986 24370 raft_consensus.cc:3032] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:35.478226 24370 raft_consensus.cc:729] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:35.478607 25521 tablet_copy_client.cc:316] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:35.478928 24370 consensus_queue.cc:260] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.480011 24370 raft_consensus.cc:3037] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.480336 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet 93b3f9be3c4846a8b75e5e42099e6dbc from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:35.481060 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet 93b3f9be3c4846a8b75e5e42099e6dbc from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-93b3f9be3c4846a8b75e5e42099e6dbc
I20260924 14:28:35.491348 24370 raft_consensus.cc:2454] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:35.492733 24978 leader_election.cc:302] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:35.493643 25497 raft_consensus.cc:2781] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:35.494177 24313 tablet_copy_source_session.cc:205] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:35.494321 25497 raft_consensus.cc:686] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:35.495466 25497 consensus_queue.cc:237] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.508431 25009 raft_consensus.cc:1203] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:35.510433 24446 raft_consensus.cc:2454] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:35.528390 25531 tablet_copy_client.cc:791] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:35.529098 25531 tablet_copy_client.cc:655] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:35.538185 25531 tablet_copy_client.cc:531] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:35.539209 25521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93b3f9be3c4846a8b75e5e42099e6dbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:35.556777 25531 tablet_bootstrap.cc:492] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:35.559422 25512 raft_consensus.cc:979] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:35.559942 25512 raft_consensus.cc:1067] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:35.561693 25008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2c4bc71d12c14157982c98b4c7ca505c"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45616
I20260924 14:28:35.562228 25008 raft_consensus.cc:482] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:35.562608 25008 raft_consensus.cc:3037] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.564636 24199 catalog_manager.cc:5495] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:35.567252 25521 tablet_copy_client.cc:791] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:35.567848 25521 tablet_copy_client.cc:655] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:35.571210 25008 raft_consensus.cc:504] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.573872 25008 leader_election.cc:288] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:35.574798 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c4bc71d12c14157982c98b4c7ca505c" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:35.575392 24449 raft_consensus.cc:3032] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:35.575690 24449 raft_consensus.cc:729] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:35.575947 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c4bc71d12c14157982c98b4c7ca505c" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:35.576681 24303 raft_consensus.cc:3037] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.576403 24449 consensus_queue.cc:260] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.577572 24449 raft_consensus.cc:3037] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.578269 25472 raft_consensus.cc:979] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:35.578732 25472 raft_consensus.cc:1067] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Signalling peer 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:35.585170 24449 raft_consensus.cc:2454] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:35.586474 24978 leader_election.cc:302] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:35.587520 25497 raft_consensus.cc:2781] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:35.587991 25497 raft_consensus.cc:686] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:35.590112 25497 consensus_queue.cc:237] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.594576 25531 tablet_bootstrap.cc:492] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:35.595503 25531 tablet_bootstrap.cc:492] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:35.596146 25531 ts_tablet_manager.cc:1397] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.040s	user 0.025s	sys 0.007s
I20260924 14:28:35.603523 24370 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4995099d467c4203bc2872f519b99950"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:49700
I20260924 14:28:35.605585 24370 raft_consensus.cc:482] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:35.606048 24370 raft_consensus.cc:3037] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.602993 25531 raft_consensus.cc:348] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:35.604692 24303 raft_consensus.cc:2454] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:35.611284 25531 raft_consensus.cc:729] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:35.612628 25531 consensus_queue.cc:260] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:35.616544 25531 ts_tablet_manager.cc:1428] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.002s
I20260924 14:28:35.617563 25472 raft_consensus.cc:979] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:35.618054 25472 raft_consensus.cc:1067] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:35.618673 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-e838ce343bad4d42aa930f68fe4a4b2c received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:35.619271 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-e838ce343bad4d42aa930f68fe4a4b2c on tablet e838ce343bad4d42aa930f68fe4a4b2c with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:35.619534 25009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce753b9bac814f17a869f4f8d66deca6"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45616
I20260924 14:28:35.620195 25009 raft_consensus.cc:482] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:35.619607 24199 catalog_manager.cc:5495] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:35.620558 25009 raft_consensus.cc:3037] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.626307 24370 raft_consensus.cc:504] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.627583 25521 tablet_copy_client.cc:531] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:35.629205 24370 leader_election.cc:288] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:35.629084 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:35.629788 24449 raft_consensus.cc:3032] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:35.630066 24449 raft_consensus.cc:729] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:35.630643 25008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:35.630729 24449 consensus_queue.cc:260] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.631433 25008 raft_consensus.cc:3037] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.637565 25009 raft_consensus.cc:504] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.639779 25009 leader_election.cc:288] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:35.632109 24449 raft_consensus.cc:3037] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.641023 25521 tablet_bootstrap.cc:492] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:35.641680 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce753b9bac814f17a869f4f8d66deca6" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:35.642282 24446 raft_consensus.cc:3032] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:35.642552 24446 raft_consensus.cc:729] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:35.643842 24303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce753b9bac814f17a869f4f8d66deca6" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:35.644392 24303 raft_consensus.cc:3037] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.647257 24446 consensus_queue.cc:260] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.648590 24446 raft_consensus.cc:3037] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.655514 24449 raft_consensus.cc:2454] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:35.657311 24303 raft_consensus.cc:2454] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:35.657375 24343 leader_election.cc:302] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:35.658962 24979 leader_election.cc:302] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:35.659807 25497 raft_consensus.cc:2781] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:35.660319 25497 raft_consensus.cc:686] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:35.661247 25497 consensus_queue.cc:237] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.663985 25480 raft_consensus.cc:2781] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:35.664580 25480 raft_consensus.cc:686] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:35.664460 25008 raft_consensus.cc:2454] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:35.665527 25480 consensus_queue.cc:237] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.668052 24446 raft_consensus.cc:2454] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:35.673364 25492 raft_consensus.cc:979] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:35.673810 25492 raft_consensus.cc:1067] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Signalling peer 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:35.682250 24199 catalog_manager.cc:5495] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:35.685695 24446 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:48734
I20260924 14:28:35.686261 24446 raft_consensus.cc:482] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:35.686633 24446 raft_consensus.cc:3037] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.691341 25480 raft_consensus.cc:979] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:35.691861 25475 raft_consensus.cc:1067] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:35.693701 25008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45626
I20260924 14:28:35.693871 24446 raft_consensus.cc:504] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:35.694307 25008 raft_consensus.cc:482] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:35.694839 25008 raft_consensus.cc:3037] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.691883 24200 catalog_manager.cc:5495] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 1 to 2, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:35.696830 24446 leader_election.cc:288] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:35.697785 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:35.698356 24370 raft_consensus.cc:3037] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.702932 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:35.703598 24300 raft_consensus.cc:3032] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:35.703888 24300 raft_consensus.cc:729] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:35.705060 24370 raft_consensus.cc:2454] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:35.706374 24417 leader_election.cc:302] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:35.706671 24300 consensus_queue.cc:260] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:35.707226 25512 raft_consensus.cc:2781] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:35.707737 25512 raft_consensus.cc:686] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:35.708040 24300 raft_consensus.cc:3037] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.708657 25512 consensus_queue.cc:237] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:35.745497 25008 raft_consensus.cc:504] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.749379 25008 leader_election.cc:288] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:35.757005 24200 catalog_manager.cc:5495] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:35.762714 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:35.763367 24449 raft_consensus.cc:3037] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.775238 25008 raft_consensus.cc:1203] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:35.779395 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9aebcc9d3ed438daf87d68bf2b8f6a9" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:35.780234 24370 raft_consensus.cc:3032] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:35.780592 24370 raft_consensus.cc:729] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:35.781325 24370 consensus_queue.cc:260] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.782609 24370 raft_consensus.cc:3037] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:35.794523 24300 raft_consensus.cc:2454] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:35.798440 24449 raft_consensus.cc:2454] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:35.799552 24978 leader_election.cc:302] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:35.800452 25483 raft_consensus.cc:2781] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:35.801198 25483 raft_consensus.cc:686] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:35.802110 25483 consensus_queue.cc:237] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:35.838200 24199 catalog_manager.cc:5495] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:35.859746 25521 tablet_bootstrap.cc:492] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:35.861188 25521 tablet_bootstrap.cc:492] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:35.861811 25521 ts_tablet_manager.cc:1397] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.221s	user 0.029s	sys 0.084s
I20260924 14:28:35.864194 25521 raft_consensus.cc:348] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:35.865010 25521 raft_consensus.cc:729] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:35.865557 25521 consensus_queue.cc:260] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:35.867267 24370 raft_consensus.cc:2454] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:35.867656 25521 ts_tablet_manager.cc:1428] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260924 14:28:35.869402 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-93b3f9be3c4846a8b75e5e42099e6dbc received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:35.869726 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-93b3f9be3c4846a8b75e5e42099e6dbc on tablet 93b3f9be3c4846a8b75e5e42099e6dbc with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:35.889782 25008 raft_consensus.cc:1203] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:35.949167 25008 raft_consensus.cc:1203] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:35.988790 24446 raft_consensus.cc:1261] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:35.989993 25484 consensus_queue.cc:1035] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:36.015895 24303 raft_consensus.cc:1261] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:36.015980 25458 raft_consensus.cc:1050] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:36.018723 25458 consensus_queue.cc:237] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.024876 25484 consensus_queue.cc:1035] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:36.042083 25008 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.043828 25492 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:36.047103 24449 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.048056 24370 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.048735 25492 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:36.049834 25525 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:36.062659 24446 raft_consensus.cc:1261] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:36.063966 25484 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:36.092326 25492 raft_consensus.cc:2932] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.097143 24370 raft_consensus.cc:2932] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.101820 24372 raft_consensus.cc:1261] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:36.103716 25483 consensus_queue.cc:1035] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:28:36.101255 24446 raft_consensus.cc:2932] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.150209 24200 catalog_manager.cc:5495] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.153633 24446 raft_consensus.cc:1261] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:36.155107 25483 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:36.157287 25009 raft_consensus.cc:2932] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.184980 24298 consensus_queue.cc:237] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.196509 24451 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:36.197512 24300 raft_consensus.cc:1261] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:36.202354 25557 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:36.203287 25008 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:36.213413 25557 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:36.217641 25484 consensus_queue.cc:1035] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:36.225380 25525 raft_consensus.cc:2932] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.243347 24446 raft_consensus.cc:2932] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.246860 25007 raft_consensus.cc:2932] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.256407 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 095719203855447b9091cc7846d566de with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:36.257193 24451 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:36.259282 25543 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:28:36.267151 24372 raft_consensus.cc:1261] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:36.268428 25484 consensus_queue.cc:1035] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:36.269121 24200 catalog_manager.cc:5495] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:36.278882 24449 raft_consensus.cc:1261] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:36.289405 24372 raft_consensus.cc:1261] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:36.272722 25008 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:36.292080 25543 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:36.292820 25564 consensus_queue.cc:1035] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:36.294376 25512 raft_consensus.cc:1050] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:36.298046 25497 consensus_queue.cc:1035] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:28:36.300729 25512 consensus_queue.cc:237] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.315317 24298 raft_consensus.cc:1261] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:36.317386 25481 consensus_queue.cc:1035] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:36.341284 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet 095719203855447b9091cc7846d566de with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:36.346755 25572 tablet_replica.cc:330] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:36.351351 24370 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.353303 25572 raft_consensus.cc:2229] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:36.353293 25512 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:36.360898 25572 raft_consensus.cc:2258] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:36.366674 25572 ts_tablet_manager.cc:1905] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260924 14:28:36.407136 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 78050.5us
I20260924 14:28:36.414981 24298 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.416090 25007 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.442354 25572 ts_tablet_manager.cc:1918] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:36.442945 25572 log.cc:1196] T 095719203855447b9091cc7846d566de P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/095719203855447b9091cc7846d566de
I20260924 14:28:36.444113 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:36.444499 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet 095719203855447b9091cc7846d566de (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:36.449087 25569 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:36.453627 25564 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:36.463019 25558 raft_consensus.cc:1050] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:36.467582 25558 consensus_queue.cc:237] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.480865 25458 raft_consensus.cc:1050] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:36.489719 25009 raft_consensus.cc:1261] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.492957 25558 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:36.495774 24451 raft_consensus.cc:1261] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.501170 25525 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:36.507275 25458 consensus_queue.cc:237] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.510700 24370 raft_consensus.cc:1261] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.517508 25525 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:36.519066 25570 raft_consensus.cc:2932] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.520654 25009 raft_consensus.cc:2932] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.523940 24297 raft_consensus.cc:2932] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.536108 24376 raft_consensus.cc:2932] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.553118 25558 raft_consensus.cc:2932] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.563652 24372 raft_consensus.cc:2932] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.566875 24451 raft_consensus.cc:2932] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.572247 24198 catalog_manager.cc:5495] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.574925 25009 raft_consensus.cc:1261] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.576839 25458 consensus_queue.cc:1035] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:36.586422 24371 raft_consensus.cc:1261] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.586670 25009 raft_consensus.cc:2932] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.593961 25459 consensus_queue.cc:1035] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:36.597741 24446 raft_consensus.cc:1261] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:36.604573 25525 consensus_queue.cc:1035] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:36.622824 24449 consensus_queue.cc:237] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.629213 24197 catalog_manager.cc:5495] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:36.631362 24376 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4995099d467c4203bc2872f519b99950"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:36.632006 24376 raft_consensus.cc:595] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Received request to transfer leadership to TS 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:36.651774 24303 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8b62b316cc05443990141d24402b1f4a"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:36.652398 24303 raft_consensus.cc:595] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Received request to transfer leadership to TS 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:36.654879 24372 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:36.658066 24303 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:36.664261 24303 raft_consensus.cc:595] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:36.669807 25009 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:36.672130 25512 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:36.673969 25569 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:36.677976 25557 raft_consensus.cc:2932] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.684314 24446 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:36.684996 24446 raft_consensus.cc:595] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Received request to transfer leadership to TS 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:36.685978 24297 consensus_queue.cc:237] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.690995 25009 raft_consensus.cc:2932] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.692951 24446 raft_consensus.cc:2932] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.703408 24298 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:36.704305 25569 raft_consensus.cc:979] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:36.704553 24298 raft_consensus.cc:595] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Received request to transfer leadership to TS 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:36.704824 25569 raft_consensus.cc:1067] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Signalling peer 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:36.707357 25007 raft_consensus.cc:1261] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:36.708878 24448 raft_consensus.cc:1261] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:36.709370 24372 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:49700
I20260924 14:28:36.709937 24372 raft_consensus.cc:482] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:36.710309 24372 raft_consensus.cc:3037] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:36.708997 25458 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260924 14:28:36.711165 25459 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:36.712218 25564 raft_consensus.cc:2932] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.713073 25007 raft_consensus.cc:2932] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.722929 25492 raft_consensus.cc:2932] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.724231 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:28:36.726037 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:36.726655 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:36.735989 24449 raft_consensus.cc:2932] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.741770 25008 raft_consensus.cc:2932] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.748297 24199 catalog_manager.cc:5495] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.751920 24371 raft_consensus.cc:2932] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.753424 24372 raft_consensus.cc:504] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.754477 24376 raft_consensus.cc:1226] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Rejecting Update request from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 for earlier term 1. Current term is 2. Ops: []
I20260924 14:28:36.754611 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 93b3f9be3c4846a8b75e5e42099e6dbc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:36.764801 24372 leader_election.cc:288] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:36.766481 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:36.769650 25475 raft_consensus.cc:979] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:36.770116 25475 raft_consensus.cc:1067] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Signalling peer 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:36.771396 24197 catalog_manager.cc:5495] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.773240 24447 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4995099d467c4203bc2872f519b99950"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:48740
I20260924 14:28:36.773761 24447 raft_consensus.cc:482] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:36.774120 24447 raft_consensus.cc:3037] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:36.775584 25009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:36.782194 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet bd5890410f3f4f22a1f26e09cc824bb2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:36.782437 24451 raft_consensus.cc:3032] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:36.782753 24451 raft_consensus.cc:729] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:36.783424 24451 consensus_queue.cc:260] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.784441 25481 consensus_queue.cc:1046] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:36.784703 25009 raft_consensus.cc:3037] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:36.790485 24447 raft_consensus.cc:504] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.792028 25009 raft_consensus.cc:2454] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:36.792763 24451 raft_consensus.cc:3037] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:36.793301 24447 leader_election.cc:288] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 3 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:36.795403 24344 leader_election.cc:302] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:36.797394 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5890410f3f4f22a1f26e09cc824bb2 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:36.798414 25477 raft_consensus.cc:2781] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:36.797592 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:36.799418 24372 raft_consensus.cc:3032] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:28:36.798893 25477 raft_consensus.cc:686] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:36.800798 25477 consensus_queue.cc:237] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.799713 24372 raft_consensus.cc:729] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:36.802776 24372 consensus_queue.cc:260] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.804596 24372 raft_consensus.cc:3037] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:36.795032 25009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4995099d467c4203bc2872f519b99950" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:36.812307 25009 raft_consensus.cc:3037] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:36.817624 24451 raft_consensus.cc:2454] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:36.824733 24372 raft_consensus.cc:2454] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 3.
I20260924 14:28:36.831100 24417 leader_election.cc:302] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:36.831986 25569 raft_consensus.cc:2781] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:28:36.832635 25569 raft_consensus.cc:686] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:36.834122 25569 consensus_queue.cc:237] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.836860 25525 raft_consensus.cc:979] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:36.837328 25525 raft_consensus.cc:1067] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Signalling peer 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:36.857640 24448 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8b62b316cc05443990141d24402b1f4a"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:48734
I20260924 14:28:36.864190 24448 raft_consensus.cc:482] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:36.873371 24448 raft_consensus.cc:3037] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:36.865301 24197 catalog_manager.cc:5495] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.871830 25009 raft_consensus.cc:2454] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 3.
I20260924 14:28:36.872573 24199 catalog_manager.cc:5495] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 4 to 3, term changed from 1 to 2, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194), VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) added, now has a pending config: VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194), VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195), VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.869709 25584 tablet_replica.cc:330] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:36.881381 25584 raft_consensus.cc:2229] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:36.882141 25584 raft_consensus.cc:2258] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:36.887866 25584 ts_tablet_manager.cc:1905] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:36.891290 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet 93b3f9be3c4846a8b75e5e42099e6dbc with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:36.893079 25572 tablet_replica.cc:330] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:36.894017 25572 raft_consensus.cc:2229] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:36.894722 25572 raft_consensus.cc:2258] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:36.894629 24198 catalog_manager.cc:5495] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted, no longer has a pending config: VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194), VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195), VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.895304 24298 consensus_queue.cc:237] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:36.901829 25572 ts_tablet_manager.cc:1905] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:36.906392 24372 raft_consensus.cc:1261] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:36.911502 25557 consensus_queue.cc:1035] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:36.915720 25009 raft_consensus.cc:1261] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:36.920219 25557 consensus_queue.cc:1035] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:36.925318 25584 ts_tablet_manager.cc:1918] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:36.925877 25584 log.cc:1196] T bd5890410f3f4f22a1f26e09cc824bb2 P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/bd5890410f3f4f22a1f26e09cc824bb2
I20260924 14:28:36.927606 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet bd5890410f3f4f22a1f26e09cc824bb2 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:36.928283 25458 raft_consensus.cc:2932] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.932824 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5890410f3f4f22a1f26e09cc824bb2 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:36.934311 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet bd5890410f3f4f22a1f26e09cc824bb2 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:36.941538 24448 raft_consensus.cc:504] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:36.943522 24448 leader_election.cc:288] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:36.943138 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet e838ce343bad4d42aa930f68fe4a4b2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:36.948457 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b62b316cc05443990141d24402b1f4a" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:36.949084 24298 raft_consensus.cc:3032] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:36.949424 24298 raft_consensus.cc:729] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:36.950127 24298 consensus_queue.cc:260] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:36.951191 24298 raft_consensus.cc:3037] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:36.951246 25009 raft_consensus.cc:2932] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.951814 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b62b316cc05443990141d24402b1f4a" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:36.958465 24372 raft_consensus.cc:3037] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:36.958563 25572 ts_tablet_manager.cc:1918] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:36.959136 25572 log.cc:1196] T 93b3f9be3c4846a8b75e5e42099e6dbc P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/93b3f9be3c4846a8b75e5e42099e6dbc
I20260924 14:28:36.961233 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet 93b3f9be3c4846a8b75e5e42099e6dbc (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:36.968885 24372 raft_consensus.cc:2454] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:36.969903 24417 leader_election.cc:302] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:36.970710 25481 raft_consensus.cc:2781] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:36.971323 25481 raft_consensus.cc:686] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:36.971495 24198 catalog_manager.cc:5495] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 2 to 3, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:36.972406 25481 consensus_queue.cc:237] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:36.977846 24376 raft_consensus.cc:2932] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:36.986508 24199 catalog_manager.cc:5495] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:36.989213 24298 raft_consensus.cc:2454] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:37.000928 25459 raft_consensus.cc:979] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:37.001513 25459 raft_consensus.cc:1067] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:37.004725 25009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45634
I20260924 14:28:37.005400 25009 raft_consensus.cc:482] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:37.005795 25009 raft_consensus.cc:3037] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:37.007279 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet e838ce343bad4d42aa930f68fe4a4b2c with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:37.015472 24198 catalog_manager.cc:5495] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:37.029309 25009 raft_consensus.cc:504] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:37.032356 25586 tablet_replica.cc:330] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:37.032896 25009 leader_election.cc:288] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:37.035470 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:37.033965 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18bcddd405d4bbd9d39081d45429923" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:37.044697 24297 raft_consensus.cc:3032] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:37.045114 24297 raft_consensus.cc:729] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:37.045801 24297 consensus_queue.cc:260] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:37.047096 24297 raft_consensus.cc:3037] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:37.051661 24372 raft_consensus.cc:3037] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:37.053296 25586 raft_consensus.cc:2229] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:37.054261 25586 raft_consensus.cc:2258] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:37.054700 24297 raft_consensus.cc:2454] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:37.055708 24979 leader_election.cc:302] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:37.056435 25497 raft_consensus.cc:2781] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:37.056895 25497 raft_consensus.cc:686] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:37.058022 24372 raft_consensus.cc:2454] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:37.057732 25497 consensus_queue.cc:237] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:37.060489 25586 ts_tablet_manager.cc:1905] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:37.062956 25458 raft_consensus.cc:979] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:37.063376 25458 raft_consensus.cc:1067] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Signalling peer 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:37.064849 24372 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:49710
I20260924 14:28:37.065747 24372 raft_consensus.cc:482] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:37.066094 24372 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:37.072455 24372 raft_consensus.cc:504] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:37.072561 24198 catalog_manager.cc:5495] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:37.075344 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:37.075949 24298 raft_consensus.cc:3032] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:37.076287 24372 leader_election.cc:288] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:37.076260 24298 raft_consensus.cc:729] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:37.077476 24298 consensus_queue.cc:260] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:37.078847 24298 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:37.078965 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:37.079739 24448 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:37.079859 25586 ts_tablet_manager.cc:1918] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:37.082952 25586 log.cc:1196] T e838ce343bad4d42aa930f68fe4a4b2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/e838ce343bad4d42aa930f68fe4a4b2c
I20260924 14:28:37.084852 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet e838ce343bad4d42aa930f68fe4a4b2c (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:37.084960 24298 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:37.085985 24344 leader_election.cc:302] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:37.086812 25477 raft_consensus.cc:2781] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:37.087306 25477 raft_consensus.cc:686] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:37.087409 24448 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:37.088238 25477 consensus_queue.cc:237] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:37.100335 24198 catalog_manager.cc:5495] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:37.270279 24449 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:37.271639 25477 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:37.277493 25475 raft_consensus.cc:2932] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:37.281157 24376 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:37.282624 25569 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:28:37.348739 25009 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:37.350368 25480 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:37.361835 25008 raft_consensus.cc:1261] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:37.396649 25512 consensus_queue.cc:1035] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:28:37.464190 24376 raft_consensus.cc:1261] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:37.465504 25569 consensus_queue.cc:1035] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:37.468160 24298 raft_consensus.cc:1261] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:37.469331 25597 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:37.478571 24297 raft_consensus.cc:1261] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:37.479907 25481 consensus_queue.cc:1035] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:37.494431 24449 raft_consensus.cc:1261] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:37.496316 25475 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:37.606469 24376 raft_consensus.cc:1261] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:37.607708 25497 consensus_queue.cc:1035] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:37.616019 24298 raft_consensus.cc:1261] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:37.617552 25606 consensus_queue.cc:1035] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:37.970003 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:38.028123 24448 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:38.028678 24448 raft_consensus.cc:595] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Received request to transfer leadership to TS 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:38.031039 24372 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "86057fee00d44a4c9607d990f2782813"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:38.031615 24372 raft_consensus.cc:595] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:38.033763 24372 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:38.034235 24372 raft_consensus.cc:595] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Received request to transfer leadership to TS cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:38.036942 24300 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fb4aa8a7302f42b688f37d4fdb892b67"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:38.037473 24300 raft_consensus.cc:595] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Received request to transfer leadership to TS 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:38.038703 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260924 14:28:38.040558 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:38.041112 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:38.045789 24372 consensus_queue.cc:237] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:38.052577 24344 consensus_peers.cc:487] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8df40ee44e21496dbc7ef5dd5e624432. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:38.052592 24448 raft_consensus.cc:1261] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:38.054313 24300 raft_consensus.cc:1261] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:38.055321 25480 consensus_queue.cc:1035] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:38.059017 25475 consensus_queue.cc:1035] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:38.080816 25480 raft_consensus.cc:2932] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.082756 24300 raft_consensus.cc:2932] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.092598 24449 raft_consensus.cc:2932] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.100752 24300 consensus_queue.cc:237] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.106884 24199 catalog_manager.cc:5495] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:38.110065 24269 consensus_peers.cc:487] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb1ed21b7eb4447b93b9cd4b579d4031. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:38.110056 24448 raft_consensus.cc:1261] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:38.111754 24372 raft_consensus.cc:1261] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:38.113281 25492 consensus_queue.cc:1035] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:38.115144 25492 consensus_queue.cc:1035] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:38.128996 24372 raft_consensus.cc:2932] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.139770 25475 raft_consensus.cc:979] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:38.140341 25475 raft_consensus.cc:1067] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:38.141793 25009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "86057fee00d44a4c9607d990f2782813"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45626
I20260924 14:28:38.142323 25009 raft_consensus.cc:482] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:38.142625 25009 raft_consensus.cc:3037] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:38.133498 24449 raft_consensus.cc:2932] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.149236 25009 raft_consensus.cc:504] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:38.151758 25009 leader_election.cc:288] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:38.151803 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86057fee00d44a4c9607d990f2782813" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:38.152459 24300 raft_consensus.cc:3037] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:38.160511 25618 raft_consensus.cc:2932] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.159265 24201 catalog_manager.cc:5495] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.167323 24300 raft_consensus.cc:2454] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:38.168522 24979 leader_election.cc:302] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:38.171187 25497 raft_consensus.cc:2781] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:38.171736 25497 raft_consensus.cc:686] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:38.172590 25497 consensus_queue.cc:237] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:38.174798 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86057fee00d44a4c9607d990f2782813" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:38.178125 24372 raft_consensus.cc:3032] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:38.178424 24372 raft_consensus.cc:729] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:38.180188 24376 consensus_queue.cc:237] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.183507 24372 consensus_queue.cc:260] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:38.186883 24372 raft_consensus.cc:3037] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:38.195699 24298 raft_consensus.cc:1261] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:38.195981 24344 consensus_peers.cc:487] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7a8256035984c7fb87f7e9212b21dc7. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:38.195770 24448 raft_consensus.cc:1261] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:38.197379 25597 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:38.198323 25475 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:38.200052 24372 raft_consensus.cc:2454] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:38.211436 25597 raft_consensus.cc:2932] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.219647 24448 raft_consensus.cc:2932] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.218780 24300 raft_consensus.cc:2932] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.233184 24449 consensus_queue.cc:237] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.234826 24201 catalog_manager.cc:5495] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:38.237218 24198 catalog_manager.cc:5495] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260924 14:28:38.244918 24418 consensus_peers.cc:487] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3550e2841c1f41939833b5588fb35875. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:38.245973 24372 raft_consensus.cc:1261] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:38.246582 24298 raft_consensus.cc:1261] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:38.247838 25608 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:38.248780 25481 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:38.261101 25595 raft_consensus.cc:2932] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.271879 24376 raft_consensus.cc:2932] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.286708 24300 raft_consensus.cc:2932] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:38.291754 24199 catalog_manager.cc:5495] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:38.362278 25618 raft_consensus.cc:979] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:38.362784 25620 raft_consensus.cc:1067] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Signalling peer 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:38.364218 24448 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fb4aa8a7302f42b688f37d4fdb892b67"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:48734
I20260924 14:28:38.364709 24448 raft_consensus.cc:482] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:38.365075 24448 raft_consensus.cc:3037] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:38.371803 24448 raft_consensus.cc:504] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:38.374289 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4aa8a7302f42b688f37d4fdb892b67" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:38.374765 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4aa8a7302f42b688f37d4fdb892b67" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:38.374914 24300 raft_consensus.cc:3032] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:38.374562 24448 leader_election.cc:288] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:38.375471 24371 raft_consensus.cc:3037] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:38.375707 24300 raft_consensus.cc:729] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:38.376964 24300 consensus_queue.cc:260] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:38.378194 24300 raft_consensus.cc:3037] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:38.385773 24371 raft_consensus.cc:2454] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:38.386941 24417 leader_election.cc:302] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:38.387696 25512 raft_consensus.cc:2781] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:38.388171 25512 raft_consensus.cc:686] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:38.388478 24300 raft_consensus.cc:2454] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:38.388952 25512 consensus_queue.cc:237] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:38.401496 25480 raft_consensus.cc:979] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:38.402022 25475 raft_consensus.cc:1067] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Signalling peer cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:38.404626 24297 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:49178
I20260924 14:28:38.405215 24297 raft_consensus.cc:482] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:38.405577 24297 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:38.400799 24201 catalog_manager.cc:5495] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:38.416599 24297 raft_consensus.cc:504] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:38.419320 24297 leader_election.cc:288] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 3 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:38.423213 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:38.424013 24449 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:38.428635 25608 raft_consensus.cc:979] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:38.429767 25608 raft_consensus.cc:1067] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Signalling peer 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:38.431282 24371 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:49700
I20260924 14:28:38.431388 24449 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 3.
I20260924 14:28:38.433209 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:38.448271 24376 raft_consensus.cc:3032] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:28:38.448640 24376 raft_consensus.cc:729] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:38.448911 24268 leader_election.cc:302] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:38.449344 24376 consensus_queue.cc:260] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:38.449806 25618 raft_consensus.cc:2781] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:28:38.431921 24371 raft_consensus.cc:482] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:38.450312 25618 raft_consensus.cc:686] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 3 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:38.450934 24371 raft_consensus.cc:3037] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:38.451364 24376 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:38.451327 25618 consensus_queue.cc:237] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:38.461256 24371 raft_consensus.cc:504] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:38.463732 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:38.464344 24300 raft_consensus.cc:3037] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:38.463610 24201 catalog_manager.cc:5495] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 2 to 3, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:38.465194 24371 leader_election.cc:288] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:38.466751 24376 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 3.
I20260924 14:28:38.468665 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a06a0e1988d4a53b9590e40ca1481b0" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:38.470058 24449 raft_consensus.cc:3032] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:38.470391 24449 raft_consensus.cc:729] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:38.471251 24449 consensus_queue.cc:260] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:38.472667 24449 raft_consensus.cc:3037] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:38.476300 24300 raft_consensus.cc:2454] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:38.479803 24344 leader_election.cc:302] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:38.480753 25480 raft_consensus.cc:2781] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:38.481421 25480 raft_consensus.cc:686] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:38.482234 25480 consensus_queue.cc:237] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:38.483403 24449 raft_consensus.cc:2454] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:38.494966 24201 catalog_manager.cc:5495] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 1 to 2, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:38.559857 24376 raft_consensus.cc:1261] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:38.561591 25497 consensus_queue.cc:1035] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:38.575186 24300 raft_consensus.cc:1261] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:38.581171 25497 consensus_queue.cc:1035] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260924 14:28:38.592115 25636 ts_tablet_manager.cc:927] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:38.598414 25636 tablet_copy_client.cc:316] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:38.601464 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet 8df40ee44e21496dbc7ef5dd5e624432 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:38.601987 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet 8df40ee44e21496dbc7ef5dd5e624432 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-8df40ee44e21496dbc7ef5dd5e624432
I20260924 14:28:38.612004 24387 tablet_copy_source_session.cc:205] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:38.619035 25636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df40ee44e21496dbc7ef5dd5e624432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:38.632287 25636 tablet_copy_client.cc:791] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:38.632835 25636 tablet_copy_client.cc:655] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:38.637041 25636 tablet_copy_client.cc:531] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:38.644547 25636 tablet_bootstrap.cc:492] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:38.664467 25636 tablet_bootstrap.cc:492] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:38.665200 25636 tablet_bootstrap.cc:492] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:38.665755 25636 ts_tablet_manager.cc:1397] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260924 14:28:38.668256 25636 raft_consensus.cc:348] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.668902 25636 raft_consensus.cc:729] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:38.669535 25636 consensus_queue.cc:260] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.672986 25636 ts_tablet_manager.cc:1428] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:28:38.675050 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-8df40ee44e21496dbc7ef5dd5e624432 received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:38.675432 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-8df40ee44e21496dbc7ef5dd5e624432 on tablet 8df40ee44e21496dbc7ef5dd5e624432 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:38.684801 25636 ts_tablet_manager.cc:927] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:38.687595 25636 tablet_copy_client.cc:316] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:38.689172 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet d7a8256035984c7fb87f7e9212b21dc7 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:38.689730 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet d7a8256035984c7fb87f7e9212b21dc7 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-d7a8256035984c7fb87f7e9212b21dc7
I20260924 14:28:38.697767 24387 tablet_copy_source_session.cc:205] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:38.700539 25636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7a8256035984c7fb87f7e9212b21dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:38.717412 25636 tablet_copy_client.cc:791] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:38.717999 25636 tablet_copy_client.cc:655] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:38.729102 25636 tablet_copy_client.cc:531] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:38.735898 25640 ts_tablet_manager.cc:927] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:38.737542 25640 tablet_copy_client.cc:316] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:38.739023 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet bb1ed21b7eb4447b93b9cd4b579d4031 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:38.739560 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet bb1ed21b7eb4447b93b9cd4b579d4031 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-bb1ed21b7eb4447b93b9cd4b579d4031
I20260924 14:28:38.745136 25636 tablet_bootstrap.cc:492] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:38.747603 24313 tablet_copy_source_session.cc:205] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:38.751557 25640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1ed21b7eb4447b93b9cd4b579d4031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:38.769852 25636 tablet_bootstrap.cc:492] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:38.770524 25636 tablet_bootstrap.cc:492] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:38.771157 25636 ts_tablet_manager.cc:1397] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.014s
I20260924 14:28:38.773922 25636 raft_consensus.cc:348] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.774766 25636 raft_consensus.cc:729] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:38.775398 25636 consensus_queue.cc:260] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.780808 25640 tablet_copy_client.cc:791] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:38.781428 25640 tablet_copy_client.cc:655] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:38.781543 25636 ts_tablet_manager.cc:1428] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260924 14:28:38.783041 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-d7a8256035984c7fb87f7e9212b21dc7 received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:38.783422 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-d7a8256035984c7fb87f7e9212b21dc7 on tablet d7a8256035984c7fb87f7e9212b21dc7 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:38.786334 25640 tablet_copy_client.cc:531] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:38.795279 25640 tablet_bootstrap.cc:492] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:38.800465 25636 ts_tablet_manager.cc:927] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:38.802163 25636 tablet_copy_client.cc:316] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:38.803599 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet 3550e2841c1f41939833b5588fb35875 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:38.804121 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet 3550e2841c1f41939833b5588fb35875 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-3550e2841c1f41939833b5588fb35875
I20260924 14:28:38.814759 25640 tablet_bootstrap.cc:492] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:38.815670 25640 tablet_bootstrap.cc:492] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:38.815830 24462 tablet_copy_source_session.cc:205] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:38.816334 25640 ts_tablet_manager.cc:1397] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.007s
I20260924 14:28:38.817786 24297 raft_consensus.cc:1261] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:38.819096 25640 raft_consensus.cc:348] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.819978 25640 raft_consensus.cc:729] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:38.820176 25595 consensus_queue.cc:1035] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:38.821179 25636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3550e2841c1f41939833b5588fb35875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:38.820583 25640 consensus_queue.cc:260] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.823534 25640 ts_tablet_manager.cc:1428] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260924 14:28:38.829587 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-bb1ed21b7eb4447b93b9cd4b579d4031 received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:38.829988 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-bb1ed21b7eb4447b93b9cd4b579d4031 on tablet bb1ed21b7eb4447b93b9cd4b579d4031 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:38.844215 25636 tablet_copy_client.cc:791] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:38.844877 25636 tablet_copy_client.cc:655] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:38.849491 24373 raft_consensus.cc:1261] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:38.850956 25595 consensus_queue.cc:1035] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:38.856503 25636 tablet_copy_client.cc:531] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:38.871788 25636 tablet_bootstrap.cc:492] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:38.892443 25636 tablet_bootstrap.cc:492] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:38.893290 25636 tablet_bootstrap.cc:492] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:38.893903 25636 ts_tablet_manager.cc:1397] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260924 14:28:38.896212 25636 raft_consensus.cc:348] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.897027 25636 raft_consensus.cc:729] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:38.897606 25636 consensus_queue.cc:260] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:38.899785 25636 ts_tablet_manager.cc:1428] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260924 14:28:38.901471 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-3550e2841c1f41939833b5588fb35875 received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:38.901863 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-3550e2841c1f41939833b5588fb35875 on tablet 3550e2841c1f41939833b5588fb35875 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:38.906656 24451 raft_consensus.cc:1261] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:28:38.907886 25618 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:38.936702 24449 raft_consensus.cc:1261] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:38.937294 24376 raft_consensus.cc:1261] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:28:38.938768 25475 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:38.938978 25618 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260924 14:28:39.013543 24979 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 45898.8us
I20260924 14:28:39.029996 25009 raft_consensus.cc:1203] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:39.057096 24300 raft_consensus.cc:1261] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260924 14:28:39.076500 24343 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 57350.4us
I20260924 14:28:39.081084 25475 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:39.136044 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:39.214753 24300 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "095719203855447b9091cc7846d566de"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:39.215408 24300 raft_consensus.cc:595] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Received request to transfer leadership to TS 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:39.216019 25009 raft_consensus.cc:1203] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:39.220685 24449 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3550e2841c1f41939833b5588fb35875"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:39.221236 24449 raft_consensus.cc:595] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Received request to transfer leadership to TS 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:39.226861 24373 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:39.227496 24373 raft_consensus.cc:595] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Received request to transfer leadership to TS cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:39.228744 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260924 14:28:39.229940 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:39.230500 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:39.242591 25009 raft_consensus.cc:1203] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:39.304617 25595 raft_consensus.cc:979] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:39.305770 25660 raft_consensus.cc:979] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:39.305119 25595 raft_consensus.cc:1067] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Signalling peer 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:39.306319 25660 raft_consensus.cc:1067] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Signalling peer cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:39.307523 24376 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3550e2841c1f41939833b5588fb35875"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:49700
I20260924 14:28:39.307724 24300 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:49178
I20260924 14:28:39.308103 24376 raft_consensus.cc:482] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:39.308178 24300 raft_consensus.cc:482] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:39.308496 24376 raft_consensus.cc:3037] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:39.308540 24300 raft_consensus.cc:3037] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:39.314322 24376 raft_consensus.cc:504] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:39.316371 24376 leader_election.cc:288] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:39.317829 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3550e2841c1f41939833b5588fb35875" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:39.318383 24298 raft_consensus.cc:3037] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:39.318616 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3550e2841c1f41939833b5588fb35875" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:39.319262 24451 raft_consensus.cc:3032] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:39.319620 24451 raft_consensus.cc:729] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:39.320363 24451 consensus_queue.cc:260] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:39.321973 24451 raft_consensus.cc:3037] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:39.323805 24300 raft_consensus.cc:504] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.325331 24298 raft_consensus.cc:2454] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:39.326436 24344 leader_election.cc:302] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:39.327122 25660 raft_consensus.cc:2781] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:39.327593 25660 raft_consensus.cc:686] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:39.327749 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:39.328307 24376 raft_consensus.cc:3032] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:39.328604 24376 raft_consensus.cc:729] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:39.328493 25660 consensus_queue.cc:237] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:39.329180 24300 leader_election.cc:288] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 2 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:39.329924 25009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:39.330464 25009 raft_consensus.cc:3037] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:39.329384 24376 consensus_queue.cc:260] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.331543 24376 raft_consensus.cc:3037] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:39.340030 24376 raft_consensus.cc:2454] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 2.
I20260924 14:28:39.341259 24268 leader_election.cc:302] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:39.344161 25654 raft_consensus.cc:2781] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:39.344769 25654 raft_consensus.cc:686] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 2 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:39.345571 25654 consensus_queue.cc:237] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.349385 25009 raft_consensus.cc:2454] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 2.
I20260924 14:28:39.353197 24451 raft_consensus.cc:2454] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:39.363602 24198 catalog_manager.cc:5495] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 1 to 2, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:39.365650 24201 catalog_manager.cc:5495] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 1 to 2, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:39.418233 25660 raft_consensus.cc:1050] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:39.420923 25660 consensus_queue.cc:237] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.426980 24451 raft_consensus.cc:1261] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:39.427006 25009 raft_consensus.cc:1261] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:39.430656 25597 consensus_queue.cc:1035] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:39.431641 25660 consensus_queue.cc:1035] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:39.432006 24298 raft_consensus.cc:1261] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:39.433606 25659 consensus_queue.cc:1035] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:39.441993 25597 raft_consensus.cc:2932] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.451891 25009 raft_consensus.cc:2932] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.458325 24451 raft_consensus.cc:2932] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.461159 24300 raft_consensus.cc:2932] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.464483 24201 catalog_manager.cc:5495] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:39.484969 24373 consensus_queue.cc:237] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.496236 25009 raft_consensus.cc:1261] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:39.497836 25597 consensus_queue.cc:1035] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:39.499693 24451 raft_consensus.cc:1261] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:39.500890 25597 consensus_queue.cc:1035] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:39.504388 25660 raft_consensus.cc:2932] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.505946 25009 raft_consensus.cc:2932] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.510274 24451 raft_consensus.cc:2932] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.515772 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 8df40ee44e21496dbc7ef5dd5e624432 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:39.527694 24198 catalog_manager.cc:5495] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:39.556124 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet 8df40ee44e21496dbc7ef5dd5e624432 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:39.558890 25677 tablet_replica.cc:330] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:39.560009 25677 raft_consensus.cc:2229] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:39.560730 25677 raft_consensus.cc:2258] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:39.563946 25677 ts_tablet_manager.cc:1905] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:39.585388 25656 raft_consensus.cc:979] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:39.585918 25656 raft_consensus.cc:1067] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Signalling peer 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:39.586898 25677 ts_tablet_manager.cc:1918] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:39.587410 25677 log.cc:1196] T 8df40ee44e21496dbc7ef5dd5e624432 P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/8df40ee44e21496dbc7ef5dd5e624432
I20260924 14:28:39.587356 24449 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "095719203855447b9091cc7846d566de"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:48734
I20260924 14:28:39.588040 24449 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:39.588392 24449 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:39.590041 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet 8df40ee44e21496dbc7ef5dd5e624432 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:39.595554 24449 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.597482 24449 leader_election.cc:288] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:39.598280 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:39.598361 25009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:39.598991 24297 raft_consensus.cc:3032] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:39.599054 25009 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:39.599385 24297 raft_consensus.cc:729] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:39.600031 24297 consensus_queue.cc:260] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.600859 24297 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:39.605567 25009 raft_consensus.cc:2454] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:39.606858 24418 leader_election.cc:302] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:39.607576 24297 raft_consensus.cc:2454] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:39.609269 25665 raft_consensus.cc:2781] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:39.609954 25665 raft_consensus.cc:686] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:39.610837 25665 consensus_queue.cc:237] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.611980 25666 raft_consensus.cc:1050] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:39.617898 25666 consensus_queue.cc:237] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.625900 25008 raft_consensus.cc:1261] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:39.627318 25656 consensus_queue.cc:1035] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:39.633885 24451 raft_consensus.cc:1261] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:39.634941 24373 raft_consensus.cc:1261] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:39.636497 25666 consensus_queue.cc:1035] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:39.638875 25618 consensus_queue.cc:1035] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:39.645525 24198 catalog_manager.cc:5495] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:39.653255 25618 raft_consensus.cc:2932] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.672739 25009 raft_consensus.cc:2932] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.673040 24373 raft_consensus.cc:2932] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.676422 24451 raft_consensus.cc:2932] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.682879 24198 catalog_manager.cc:5495] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:39.711627 24297 consensus_queue.cc:237] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.716457 25009 raft_consensus.cc:1261] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:39.718120 25618 consensus_queue.cc:1035] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:39.720649 24373 raft_consensus.cc:1261] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:39.722429 25666 consensus_queue.cc:1035] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:39.730187 25666 raft_consensus.cc:2932] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.736914 25008 raft_consensus.cc:2932] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.733528 24371 raft_consensus.cc:2932] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.752421 24373 raft_consensus.cc:1261] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:39.753355 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet bb1ed21b7eb4447b93b9cd4b579d4031 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:39.757009 24451 raft_consensus.cc:1261] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 14:28:39.760695 25659 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:39.761428 25654 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:39.758489 24199 catalog_manager.cc:5495] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.805012 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet bb1ed21b7eb4447b93b9cd4b579d4031 with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:39.806113 25008 raft_consensus.cc:3037] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Advancing to term 2
I20260924 14:28:39.806954 24297 raft_consensus.cc:1261] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 14:28:39.808400 25659 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:39.839264 25009 raft_consensus.cc:1261] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:39.841339 25654 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:39.841038 25660 raft_consensus.cc:1050] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:39.844848 25660 consensus_queue.cc:237] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.857159 25008 raft_consensus.cc:1261] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 14:28:39.858856 25659 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:39.865281 25689 tablet_replica.cc:330] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:39.866581 25689 raft_consensus.cc:2229] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:39.867480 25689 raft_consensus.cc:2258] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:39.870819 25689 ts_tablet_manager.cc:1905] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:39.879124 25007 raft_consensus.cc:1261] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:39.880062 24449 raft_consensus.cc:1261] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:39.881556 25660 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:39.882118 24297 raft_consensus.cc:1261] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:39.884656 25659 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:39.896296 25597 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:39.922649 25689 ts_tablet_manager.cc:1918] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:39.923107 25689 log.cc:1196] T bb1ed21b7eb4447b93b9cd4b579d4031 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/bb1ed21b7eb4447b93b9cd4b579d4031
I20260924 14:28:39.924985 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet bb1ed21b7eb4447b93b9cd4b579d4031 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:39.940979 25659 raft_consensus.cc:2932] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.945768 25480 raft_consensus.cc:1050] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:39.956916 25480 consensus_queue.cc:237] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:39.959093 25009 raft_consensus.cc:2932] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.968608 24451 raft_consensus.cc:2932] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.990854 25009 raft_consensus.cc:1261] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 14:28:39.992458 25480 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:39.997543 24300 raft_consensus.cc:1261] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 14:28:39.998929 24451 raft_consensus.cc:1261] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 14:28:40.000370 24197 catalog_manager.cc:5495] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:39.999943 24300 raft_consensus.cc:2932] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.001993 25660 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:40.008548 25480 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:40.028231 24451 raft_consensus.cc:2932] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.022630 25480 raft_consensus.cc:2932] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.042857 24300 raft_consensus.cc:2932] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.046738 25009 raft_consensus.cc:2932] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.049139 24376 consensus_queue.cc:237] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:40.050719 24197 catalog_manager.cc:5495] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 2 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.057320 25007 raft_consensus.cc:1261] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:40.065269 25701 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:40.069183 24298 raft_consensus.cc:1261] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:40.075570 25701 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:40.076037 24300 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:40.080556 25700 raft_consensus.cc:2932] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.081449 25481 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:40.092474 25009 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:40.093798 25665 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:40.099737 24371 consensus_queue.cc:237] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:40.116362 24201 catalog_manager.cc:5495] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:40.117933 24451 raft_consensus.cc:1261] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:40.120518 25701 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:40.123875 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet d7a8256035984c7fb87f7e9212b21dc7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:40.130744 25008 raft_consensus.cc:1261] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:40.130064 25660 raft_consensus.cc:2932] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.131906 25009 raft_consensus.cc:2932] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.131938 24449 raft_consensus.cc:2932] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.142547 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet d7a8256035984c7fb87f7e9212b21dc7 with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:40.143575 25689 tablet_replica.cc:330] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:40.144546 25689 raft_consensus.cc:2229] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:40.145285 25689 raft_consensus.cc:2258] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:40.147343 24298 raft_consensus.cc:2932] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.147436 25700 consensus_queue.cc:1035] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260924 14:28:40.148140 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 3550e2841c1f41939833b5588fb35875 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:40.153746 25689 ts_tablet_manager.cc:1905] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:40.173717 24199 catalog_manager.cc:5495] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:40.191744 25009 raft_consensus.cc:2932] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:40.204105 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet 3550e2841c1f41939833b5588fb35875 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:40.215437 25706 tablet_replica.cc:330] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:40.216655 25706 raft_consensus.cc:2229] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:40.217479 25706 raft_consensus.cc:2258] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:40.221755 25689 ts_tablet_manager.cc:1918] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:40.222268 25689 log.cc:1196] T d7a8256035984c7fb87f7e9212b21dc7 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/d7a8256035984c7fb87f7e9212b21dc7
I20260924 14:28:40.223127 25706 ts_tablet_manager.cc:1905] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:40.223891 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet d7a8256035984c7fb87f7e9212b21dc7 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:40.235433 25706 ts_tablet_manager.cc:1918] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260924 14:28:40.235978 25706 log.cc:1196] T 3550e2841c1f41939833b5588fb35875 P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/3550e2841c1f41939833b5588fb35875
I20260924 14:28:40.237347 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet 3550e2841c1f41939833b5588fb35875 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:40.283490 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:40.322577 24449 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:40.323194 24449 raft_consensus.cc:595] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Received request to transfer leadership to TS 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:40.325469 24449 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:40.325965 24449 raft_consensus.cc:595] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Received request to transfer leadership to TS cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:40.328028 24298 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:40.328490 24298 raft_consensus.cc:595] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Received request to transfer leadership to TS 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:40.331634 24376 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:40.332258 24376 raft_consensus.cc:595] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:40.333408 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260924 14:28:40.334897 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:40.335497 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:40.452433 25704 raft_consensus.cc:979] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:40.453001 25704 raft_consensus.cc:1067] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Signalling peer 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:40.454485 24371 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:49700
I20260924 14:28:40.455072 24371 raft_consensus.cc:482] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:40.455415 24371 raft_consensus.cc:3037] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:40.460507 24371 raft_consensus.cc:504] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:40.461920 24371 leader_election.cc:288] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 3 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:40.462931 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:40.462931 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8ea1ccd8b24bb3a8afb3e6f673c774" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:40.463779 24449 raft_consensus.cc:3032] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:28:40.463810 24298 raft_consensus.cc:3037] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:40.464192 24449 raft_consensus.cc:729] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:40.464917 24449 consensus_queue.cc:260] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:40.466084 24449 raft_consensus.cc:3037] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:40.470894 24298 raft_consensus.cc:2454] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 3.
I20260924 14:28:40.472208 24344 leader_election.cc:302] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:40.473030 25660 raft_consensus.cc:2781] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:28:40.473771 24449 raft_consensus.cc:2454] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 3.
I20260924 14:28:40.474462 25660 raft_consensus.cc:686] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 3 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:40.475358 25660 consensus_queue.cc:237] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:40.486164 24199 catalog_manager.cc:5495] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 2 to 3, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 3 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:40.496078 25700 raft_consensus.cc:979] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:40.496650 25660 raft_consensus.cc:1067] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:40.498298 25009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45626
I20260924 14:28:40.498832 25009 raft_consensus.cc:482] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:40.499167 25009 raft_consensus.cc:3037] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:40.504874 25009 raft_consensus.cc:504] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:40.506362 25009 leader_election.cc:288] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:40.507380 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:40.507792 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5890410f3f4f22a1f26e09cc824bb2" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:40.508111 24371 raft_consensus.cc:3032] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:28:40.508376 24449 raft_consensus.cc:3037] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:40.508420 24371 raft_consensus.cc:729] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:40.509258 24371 consensus_queue.cc:260] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:40.510548 24371 raft_consensus.cc:3037] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:40.516564 24449 raft_consensus.cc:2454] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 3.
I20260924 14:28:40.517720 24371 raft_consensus.cc:2454] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 3.
I20260924 14:28:40.517923 24978 leader_election.cc:302] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:40.518713 25655 raft_consensus.cc:2781] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:28:40.519268 25655 raft_consensus.cc:686] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 3 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:40.520150 25655 consensus_queue.cc:237] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:40.535085 24199 catalog_manager.cc:5495] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 2 to 3, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 3 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:40.576869 25656 raft_consensus.cc:979] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:40.577396 25618 raft_consensus.cc:1067] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Signalling peer 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:40.578804 24449 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:48734
I20260924 14:28:40.579311 24449 raft_consensus.cc:482] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:40.579679 24449 raft_consensus.cc:3037] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:40.584188 24449 raft_consensus.cc:504] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:40.585562 24449 leader_election.cc:288] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:40.586474 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:40.586483 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b642240b2a48ff984338b3604080ef" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:40.587424 24371 raft_consensus.cc:3037] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:40.587424 24298 raft_consensus.cc:3032] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:40.587953 24298 raft_consensus.cc:729] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:40.588711 24298 consensus_queue.cc:260] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:40.589901 24298 raft_consensus.cc:3037] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:40.594365 24371 raft_consensus.cc:2454] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:40.595582 24417 leader_election.cc:302] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:40.596346 25665 raft_consensus.cc:2781] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:40.596925 25665 raft_consensus.cc:686] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:40.597527 24298 raft_consensus.cc:2454] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:40.597769 25665 consensus_queue.cc:237] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } }
I20260924 14:28:40.611084 24199 catalog_manager.cc:5495] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:40.775720 25481 raft_consensus.cc:979] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:40.790643 25481 raft_consensus.cc:1067] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Signalling peer cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:40.800881 24297 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:49166
I20260924 14:28:40.802161 24297 raft_consensus.cc:482] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:40.807065 24297 raft_consensus.cc:3037] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:40.848969 24297 raft_consensus.cc:504] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:40.853310 24297 leader_election.cc:288] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 2 election: Requested vote from peers 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:40.854480 25007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:40.855484 25007 raft_consensus.cc:3037] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:40.861203 24449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:40.862077 24449 raft_consensus.cc:3032] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:40.862427 24449 raft_consensus.cc:729] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:40.863240 24449 consensus_queue.cc:260] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:40.864967 24449 raft_consensus.cc:3037] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:40.867599 24451 raft_consensus.cc:1261] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:28:40.875357 24449 raft_consensus.cc:2454] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 2.
I20260924 14:28:40.876277 25007 raft_consensus.cc:2454] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 2.
I20260924 14:28:40.876672 25660 consensus_queue.cc:1035] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:40.877581 24269 leader_election.cc:302] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:40.878508 25618 raft_consensus.cc:2781] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:40.878998 25618 raft_consensus.cc:686] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:40.887617 25618 consensus_queue.cc:237] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:40.918015 24201 catalog_manager.cc:5495] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 1 to 2, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:40.948875 24297 raft_consensus.cc:1261] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:28:40.950908 25480 consensus_queue.cc:1035] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:40.984804 24371 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:40.986192 25714 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:28:41.000946 24448 raft_consensus.cc:1261] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:41.002774 25705 consensus_queue.cc:1035] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:28:41.143639 24297 raft_consensus.cc:1261] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:41.145113 25704 consensus_queue.cc:1035] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:41.155259 24371 raft_consensus.cc:1261] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:41.158962 25665 consensus_queue.cc:1035] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:41.283676 24297 consensus_queue.cc:237] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:41.291168 24448 raft_consensus.cc:1261] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:41.291548 24269 consensus_peers.cc:487] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cbae9b0be0e4421eab738bd8fd45a1e1. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:41.294687 24371 raft_consensus.cc:1261] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:41.296000 25656 consensus_queue.cc:1035] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:41.297029 25654 consensus_queue.cc:1035] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:41.305997 25618 raft_consensus.cc:2932] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.308074 24448 raft_consensus.cc:2932] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.307760 24371 raft_consensus.cc:2932] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.321254 25009 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:41.322817 25656 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:41.324249 24447 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:41.325691 25656 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:41.326661 24376 consensus_queue.cc:237] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:41.348162 24448 raft_consensus.cc:1261] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:41.350186 24344 consensus_peers.cc:487] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a57dc14600c450485f596239b30df0d. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:41.350373 24298 raft_consensus.cc:1261] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:41.351953 25660 consensus_queue.cc:1035] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:41.363891 25700 consensus_queue.cc:1035] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:41.375321 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:41.386380 24201 catalog_manager.cc:5495] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.403303 25480 raft_consensus.cc:2932] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:41.408704 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 51684.5us
I20260924 14:28:41.405032 24297 raft_consensus.cc:2932] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.432411 24447 raft_consensus.cc:2932] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.453002 24451 consensus_queue.cc:237] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:41.464285 24418 consensus_peers.cc:487] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2e4953bd67e40f5823697e67d029f52. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:41.464534 24371 raft_consensus.cc:1261] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:41.463064 24198 catalog_manager.cc:5495] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.466144 25665 consensus_queue.cc:1035] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:41.479470 24300 raft_consensus.cc:1261] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:41.481694 25712 consensus_queue.cc:1035] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:41.483574 24300 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:41.484329 24300 raft_consensus.cc:595] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 3 LEADER]: Received request to transfer leadership to TS 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:41.489666 24451 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e4aee388a7474ac1bb46b761831b809e"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:41.490245 24451 raft_consensus.cc:595] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Received request to transfer leadership to TS 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:41.495940 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260924 14:28:41.497763 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:41.498538 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:41.510253 25704 raft_consensus.cc:2932] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.511972 24297 raft_consensus.cc:2932] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.523690 24376 raft_consensus.cc:2932] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.529959 24298 consensus_queue.cc:237] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:41.533025 24201 catalog_manager.cc:5495] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260924 14:28:41.556798 24269 consensus_peers.cc:487] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 870bbbde2a2f4c76bb558d8c43d7587b. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:41.559049 24448 raft_consensus.cc:1261] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:41.560572 25654 consensus_queue.cc:1035] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:41.560875 24376 raft_consensus.cc:1261] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:41.562240 25656 consensus_queue.cc:1035] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:41.576077 25654 raft_consensus.cc:2932] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.577797 24376 raft_consensus.cc:2932] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.583370 24448 raft_consensus.cc:2932] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:41.593559 24201 catalog_manager.cc:5495] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:41.621896 25732 raft_consensus.cc:979] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:41.622458 25618 raft_consensus.cc:1067] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 3 LEADER]: Signalling peer 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:41.624195 24451 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:48734
I20260924 14:28:41.624702 24451 raft_consensus.cc:482] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:41.625051 24451 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:28:41.632570 24451 raft_consensus.cc:504] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:41.635301 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:41.635895 24297 raft_consensus.cc:3032] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:28:41.636222 24297 raft_consensus.cc:729] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:41.636895 24297 consensus_queue.cc:260] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:41.637938 24297 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:28:41.644685 24297 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 4.
I20260924 14:28:41.645844 24418 leader_election.cc:302] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:41.646628 25712 raft_consensus.cc:2781] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:28:41.648707 25712 raft_consensus.cc:686] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:41.648859 24451 leader_election.cc:288] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:41.649658 25712 consensus_queue.cc:237] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:41.650030 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:41.650619 24376 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:28:41.656878 24376 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 4.
I20260924 14:28:41.661405 24201 catalog_manager.cc:5495] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 3 to 4, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 4 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:41.770395 25665 raft_consensus.cc:979] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:41.770926 25712 raft_consensus.cc:1067] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Signalling peer 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:41.773552 24376 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e4aee388a7474ac1bb46b761831b809e"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:49700
I20260924 14:28:41.774507 24376 raft_consensus.cc:482] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:41.774830 24376 raft_consensus.cc:3037] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:41.783874 24376 raft_consensus.cc:504] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:41.785954 24376 leader_election.cc:288] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:41.787106 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4aee388a7474ac1bb46b761831b809e" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:41.787734 24298 raft_consensus.cc:3037] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:41.787664 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4aee388a7474ac1bb46b761831b809e" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:41.788399 24448 raft_consensus.cc:3032] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:41.788702 24448 raft_consensus.cc:729] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:41.789364 24448 consensus_queue.cc:260] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:41.790217 24448 raft_consensus.cc:3037] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:41.799559 24448 raft_consensus.cc:2454] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:41.802910 24343 leader_election.cc:302] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:41.803782 25480 raft_consensus.cc:2781] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:41.804191 25480 raft_consensus.cc:686] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:41.804855 25480 consensus_queue.cc:237] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:41.807047 24298 raft_consensus.cc:2454] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:41.818853 25742 ts_tablet_manager.cc:927] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:41.825546 25742 tablet_copy_client.cc:316] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:41.825695 24201 catalog_manager.cc:5495] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 1 to 2, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:41.828204 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet 3a57dc14600c450485f596239b30df0d from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:41.828801 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet 3a57dc14600c450485f596239b30df0d from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-3a57dc14600c450485f596239b30df0d
I20260924 14:28:41.839772 24387 tablet_copy_source_session.cc:205] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:41.843412 25742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a57dc14600c450485f596239b30df0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:41.859414 25742 tablet_copy_client.cc:791] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:41.860266 25742 tablet_copy_client.cc:655] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:41.863641 25742 tablet_copy_client.cc:531] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:41.871196 25742 tablet_bootstrap.cc:492] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:41.891376 25742 tablet_bootstrap.cc:492] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:41.892278 25742 tablet_bootstrap.cc:492] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:41.892958 25742 ts_tablet_manager.cc:1397] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.005s
I20260924 14:28:41.896018 25742 raft_consensus.cc:348] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:41.896726 25742 raft_consensus.cc:729] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:41.897374 25742 consensus_queue.cc:260] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:41.902046 25742 ts_tablet_manager.cc:1428] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260924 14:28:41.908661 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-3a57dc14600c450485f596239b30df0d received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:41.909070 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-3a57dc14600c450485f596239b30df0d on tablet 3a57dc14600c450485f596239b30df0d with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:41.911944 25742 ts_tablet_manager.cc:927] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:41.918505 25742 tablet_copy_client.cc:316] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:41.920392 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet cbae9b0be0e4421eab738bd8fd45a1e1 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:41.920923 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet cbae9b0be0e4421eab738bd8fd45a1e1 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-cbae9b0be0e4421eab738bd8fd45a1e1
I20260924 14:28:41.935407 24313 tablet_copy_source_session.cc:205] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:41.938076 25742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbae9b0be0e4421eab738bd8fd45a1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:41.953743 25742 tablet_copy_client.cc:791] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:41.954320 25742 tablet_copy_client.cc:655] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:41.960901 25742 tablet_copy_client.cc:531] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:41.969725 25742 tablet_bootstrap.cc:492] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:41.988745 25742 tablet_bootstrap.cc:492] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:41.989590 25742 tablet_bootstrap.cc:492] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:41.990341 25742 ts_tablet_manager.cc:1397] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260924 14:28:41.993031 25742 raft_consensus.cc:348] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:41.993868 25742 raft_consensus.cc:729] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:41.994462 25742 consensus_queue.cc:260] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:41.998442 25742 ts_tablet_manager.cc:1428] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260924 14:28:42.000012 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-cbae9b0be0e4421eab738bd8fd45a1e1 received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:42.000406 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-cbae9b0be0e4421eab738bd8fd45a1e1 on tablet cbae9b0be0e4421eab738bd8fd45a1e1 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:42.036461 24298 raft_consensus.cc:1261] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 4 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:28:42.037948 25712 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:42.056934 25742 ts_tablet_manager.cc:927] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:42.058606 24376 raft_consensus.cc:1261] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 4 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:28:42.058606 25742 tablet_copy_client.cc:316] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:42.061062 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet c2e4953bd67e40f5823697e67d029f52 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:42.061534 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet c2e4953bd67e40f5823697e67d029f52 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-c2e4953bd67e40f5823697e67d029f52
I20260924 14:28:42.063784 25712 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:42.071864 25746 ts_tablet_manager.cc:927] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:42.089268 25746 tablet_copy_client.cc:316] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:42.090024 24462 tablet_copy_source_session.cc:205] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:42.095399 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet 870bbbde2a2f4c76bb558d8c43d7587b from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:42.095993 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet 870bbbde2a2f4c76bb558d8c43d7587b from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-870bbbde2a2f4c76bb558d8c43d7587b
I20260924 14:28:42.107267 25742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e4953bd67e40f5823697e67d029f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:42.108745 24313 tablet_copy_source_session.cc:205] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:42.113941 25746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 870bbbde2a2f4c76bb558d8c43d7587b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:42.122130 25742 tablet_copy_client.cc:791] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:42.122718 25742 tablet_copy_client.cc:655] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:42.130343 25742 tablet_copy_client.cc:531] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:42.131649 25746 tablet_copy_client.cc:791] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:42.132241 25746 tablet_copy_client.cc:655] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:42.135779 25746 tablet_copy_client.cc:531] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:42.138618 25742 tablet_bootstrap.cc:492] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:42.156759 25746 tablet_bootstrap.cc:492] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:42.192940 25742 tablet_bootstrap.cc:492] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:42.193269 25746 tablet_bootstrap.cc:492] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:42.193835 25742 tablet_bootstrap.cc:492] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:42.194314 25746 tablet_bootstrap.cc:492] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:42.194499 25742 ts_tablet_manager.cc:1397] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.056s	user 0.022s	sys 0.021s
I20260924 14:28:42.194991 25746 ts_tablet_manager.cc:1397] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.038s	user 0.018s	sys 0.010s
I20260924 14:28:42.197435 25007 raft_consensus.cc:1203] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:42.206316 25746 raft_consensus.cc:348] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:42.207211 25746 raft_consensus.cc:729] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:42.207851 25746 consensus_queue.cc:260] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:42.198195 25742 raft_consensus.cc:348] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:42.210412 25742 raft_consensus.cc:729] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:42.210716 25746 ts_tablet_manager.cc:1428] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:28:42.211158 25742 consensus_queue.cc:260] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:42.214176 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-870bbbde2a2f4c76bb558d8c43d7587b received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:42.214738 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-870bbbde2a2f4c76bb558d8c43d7587b on tablet 870bbbde2a2f4c76bb558d8c43d7587b with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:42.226020 25742 ts_tablet_manager.cc:1428] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.005s
I20260924 14:28:42.235814 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-c2e4953bd67e40f5823697e67d029f52 received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:42.236320 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-c2e4953bd67e40f5823697e67d029f52 on tablet c2e4953bd67e40f5823697e67d029f52 with peer b8062fa80b5f415e991884a029861641
W20260924 14:28:42.244715 24343 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 91354.7us
I20260924 14:28:42.568442 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:42.793138 24445 raft_consensus.cc:1261] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:42.801421 25757 consensus_queue.cc:1035] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:28:42.910060 24978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 579485us
I20260924 14:28:42.913504 24447 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "095719203855447b9091cc7846d566de"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:42.914093 24447 raft_consensus.cc:595] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Received request to transfer leadership to TS cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:42.936545 24371 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bec046486fdc464290fafd6e4978632e"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:42.937175 24371 raft_consensus.cc:595] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Received request to transfer leadership to TS 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:42.943842 24302 raft_consensus.cc:1261] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:42.947999 24371 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:42.948585 24371 raft_consensus.cc:595] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Received request to transfer leadership to TS cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:42.951411 25660 raft_consensus.cc:979] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:42.951936 25779 raft_consensus.cc:1067] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Signalling peer cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:42.953516 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260924 14:28:42.956630 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:42.957257 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:42.967237 25660 consensus_queue.cc:1035] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:42.968024 25779 raft_consensus.cc:979] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:42.968560 25779 raft_consensus.cc:1067] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Signalling peer 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:42.970611 24448 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bec046486fdc464290fafd6e4978632e"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:48740
I20260924 14:28:42.971164 24448 raft_consensus.cc:482] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:42.973122 25009 raft_consensus.cc:1203] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:42.976181 25011 raft_consensus.cc:1203] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:42.971525 24448 raft_consensus.cc:3037] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:42.985618 24448 raft_consensus.cc:504] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:42.998602 25760 raft_consensus.cc:979] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:43.007949 25760 raft_consensus.cc:1067] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Signalling peer cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:43.009888 24297 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "095719203855447b9091cc7846d566de"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:49166
I20260924 14:28:43.010433 24297 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:43.010804 24297 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:43.000228 25012 raft_consensus.cc:1203] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:43.033030 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bec046486fdc464290fafd6e4978632e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:43.035725 24371 raft_consensus.cc:3032] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:43.048168 24371 raft_consensus.cc:729] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:43.033053 24302 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:49178
I20260924 14:28:43.019111 24297 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.049820 24302 raft_consensus.cc:482] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:43.035192 24448 leader_election.cc:288] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:43.050211 24302 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:43.052803 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:43.057986 24448 raft_consensus.cc:3032] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:28:43.067291 24448 raft_consensus.cc:729] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:43.053735 24371 consensus_queue.cc:260] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.063896 25008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:43.072777 25008 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:43.066694 24297 leader_election.cc:288] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 3 election: Requested vote from peers 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:43.074743 24371 raft_consensus.cc:3037] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:43.081122 25008 raft_consensus.cc:2454] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 3.
I20260924 14:28:43.085086 24269 leader_election.cc:302] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:43.086006 25772 raft_consensus.cc:2781] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:28:43.086494 25772 raft_consensus.cc:686] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 3 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:43.087320 25772 consensus_queue.cc:237] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.093161 25008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bec046486fdc464290fafd6e4978632e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:43.095711 25008 raft_consensus.cc:3037] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:43.083065 24448 consensus_queue.cc:260] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.059254 24302 raft_consensus.cc:504] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:43.114104 24448 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:43.115674 24302 leader_election.cc:288] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 3 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:43.104954 25008 raft_consensus.cc:2454] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:43.116605 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:43.119570 24445 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:43.117240 25773 raft_consensus.cc:1050] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:43.126241 24371 raft_consensus.cc:2454] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:43.126602 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:43.127241 24376 raft_consensus.cc:3032] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:28:43.127605 24376 raft_consensus.cc:729] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:43.127705 24417 leader_election.cc:302] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:43.128623 25665 raft_consensus.cc:2781] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:43.128450 24376 consensus_queue.cc:260] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:43.129093 25665 raft_consensus.cc:686] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:43.127002 25773 consensus_queue.cc:237] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.129765 24376 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:43.130003 25665 consensus_queue.cc:237] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.137408 24448 raft_consensus.cc:2454] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 3.
I20260924 14:28:43.137408 24376 raft_consensus.cc:2454] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 3.
I20260924 14:28:43.140715 24445 raft_consensus.cc:2454] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 3.
I20260924 14:28:43.142643 24268 leader_election.cc:302] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:43.144902 25772 raft_consensus.cc:2781] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:28:43.145365 25772 raft_consensus.cc:686] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 3 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:43.146286 25772 consensus_queue.cc:237] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:43.155247 24198 catalog_manager.cc:5495] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 2 to 3, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:43.179227 24447 raft_consensus.cc:1261] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.179747 24302 raft_consensus.cc:1261] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.180899 25008 raft_consensus.cc:1261] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.186997 25757 consensus_queue.cc:1035] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:43.188014 25773 consensus_queue.cc:1035] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:43.188405 24197 catalog_manager.cc:5495] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 1 to 2, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:43.189914 25779 consensus_queue.cc:1035] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:43.205724 25767 raft_consensus.cc:1050] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:43.210747 25767 consensus_queue.cc:237] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.242935 24371 raft_consensus.cc:1261] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.243515 24445 raft_consensus.cc:1261] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.242782 25779 raft_consensus.cc:2932] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.245290 25772 consensus_queue.cc:1035] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:43.246555 25012 raft_consensus.cc:1261] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.248731 25766 consensus_queue.cc:1035] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:43.253913 24302 raft_consensus.cc:2932] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.253556 25771 consensus_queue.cc:1035] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260924 14:28:43.271945 24447 raft_consensus.cc:2932] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.285857 25760 raft_consensus.cc:1050] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:43.296582 24201 catalog_manager.cc:5495] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:43.309211 25012 raft_consensus.cc:2932] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.306099 25760 consensus_queue.cc:237] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.337838 25766 raft_consensus.cc:2932] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.339778 24452 raft_consensus.cc:2932] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.367789 25008 raft_consensus.cc:2932] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.377571 24376 raft_consensus.cc:2932] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.431602 25008 raft_consensus.cc:1261] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.435883 25761 consensus_queue.cc:1035] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:43.441314 24298 raft_consensus.cc:1261] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.441869 24376 raft_consensus.cc:1261] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.446056 25761 consensus_queue.cc:1035] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:43.456380 24201 catalog_manager.cc:5495] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.460032 25665 consensus_queue.cc:1035] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:43.464468 24376 consensus_queue.cc:237] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.480183 25012 raft_consensus.cc:1261] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:43.480837 24297 raft_consensus.cc:1261] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:43.482887 25770 consensus_queue.cc:1035] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:43.485172 25770 consensus_queue.cc:1035] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:43.500757 25773 raft_consensus.cc:2932] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.505005 25618 raft_consensus.cc:1050] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:43.507423 25618 consensus_queue.cc:237] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.512563 24201 catalog_manager.cc:5495] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 2 to 3, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:43.513460 25665 raft_consensus.cc:2932] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.516163 24452 raft_consensus.cc:1261] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.516919 24298 raft_consensus.cc:2932] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.518054 25618 consensus_queue.cc:1035] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:43.528826 25012 raft_consensus.cc:2932] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.536010 24297 raft_consensus.cc:2932] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.525624 25008 raft_consensus.cc:2932] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.541000 24373 raft_consensus.cc:1261] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.543219 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 3a57dc14600c450485f596239b30df0d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:43.542825 24197 catalog_manager.cc:5495] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:43.544559 25618 consensus_queue.cc:1035] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:43.547256 25006 raft_consensus.cc:1261] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:43.549824 25803 consensus_queue.cc:1035] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:43.570580 24452 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:43.573674 25654 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:28:43.577940 24199 catalog_manager.cc:5495] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.580255 24370 raft_consensus.cc:2932] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.596236 25803 raft_consensus.cc:2932] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.603941 25006 raft_consensus.cc:2932] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.611873 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a57dc14600c450485f596239b30df0d with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:43.615347 24452 raft_consensus.cc:1261] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:28:43.617682 24370 raft_consensus.cc:2932] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.620287 25766 consensus_queue.cc:1035] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:43.640672 24447 raft_consensus.cc:2932] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.655108 24297 consensus_queue.cc:237] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.657403 25814 tablet_replica.cc:330] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:43.650506 24197 catalog_manager.cc:5495] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:43.666318 24371 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:43.667372 25008 raft_consensus.cc:1261] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:43.668674 24371 raft_consensus.cc:1261] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:28:43.668799 25618 consensus_queue.cc:1035] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:28:43.670342 25665 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:43.676147 25803 consensus_queue.cc:1035] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:43.683437 25012 raft_consensus.cc:1261] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:43.685165 25803 consensus_queue.cc:1035] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:43.693110 24302 consensus_queue.cc:237] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.700886 25814 raft_consensus.cc:2229] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:43.701886 25814 raft_consensus.cc:2258] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:43.708987 24447 raft_consensus.cc:1261] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260924 14:28:43.717772 24324 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3be0 after lost signal to thread 23138
W20260924 14:28:43.719323 24324 debug-util.cc:398] Leaking SignalData structure 0x7b08000daea0 after lost signal to thread 24235
I20260924 14:28:43.719071 25654 consensus_queue.cc:1035] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260924 14:28:43.720952 24324 debug-util.cc:398] Leaking SignalData structure 0x7b080035c0a0 after lost signal to thread 24327
W20260924 14:28:43.722343 24324 debug-util.cc:398] Leaking SignalData structure 0x7b08000d10e0 after lost signal to thread 24402
I20260924 14:28:43.732089 25011 raft_consensus.cc:1261] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:43.734578 25814 ts_tablet_manager.cc:1905] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260924 14:28:43.734828 24324 debug-util.cc:398] Leaking SignalData structure 0x7b08002d1200 after lost signal to thread 24476
I20260924 14:28:43.737859 25011 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:43.740761 25665 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260924 14:28:43.744216 24324 debug-util.cc:398] Leaking SignalData structure 0x7b080034d980 after lost signal to thread 25037
I20260924 14:28:43.749158 24445 raft_consensus.cc:1261] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:43.751091 25618 consensus_queue.cc:1035] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:43.812731 24451 consensus_queue.cc:237] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:43.775254 25803 consensus_queue.cc:1035] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260924 14:28:43.820433 25654 raft_consensus.cc:2932] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.831123 24448 raft_consensus.cc:2932] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.808704 25618 raft_consensus.cc:2932] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.838747 25008 raft_consensus.cc:2932] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.893894 25012 raft_consensus.cc:1261] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:43.916813 25761 consensus_queue.cc:1035] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:43.927340 25814 ts_tablet_manager.cc:1918] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:43.927982 25814 log.cc:1196] T 3a57dc14600c450485f596239b30df0d P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/3a57dc14600c450485f596239b30df0d
W20260924 14:28:43.954299 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 58717.5us
W20260924 14:28:43.970153 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 68796.2us
I20260924 14:28:43.895260 25010 raft_consensus.cc:2932] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:43.995779 24198 catalog_manager.cc:5495] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260924 14:28:43.976384 24269 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 77869.9us
I20260924 14:28:44.000126 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 870bbbde2a2f4c76bb558d8c43d7587b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:44.000857 25761 raft_consensus.cc:2932] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:44.007364 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet 3a57dc14600c450485f596239b30df0d (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:44.008078 24371 raft_consensus.cc:1261] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:44.017042 25011 raft_consensus.cc:2932] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
W20260924 14:28:44.039115 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 58554.7us
I20260924 14:28:44.040131 25712 consensus_queue.cc:1035] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:44.040992 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet c2e4953bd67e40f5823697e67d029f52 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:44.059492 24200 catalog_manager.cc:5495] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
W20260924 14:28:44.062939 24269 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 59405.6us
I20260924 14:28:44.063814 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet cbae9b0be0e4421eab738bd8fd45a1e1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:44.069521 24448 raft_consensus.cc:2932] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:44.074163 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet cbae9b0be0e4421eab738bd8fd45a1e1 with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:44.075747 24197 catalog_manager.cc:5495] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:44.088115 25829 tablet_replica.cc:330] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:44.089775 25829 raft_consensus.cc:2229] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:44.097242 24371 raft_consensus.cc:2932] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
W20260924 14:28:44.099650 24343 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 53493us
I20260924 14:28:44.110247 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet 870bbbde2a2f4c76bb558d8c43d7587b with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:44.111176 25829 raft_consensus.cc:2258] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:44.119367 25829 ts_tablet_manager.cc:1905] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:44.126576 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e4953bd67e40f5823697e67d029f52 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:44.127210 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:44.159834 25829 ts_tablet_manager.cc:1918] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:44.160388 25829 log.cc:1196] T cbae9b0be0e4421eab738bd8fd45a1e1 P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/cbae9b0be0e4421eab738bd8fd45a1e1
I20260924 14:28:44.162214 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet cbae9b0be0e4421eab738bd8fd45a1e1 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:44.162305 25829 tablet_replica.cc:330] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:44.163700 25829 raft_consensus.cc:2229] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:44.165781 25829 raft_consensus.cc:2258] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:44.172719 25829 ts_tablet_manager.cc:1905] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:44.173908 25830 tablet_replica.cc:330] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:44.191381 25830 raft_consensus.cc:2229] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:44.192205 25830 raft_consensus.cc:2258] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:44.200534 25830 ts_tablet_manager.cc:1905] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:44.202384 25829 ts_tablet_manager.cc:1918] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:44.202844 25829 log.cc:1196] T 870bbbde2a2f4c76bb558d8c43d7587b P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/870bbbde2a2f4c76bb558d8c43d7587b
I20260924 14:28:44.205562 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet 870bbbde2a2f4c76bb558d8c43d7587b (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:44.207590 24298 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:44.208289 24298 raft_consensus.cc:595] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 LEADER]: Received request to transfer leadership to TS 8a60ac7d6a0c4795833b917ccdd0c7d1
I20260924 14:28:44.212183 24298 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "93b3f9be3c4846a8b75e5e42099e6dbc"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:44.212898 24298 raft_consensus.cc:595] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:44.215785 24448 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bec046486fdc464290fafd6e4978632e"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:44.216382 24448 raft_consensus.cc:595] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:44.218653 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260924 14:28:44.220379 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:44.220305 25830 ts_tablet_manager.cc:1918] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:44.221138 25830 log.cc:1196] T c2e4953bd67e40f5823697e67d029f52 P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/c2e4953bd67e40f5823697e67d029f52
I20260924 14:28:44.221238 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:44.223388 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet c2e4953bd67e40f5823697e67d029f52 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:44.341259 25824 raft_consensus.cc:979] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:44.341756 25618 raft_consensus.cc:1067] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:44.344583 25008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "93b3f9be3c4846a8b75e5e42099e6dbc"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45634
I20260924 14:28:44.345258 25008 raft_consensus.cc:482] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:44.345644 25008 raft_consensus.cc:3037] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:44.352818 25008 raft_consensus.cc:504] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:44.354779 25008 leader_election.cc:288] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:44.355966 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b3f9be3c4846a8b75e5e42099e6dbc" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:44.356303 24298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b3f9be3c4846a8b75e5e42099e6dbc" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:44.356623 24448 raft_consensus.cc:3037] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:44.357055 24298 raft_consensus.cc:3032] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:44.357427 24298 raft_consensus.cc:729] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:44.358188 24298 consensus_queue.cc:260] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:44.359544 24298 raft_consensus.cc:3037] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:44.363713 24448 raft_consensus.cc:2454] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:44.364234 24298 raft_consensus.cc:2454] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:44.365017 24978 leader_election.cc:302] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:44.366400 25783 raft_consensus.cc:2781] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:44.366869 25783 raft_consensus.cc:686] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:44.367712 25783 consensus_queue.cc:237] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:44.382458 24198 catalog_manager.cc:5495] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:44.478451 25761 raft_consensus.cc:979] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:44.479509 25712 raft_consensus.cc:1067] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:44.481029 25008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bec046486fdc464290fafd6e4978632e"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45616
I20260924 14:28:44.481475 25008 raft_consensus.cc:482] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:44.481835 25008 raft_consensus.cc:3037] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:44.489074 25008 raft_consensus.cc:504] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:44.491587 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bec046486fdc464290fafd6e4978632e" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:44.492162 24371 raft_consensus.cc:3037] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:44.499785 25008 leader_election.cc:288] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:44.501175 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bec046486fdc464290fafd6e4978632e" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:44.501781 24448 raft_consensus.cc:3032] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:28:44.502116 24448 raft_consensus.cc:729] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:44.502839 24448 consensus_queue.cc:260] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:44.504232 24448 raft_consensus.cc:3037] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:44.506996 24371 raft_consensus.cc:2454] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 3.
I20260924 14:28:44.508080 24978 leader_election.cc:302] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:44.508885 25783 raft_consensus.cc:2781] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:28:44.509321 25783 raft_consensus.cc:686] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 3 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:44.510710 25783 consensus_queue.cc:237] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:44.524017 24198 catalog_manager.cc:5495] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 2 to 3, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 3 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:44.534207 24448 raft_consensus.cc:2454] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 3.
I20260924 14:28:44.659305 25824 raft_consensus.cc:979] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:44.659940 25618 raft_consensus.cc:1067] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 LEADER]: Signalling peer 8a60ac7d6a0c4795833b917ccdd0c7d1 to start an election
I20260924 14:28:44.662850 24448 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
 from {username='slave'} at 127.0.0.1:48734
I20260924 14:28:44.663546 24448 raft_consensus.cc:482] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:44.664003 24448 raft_consensus.cc:3037] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:44.671572 24448 raft_consensus.cc:504] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:44.673230 24448 leader_election.cc:288] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 3 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:44.674058 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:44.674649 24297 raft_consensus.cc:3032] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:28:44.675011 24297 raft_consensus.cc:729] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:44.674516 25008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "568b498fb63e4638b92d175b8d122c0e" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:44.675524 25008 raft_consensus.cc:3037] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:44.675663 24297 consensus_queue.cc:260] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:44.676827 24297 raft_consensus.cc:3037] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:44.684098 25008 raft_consensus.cc:2454] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 3.
I20260924 14:28:44.684801 24297 raft_consensus.cc:2454] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 3.
I20260924 14:28:44.685334 24418 leader_election.cc:302] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:44.686231 25761 raft_consensus.cc:2781] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:28:44.686658 25761 raft_consensus.cc:686] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:44.687383 25761 consensus_queue.cc:237] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:44.698292 24198 catalog_manager.cc:5495] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 2 to 3, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:44.786749 24448 raft_consensus.cc:1261] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:44.788354 25655 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:44.829674 24297 raft_consensus.cc:1261] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:44.835695 25782 consensus_queue.cc:1035] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:44.915685 24448 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:44.916965 25783 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:28:44.927093 24371 raft_consensus.cc:1261] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:44.928515 25655 consensus_queue.cc:1035] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:28:45.085700 24298 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:45.086997 25712 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:28:45.099824 25011 raft_consensus.cc:1261] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:45.109398 25712 consensus_queue.cc:1035] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:28:45.216459 24448 consensus_queue.cc:237] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:45.223358 24418 consensus_peers.cc:487] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6fb42513147f483f912eaec2c1151c50. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:45.223485 24370 raft_consensus.cc:1261] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:45.224944 24298 raft_consensus.cc:1261] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:45.224812 25842 consensus_queue.cc:1035] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:45.226126 25712 consensus_queue.cc:1035] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:45.234436 25842 raft_consensus.cc:2932] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:45.238456 24370 raft_consensus.cc:2932] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:45.247568 24298 raft_consensus.cc:2932] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:45.266757 24200 catalog_manager.cc:5495] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:45.274562 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:45.331409 24448 consensus_queue.cc:237] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:45.424496 24418 consensus_peers.cc:487] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e48f04a45264e7d8be03ae56ab10560. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:45.441569 24371 raft_consensus.cc:1261] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:45.443200 25847 consensus_queue.cc:1035] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:45.451262 24298 raft_consensus.cc:1261] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:45.453493 25847 consensus_queue.cc:1035] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:45.470968 25842 raft_consensus.cc:2932] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:45.473335 24370 raft_consensus.cc:2932] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:45.496529 24298 raft_consensus.cc:2932] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:45.502946 24452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f"
dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
mode: GRACEFUL
new_leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:51980
I20260924 14:28:45.503720 24452 raft_consensus.cc:595] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 LEADER]: Received request to transfer leadership to TS cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:45.506330 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260924 14:28:45.509614 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:45.510387 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:45.523591 24200 catalog_manager.cc:5495] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:45.528918 24371 consensus_queue.cc:237] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:45.554639 24344 consensus_peers.cc:487] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a06a0e1988d4a53b9590e40ca1481b0. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:45.559571 24451 raft_consensus.cc:1261] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 14:28:45.571800 25840 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:45.592535 24298 raft_consensus.cc:1261] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 14:28:45.595378 25840 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:45.604146 25827 raft_consensus.cc:2932] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:45.614614 24445 raft_consensus.cc:2932] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:45.685688 25857 ts_tablet_manager.cc:927] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:45.701884 24297 consensus_queue.cc:237] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:45.708559 24298 raft_consensus.cc:2932] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:45.713574 24200 catalog_manager.cc:5495] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:45.775695 25857 tablet_copy_client.cc:316] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
W20260924 14:28:45.883685 24344 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 164114us
I20260924 14:28:45.920456 25712 raft_consensus.cc:979] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1: : Instructing follower cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:45.920971 25712 raft_consensus.cc:1067] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 LEADER]: Signalling peer cafc2c5ad4df4224a5ccc4893de70144 to start an election
I20260924 14:28:45.934455 24297 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
 from {username='slave'} at 127.0.0.1:49166
I20260924 14:28:45.935241 24297 raft_consensus.cc:482] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:45.936928 24297 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:28:45.950244 24445 raft_consensus.cc:1261] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260924 14:28:45.951437 24269 consensus_peers.cc:487] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a2f5e6c143534a47b388c7cd35bd1f4f. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:45.956851 25618 consensus_queue.cc:1035] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260924 14:28:45.960510 24371 raft_consensus.cc:1261] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:45.985924 24297 raft_consensus.cc:504] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:45.988365 24297 leader_election.cc:288] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 5 election: Requested vote from peers 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:45.994863 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet 6fb42513147f483f912eaec2c1151c50 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:45.995481 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet 6fb42513147f483f912eaec2c1151c50 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-6fb42513147f483f912eaec2c1151c50
I20260924 14:28:46.004652 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:46.005368 24448 raft_consensus.cc:3032] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:28:46.005664 24448 raft_consensus.cc:729] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:46.006409 24448 consensus_queue.cc:260] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:46.007694 24448 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:28:46.012794 24462 tablet_copy_source_session.cc:205] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:46.022704 24371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "cafc2c5ad4df4224a5ccc4893de70144" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:46.023530 24371 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:28:46.025209 25618 consensus_queue.cc:1035] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:46.027709 25848 raft_consensus.cc:2932] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:46.034437 24448 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 5.
I20260924 14:28:46.036069 24371 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafc2c5ad4df4224a5ccc4893de70144 in term 5.
I20260924 14:28:46.048492 25868 ts_tablet_manager.cc:927] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:46.055795 24268 leader_election.cc:302] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:46.058102 25618 raft_consensus.cc:2781] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:28:46.058601 25618 raft_consensus.cc:686] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 5 LEADER]: Becoming Leader. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:46.059355 25618 consensus_queue.cc:237] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:46.063632 25857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fb42513147f483f912eaec2c1151c50. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:46.064405 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 53637.9us
I20260924 14:28:46.071192 24200 catalog_manager.cc:5495] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260924 14:28:46.111799 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 54428.9us
I20260924 14:28:46.117044 24448 raft_consensus.cc:2932] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:46.141254 24371 raft_consensus.cc:2932] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:46.138765 24200 catalog_manager.cc:5495] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: term changed from 4 to 5, leader changed from 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) to cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:46.166535 25857 tablet_copy_client.cc:791] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:46.167339 25857 tablet_copy_client.cc:655] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:46.175267 25876 ts_tablet_manager.cc:927] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:46.184624 25857 tablet_copy_client.cc:531] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:46.188871 25876 tablet_copy_client.cc:316] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:46.190615 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet 3a06a0e1988d4a53b9590e40ca1481b0 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:46.191171 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet 3a06a0e1988d4a53b9590e40ca1481b0 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-3a06a0e1988d4a53b9590e40ca1481b0
I20260924 14:28:46.214457 24387 tablet_copy_source_session.cc:205] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:46.216557 25857 tablet_bootstrap.cc:492] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:46.217919 25876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a06a0e1988d4a53b9590e40ca1481b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:46.227777 25868 tablet_copy_client.cc:316] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:46.230466 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet 5e48f04a45264e7d8be03ae56ab10560 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:46.231046 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet 5e48f04a45264e7d8be03ae56ab10560 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-5e48f04a45264e7d8be03ae56ab10560
I20260924 14:28:46.241914 25876 tablet_copy_client.cc:791] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:46.242573 25876 tablet_copy_client.cc:655] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:46.246438 24462 tablet_copy_source_session.cc:205] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:46.246454 25876 tablet_copy_client.cc:531] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:46.249986 25868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e48f04a45264e7d8be03ae56ab10560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:46.256872 25876 tablet_bootstrap.cc:492] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:46.276808 25857 tablet_bootstrap.cc:492] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:46.277879 25857 tablet_bootstrap.cc:492] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:46.278748 25857 ts_tablet_manager.cc:1397] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.079s	user 0.035s	sys 0.001s
I20260924 14:28:46.282104 25868 tablet_copy_client.cc:791] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:46.287288 25868 tablet_copy_client.cc:655] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:46.288802 25857 raft_consensus.cc:348] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:46.290169 25857 raft_consensus.cc:729] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:46.290923 25857 consensus_queue.cc:260] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:46.292591 25876 tablet_bootstrap.cc:492] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:46.294898 25857 ts_tablet_manager.cc:1428] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.001s
I20260924 14:28:46.303366 25868 tablet_copy_client.cc:531] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:46.304724 25876 tablet_bootstrap.cc:492] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:46.305509 25876 ts_tablet_manager.cc:1397] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.008s
I20260924 14:28:46.309811 25880 ts_tablet_manager.cc:927] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:46.311596 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-6fb42513147f483f912eaec2c1151c50 received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:46.311975 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-6fb42513147f483f912eaec2c1151c50 on tablet 6fb42513147f483f912eaec2c1151c50 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:46.318135 25868 tablet_bootstrap.cc:492] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:46.325086 25880 tablet_copy_client.cc:316] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:46.327785 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet a2f5e6c143534a47b388c7cd35bd1f4f from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:46.332799 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet a2f5e6c143534a47b388c7cd35bd1f4f from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-a2f5e6c143534a47b388c7cd35bd1f4f
I20260924 14:28:46.341670 24313 tablet_copy_source_session.cc:205] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:46.308620 25876 raft_consensus.cc:348] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:46.344722 25876 raft_consensus.cc:729] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:46.348573 25880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2f5e6c143534a47b388c7cd35bd1f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:46.345448 25876 consensus_queue.cc:260] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:46.361528 25876 ts_tablet_manager.cc:1428] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260924 14:28:46.367223 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-3a06a0e1988d4a53b9590e40ca1481b0 received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:46.367727 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-3a06a0e1988d4a53b9590e40ca1481b0 on tablet 3a06a0e1988d4a53b9590e40ca1481b0 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:46.378558 25880 tablet_copy_client.cc:791] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:46.379235 25880 tablet_copy_client.cc:655] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:46.379655 25868 tablet_bootstrap.cc:492] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:46.380788 25868 tablet_bootstrap.cc:492] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:46.381729 25868 ts_tablet_manager.cc:1397] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.064s	user 0.024s	sys 0.015s
I20260924 14:28:46.384300 25880 tablet_copy_client.cc:531] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:46.385084 25868 raft_consensus.cc:348] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:46.386304 25868 raft_consensus.cc:729] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:46.387095 25868 consensus_queue.cc:260] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:46.390304 25868 ts_tablet_manager.cc:1428] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260924 14:28:46.394312 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-5e48f04a45264e7d8be03ae56ab10560 received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:46.394731 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-5e48f04a45264e7d8be03ae56ab10560 on tablet 5e48f04a45264e7d8be03ae56ab10560 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:46.424124 25880 tablet_bootstrap.cc:492] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:46.503295 25012 raft_consensus.cc:1203] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260924 14:28:46.551256 24343 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 79027.4us
I20260924 14:28:46.566174 25008 raft_consensus.cc:1203] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260924 14:28:46.584623 24373 raft_consensus.cc:1261] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 5 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:28:46.595643 25871 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260924 14:28:46.591058 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:46.626525 24445 raft_consensus.cc:1261] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 5 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:28:46.630590 25618 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:28:46.698455 25012 raft_consensus.cc:1203] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:46.712854 25880 tablet_bootstrap.cc:492] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:46.714145 25880 tablet_bootstrap.cc:492] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:46.714952 25880 ts_tablet_manager.cc:1397] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.291s	user 0.057s	sys 0.110s
W20260924 14:28:46.717151 24268 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 48406.5us
I20260924 14:28:46.721633 25880 raft_consensus.cc:348] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:46.722847 25880 raft_consensus.cc:729] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:46.723632 25880 consensus_queue.cc:260] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:46.730808 25880 ts_tablet_manager.cc:1428] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260924 14:28:46.755563 24313 tablet_copy_service.cc:342] P cafc2c5ad4df4224a5ccc4893de70144: Request end of tablet copy session b8062fa80b5f415e991884a029861641-a2f5e6c143534a47b388c7cd35bd1f4f received from {username='slave'} at 127.0.0.1:48368
I20260924 14:28:46.755992 24313 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-a2f5e6c143534a47b388c7cd35bd1f4f on tablet a2f5e6c143534a47b388c7cd35bd1f4f with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:46.779867 24297 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b9e83b406c1944efb04163f032a3b4f6"
dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
mode: GRACEFUL
new_leader_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:59652
I20260924 14:28:46.780661 24297 raft_consensus.cc:595] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Received request to transfer leadership to TS 61d316eb5dfe405c95019383bbebecaa
I20260924 14:28:46.789686 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260924 14:28:46.791404 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:46.792083 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:46.854533 25872 raft_consensus.cc:979] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144: : Instructing follower 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:46.855261 25618 raft_consensus.cc:1067] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Signalling peer 61d316eb5dfe405c95019383bbebecaa to start an election
I20260924 14:28:46.857432 24371 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9e83b406c1944efb04163f032a3b4f6"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
 from {username='slave'} at 127.0.0.1:49710
I20260924 14:28:46.858163 24371 raft_consensus.cc:482] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:46.858608 24371 raft_consensus.cc:3037] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:46.872301 24371 raft_consensus.cc:504] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:46.875007 24371 leader_election.cc:288] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:46.875187 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e83b406c1944efb04163f032a3b4f6" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:46.876006 24445 raft_consensus.cc:3037] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:46.878296 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e83b406c1944efb04163f032a3b4f6" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:46.880209 24297 raft_consensus.cc:3032] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:46.880584 24297 raft_consensus.cc:729] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cafc2c5ad4df4224a5ccc4893de70144, State: Running, Role: LEADER
I20260924 14:28:46.881434 24297 consensus_queue.cc:260] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:46.882758 24297 raft_consensus.cc:3037] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:46.882939 24445 raft_consensus.cc:2454] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:46.884091 24343 leader_election.cc:302] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:46.884955 25779 raft_consensus.cc:2781] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:46.885427 25779 raft_consensus.cc:686] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:46.886248 25779 consensus_queue.cc:237] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:46.897984 24198 catalog_manager.cc:5495] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:46.907819 24297 raft_consensus.cc:2454] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:46.935982 25008 raft_consensus.cc:1203] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:46.978137 25885 raft_consensus.cc:1050] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:46.980749 25885 consensus_queue.cc:237] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:46.998526 25008 raft_consensus.cc:1261] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:46.999874 24373 raft_consensus.cc:1261] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:47.001382 25885 consensus_queue.cc:1035] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:47.002090 24297 raft_consensus.cc:1261] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:47.004786 25712 consensus_queue.cc:1035] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:47.006510 25712 consensus_queue.cc:1035] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:47.024685 25885 raft_consensus.cc:2932] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.039645 25008 raft_consensus.cc:2932] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.054013 24297 raft_consensus.cc:2932] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.060384 24199 catalog_manager.cc:5495] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.064271 24373 raft_consensus.cc:2932] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.082938 25859 raft_consensus.cc:1050] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:47.086103 25859 consensus_queue.cc:237] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:47.091585 25712 raft_consensus.cc:1050] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:47.094084 25712 consensus_queue.cc:237] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:47.101361 24448 consensus_queue.cc:237] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:47.110766 25012 raft_consensus.cc:1261] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:47.119431 24297 raft_consensus.cc:1261] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 14:28:47.118417 24452 raft_consensus.cc:1261] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 14:28:47.112251 25884 consensus_queue.cc:1035] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260924 14:28:47.124223 24371 raft_consensus.cc:1261] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:47.125468 25866 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260924 14:28:47.133070 25859 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:47.118321 25008 raft_consensus.cc:1261] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 14:28:47.137353 25866 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:47.139488 25008 raft_consensus.cc:1261] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:47.142423 24302 raft_consensus.cc:1261] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:47.144414 25761 consensus_queue.cc:1035] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:47.145496 25869 consensus_queue.cc:1035] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:47.146729 24376 raft_consensus.cc:1261] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:47.148636 25869 consensus_queue.cc:1035] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:47.175379 25779 raft_consensus.cc:2932] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.193120 25869 consensus_queue.cc:1035] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:47.197640 24370 raft_consensus.cc:2932] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.200631 25884 raft_consensus.cc:2932] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.207662 24299 raft_consensus.cc:2932] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.211774 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 5e48f04a45264e7d8be03ae56ab10560 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:47.221680 25011 raft_consensus.cc:2932] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.226464 24199 catalog_manager.cc:5495] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 3 to 4, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:47.236121 24451 raft_consensus.cc:2932] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.247946 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e48f04a45264e7d8be03ae56ab10560 with delete_type TABLET_DATA_TOMBSTONED (TS cafc2c5ad4df4224a5ccc4893de70144 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49158
I20260924 14:28:47.253360 25912 tablet_replica.cc:330] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:47.267379 25912 raft_consensus.cc:2229] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:47.270730 25912 raft_consensus.cc:2258] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:47.262697 24373 raft_consensus.cc:2932] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.271006 24302 raft_consensus.cc:2932] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.279956 25912 ts_tablet_manager.cc:1905] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:47.281757 25884 raft_consensus.cc:2932] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
W20260924 14:28:47.296221 24344 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 72805.9us
I20260924 14:28:47.302290 25912 ts_tablet_manager.cc:1918] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:47.302946 25912 log.cc:1196] T 5e48f04a45264e7d8be03ae56ab10560 P cafc2c5ad4df4224a5ccc4893de70144: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/wals/5e48f04a45264e7d8be03ae56ab10560
I20260924 14:28:47.313602 24201 catalog_manager.cc:5495] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.316319 24188 catalog_manager.cc:4841] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): tablet 5e48f04a45264e7d8be03ae56ab10560 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:47.321516 25011 raft_consensus.cc:2932] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.349736 24452 raft_consensus.cc:1261] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:47.350845 25008 raft_consensus.cc:2932] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.355571 25859 consensus_queue.cc:1035] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:47.362602 25872 raft_consensus.cc:1050] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:47.367630 25872 consensus_queue.cc:237] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:47.388921 24198 catalog_manager.cc:5495] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.404507 24299 raft_consensus.cc:1261] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:47.434332 25011 raft_consensus.cc:1261] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:47.435974 25893 consensus_queue.cc:1035] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:47.436817 25910 consensus_queue.cc:1035] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:47.438556 24451 raft_consensus.cc:1261] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:47.440248 25618 consensus_queue.cc:1035] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:47.460950 24452 consensus_queue.cc:237] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:47.463029 24371 raft_consensus.cc:1261] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:47.499760 25012 raft_consensus.cc:1261] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:47.501147 25872 consensus_queue.cc:1035] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:47.501493 25884 consensus_queue.cc:1035] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:47.509680 25842 raft_consensus.cc:2932] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.521406 25011 raft_consensus.cc:2932] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.528316 24376 consensus_queue.cc:237] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:47.535475 24299 raft_consensus.cc:1261] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:47.537218 25761 consensus_queue.cc:1035] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:47.547362 24302 raft_consensus.cc:1261] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:47.549106 25012 raft_consensus.cc:1261] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:47.558079 25872 raft_consensus.cc:2932] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.549846 24200 catalog_manager.cc:5495] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New cstate: current_term: 1 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.560300 24299 raft_consensus.cc:2932] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.556859 25910 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260924 14:28:47.568535 24451 raft_consensus.cc:2932] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.572719 25866 consensus_queue.cc:1035] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:47.590981 25008 raft_consensus.cc:2932] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.600832 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 6fb42513147f483f912eaec2c1151c50 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:47.603035 25866 raft_consensus.cc:2932] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.605942 24360 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fb42513147f483f912eaec2c1151c50 with delete_type TABLET_DATA_TOMBSTONED (TS 61d316eb5dfe405c95019383bbebecaa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49698
I20260924 14:28:47.608621 25011 raft_consensus.cc:2932] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.625897 24302 raft_consensus.cc:2932] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.629917 24200 catalog_manager.cc:5495] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 4 to 5, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.631819 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 3a06a0e1988d4a53b9590e40ca1481b0 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:47.638037 25924 tablet_replica.cc:330] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:47.643610 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a06a0e1988d4a53b9590e40ca1481b0 with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:47.670269 24200 catalog_manager.cc:5495] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.691308 25924 raft_consensus.cc:2229] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:47.692153 25924 raft_consensus.cc:2258] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:47.695410 25924 ts_tablet_manager.cc:1905] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:47.702342 25925 tablet_replica.cc:330] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:47.706807 25925 raft_consensus.cc:2229] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:47.707684 25925 raft_consensus.cc:2258] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:47.711747 25925 ts_tablet_manager.cc:1905] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:47.722352 24376 raft_consensus.cc:2932] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.747501 25925 ts_tablet_manager.cc:1918] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260924 14:28:47.748340 25925 log.cc:1196] T 3a06a0e1988d4a53b9590e40ca1481b0 P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/3a06a0e1988d4a53b9590e40ca1481b0
I20260924 14:28:47.752923 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet 3a06a0e1988d4a53b9590e40ca1481b0 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:47.761847 25924 ts_tablet_manager.cc:1918] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:47.766311 25924 log.cc:1196] T 6fb42513147f483f912eaec2c1151c50 P 61d316eb5dfe405c95019383bbebecaa: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/wals/6fb42513147f483f912eaec2c1151c50
I20260924 14:28:47.781610 24297 consensus_queue.cc:237] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:47.790246 24187 catalog_manager.cc:4841] TS 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): tablet 6fb42513147f483f912eaec2c1151c50 (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:47.791814 24371 raft_consensus.cc:1261] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:47.794458 25872 consensus_queue.cc:1035] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:47.797221 25008 raft_consensus.cc:1261] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260924 14:28:47.798583 25893 consensus_queue.cc:1035] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:28:47.807735 25893 raft_consensus.cc:2932] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.808115 25012 raft_consensus.cc:2932] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.812707 24373 raft_consensus.cc:2932] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.825985 24188 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet a2f5e6c143534a47b388c7cd35bd1f4f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:47.837380 24198 catalog_manager.cc:5495] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 3 to 4, VOTER 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195) evicted. New cstate: current_term: 1 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:47.850381 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:47.864578 24434 tablet_service.cc:1514] Processing DeleteTablet for tablet a2f5e6c143534a47b388c7cd35bd1f4f with delete_type TABLET_DATA_TOMBSTONED (TS 8a60ac7d6a0c4795833b917ccdd0c7d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48732
I20260924 14:28:47.865782 25925 tablet_replica.cc:330] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:47.866590 25925 raft_consensus.cc:2229] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:47.867250 25925 raft_consensus.cc:2258] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:47.873498 25925 ts_tablet_manager.cc:1905] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:47.895509 25925 ts_tablet_manager.cc:1918] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260924 14:28:47.896292 25925 log.cc:1196] T a2f5e6c143534a47b388c7cd35bd1f4f P 8a60ac7d6a0c4795833b917ccdd0c7d1: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/wals/a2f5e6c143534a47b388c7cd35bd1f4f
I20260924 14:28:47.897926 24187 catalog_manager.cc:4841] TS 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): tablet a2f5e6c143534a47b388c7cd35bd1f4f (table test-workload [id=2e82edb0c1f941d183d7d443712dc5f0]) successfully deleted
I20260924 14:28:47.911612 24373 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d7a8256035984c7fb87f7e9212b21dc7"
dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
mode: GRACEFUL
new_leader_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:46776
I20260924 14:28:47.912314 24373 raft_consensus.cc:595] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Received request to transfer leadership to TS b8062fa80b5f415e991884a029861641
I20260924 14:28:47.913435 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260924 14:28:47.915164 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:47.915748 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:48.094414 25866 raft_consensus.cc:979] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa: : Instructing follower b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:48.095018 25910 raft_consensus.cc:1067] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Signalling peer b8062fa80b5f415e991884a029861641 to start an election
I20260924 14:28:48.096448 25008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7a8256035984c7fb87f7e9212b21dc7"
dest_uuid: "b8062fa80b5f415e991884a029861641"
 from {username='slave'} at 127.0.0.1:45626
I20260924 14:28:48.096930 25008 raft_consensus.cc:482] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:28:48.097209 25008 raft_consensus.cc:3037] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:48.101986 25008 raft_consensus.cc:504] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:48.103500 25008 leader_election.cc:288] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:48.104413 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a8256035984c7fb87f7e9212b21dc7" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cafc2c5ad4df4224a5ccc4893de70144"
I20260924 14:28:48.105096 24297 raft_consensus.cc:3037] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:48.104543 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a8256035984c7fb87f7e9212b21dc7" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:48.110509 24376 raft_consensus.cc:3032] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:28:48.110900 24376 raft_consensus.cc:729] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:48.114459 24297 raft_consensus.cc:2454] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:48.116688 24979 leader_election.cc:302] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144; no voters: 
I20260924 14:28:48.117770 25863 raft_consensus.cc:2781] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:48.112401 24376 consensus_queue.cc:260] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:48.121062 24376 raft_consensus.cc:3037] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:48.124307 25863 raft_consensus.cc:686] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:48.125294 25863 consensus_queue.cc:237] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:48.129513 24376 raft_consensus.cc:2454] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:48.140095 24200 catalog_manager.cc:5495] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from 61d316eb5dfe405c95019383bbebecaa (127.22.151.194) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:48.541769 24373 raft_consensus.cc:1261] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:48.544077 25863 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:48.564167 24302 raft_consensus.cc:1261] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:48.566529 25863 consensus_queue.cc:1035] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:48.918941 24297 consensus_queue.cc:237] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:48.929687 24451 raft_consensus.cc:1261] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260924 14:28:48.931777 25872 consensus_queue.cc:1035] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260924 14:28:48.932775 24269 consensus_peers.cc:487] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9065d0aec064841b81b70e65f668aac. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:48.932941 24376 raft_consensus.cc:1261] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Refusing update from remote peer cafc2c5ad4df4224a5ccc4893de70144: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260924 14:28:48.935086 25934 consensus_queue.cc:1035] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:48.944027 25872 raft_consensus.cc:2932] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:48.949163 24451 raft_consensus.cc:2932] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:48.951675 24376 raft_consensus.cc:2932] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:48.954995 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:48.971647 24373 consensus_queue.cc:237] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:48.988111 24200 catalog_manager.cc:5495] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 reported cstate change: config changed from index -1 to 4, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 3 leader_uuid: "cafc2c5ad4df4224a5ccc4893de70144" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260924 14:28:49.002681 24344 consensus_peers.cc:487] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e4aee388a7474ac1bb46b761831b809e. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.048483 24297 raft_consensus.cc:1261] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260924 14:28:49.058439 24343 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:35781, user_credentials={real_user=slave}} blocked reactor thread for 43522.8us
I20260924 14:28:49.060595 24451 raft_consensus.cc:1261] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 14:28:49.064224 25939 consensus_queue.cc:1035] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:49.084821 25866 consensus_queue.cc:1035] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:49.101670 25866 raft_consensus.cc:2932] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:49.108688 24302 raft_consensus.cc:2932] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:49.113786 24451 raft_consensus.cc:2932] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:49.124918 24373 consensus_queue.cc:237] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:49.142755 24451 raft_consensus.cc:1261] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:49.144197 25866 consensus_queue.cc:1035] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:49.147202 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:28:49.149190 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:28:49.149945 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:49.158807 25866 raft_consensus.cc:2932] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:49.166664 24297 raft_consensus.cc:1261] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 14:28:49.168098 24445 raft_consensus.cc:2932] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:49.175200 24344 consensus_peers.cc:487] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5b55145a82545d9b769ea9d04bf1d99. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.178642 25939 consensus_queue.cc:1035] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:28:49.257534 24297 raft_consensus.cc:2932] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:49.285907 24445 consensus_queue.cc:237] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:49.312196 24297 raft_consensus.cc:1261] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 14:28:49.313632 24373 raft_consensus.cc:1261] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 14:28:49.317297 25884 consensus_queue.cc:1035] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:49.319206 25884 consensus_queue.cc:1035] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:28:49.340596 25842 raft_consensus.cc:2932] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:49.351269 24418 consensus_peers.cc:487] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b62b316cc05443990141d24402b1f4a. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.357028 24200 catalog_manager.cc:5495] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:49.362897 24200 catalog_manager.cc:5495] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index -1 to 2, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:49.366801 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:28:49.381763 24302 raft_consensus.cc:2932] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
W20260924 14:28:49.405472 24417 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 53732.9us
I20260924 14:28:49.409535 24198 catalog_manager.cc:5495] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:49.414007 25957 ts_tablet_manager.cc:927] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507)
I20260924 14:28:49.418303 24373 raft_consensus.cc:2932] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b8062fa80b5f415e991884a029861641 (127.22.151.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } } }
I20260924 14:28:49.434616 25959 ts_tablet_manager.cc:927] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:49.435495 25957 tablet_copy_client.cc:316] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.193:44507
I20260924 14:28:49.437007 24313 tablet_copy_service.cc:140] P cafc2c5ad4df4224a5ccc4893de70144: Received BeginTabletCopySession request for tablet c9065d0aec064841b81b70e65f668aac from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:48368)
I20260924 14:28:49.437556 24313 tablet_copy_service.cc:161] P cafc2c5ad4df4224a5ccc4893de70144: Beginning new tablet copy session on tablet c9065d0aec064841b81b70e65f668aac from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:48368: session id = b8062fa80b5f415e991884a029861641-c9065d0aec064841b81b70e65f668aac
I20260924 14:28:49.451231 25959 tablet_copy_client.cc:316] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:49.453112 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet e4aee388a7474ac1bb46b761831b809e from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:49.453670 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet e4aee388a7474ac1bb46b761831b809e from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-e4aee388a7474ac1bb46b761831b809e
I20260924 14:28:49.482836 23135 tablet_server.cc:178] TabletServer@127.22.151.193:0 shutting down...
W20260924 14:28:49.489046 24418 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
I20260924 14:28:49.492770 24387 tablet_copy_source_session.cc:205] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:49.498143 24313 tablet_copy_source_session.cc:205] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:49.506490 25957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9065d0aec064841b81b70e65f668aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:49.507488 25959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4aee388a7474ac1bb46b761831b809e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:49.512081 24248 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:28:49.512514 24248 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:28:49.530368 23135 tablet_copy_service.cc:362] P cafc2c5ad4df4224a5ccc4893de70144: Destroying tablet copy session b8062fa80b5f415e991884a029861641-c9065d0aec064841b81b70e65f668aac due to service shutdown
I20260924 14:28:49.530790 23135 tablet_copy_service.cc:433] P cafc2c5ad4df4224a5ccc4893de70144: ending tablet copy session b8062fa80b5f415e991884a029861641-c9065d0aec064841b81b70e65f668aac on tablet c9065d0aec064841b81b70e65f668aac with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:49.531065 25957 tablet_copy_client.cc:791] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:49.531653 25957 tablet_copy_client.cc:655] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
W20260924 14:28:49.535576 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 46565.5us
I20260924 14:28:49.540881 25963 ts_tablet_manager.cc:927] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
W20260924 14:28:49.548846 24418 consensus_peers.cc:487] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:49.601732 24418 consensus_peers.cc:487] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:49.602329 24418 consensus_peers.cc:487] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:49.603715 24418 consensus_peers.cc:487] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:49.609942 24979 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 61064.3us
I20260924 14:28:49.616546 25959 tablet_copy_client.cc:791] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:49.619745 25959 tablet_copy_client.cc:655] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
W20260924 14:28:49.621858 24979 consensus_peers.cc:487] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.618003 25963 tablet_copy_client.cc:316] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.194:42533
I20260924 14:28:49.630422 25959 tablet_copy_client.cc:531] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:49.637625 24387 tablet_copy_service.cc:140] P 61d316eb5dfe405c95019383bbebecaa: Received BeginTabletCopySession request for tablet d5b55145a82545d9b769ea9d04bf1d99 from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57284)
I20260924 14:28:49.638278 24387 tablet_copy_service.cc:161] P 61d316eb5dfe405c95019383bbebecaa: Beginning new tablet copy session on tablet d5b55145a82545d9b769ea9d04bf1d99 from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57284: session id = b8062fa80b5f415e991884a029861641-d5b55145a82545d9b769ea9d04bf1d99
W20260924 14:28:49.638584 24979 consensus_peers.cc:487] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:49.656881 24344 consensus_peers.cc:487] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.655237 24387 tablet_copy_source_session.cc:205] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:49.664197 25963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5b55145a82545d9b769ea9d04bf1d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:49.667784 25959 tablet_bootstrap.cc:492] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: Bootstrap starting.
W20260924 14:28:49.684420 24344 consensus_peers.cc:487] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.687624 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:28:49.700703 23135 tablet_replica.cc:330] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.701720 23135 raft_consensus.cc:2229] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:49.702755 23135 raft_consensus.cc:2258] T e838ce343bad4d42aa930f68fe4a4b2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:49.705173 24418 consensus_peers.cc:487] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.705633 23135 tablet_replica.cc:330] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.708142 23135 raft_consensus.cc:2229] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.709787 23135 raft_consensus.cc:2258] T b9e83b406c1944efb04163f032a3b4f6 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:49.709834 24344 consensus_peers.cc:487] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.712749 23135 tablet_replica.cc:330] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.713683 23135 raft_consensus.cc:2229] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
W20260924 14:28:49.714380 24344 consensus_peers.cc:487] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.714679 23135 raft_consensus.cc:2258] T f3667aba281046d3b974da9065022529 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.717428 23135 tablet_replica.cc:330] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.724959 25963 tablet_copy_client.cc:791] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:49.725768 25963 tablet_copy_client.cc:655] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:49.728173 23135 raft_consensus.cc:2229] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.728955 23135 raft_consensus.cc:2258] T 234eb8cdbf4c4e7599789194a0032c00 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.730551 25963 tablet_copy_client.cc:531] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:49.731642 23135 tablet_replica.cc:330] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.732936 25959 tablet_bootstrap.cc:492] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:49.735850 25959 tablet_bootstrap.cc:492] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:49.736562 25959 ts_tablet_manager.cc:1397] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.069s	user 0.025s	sys 0.002s
I20260924 14:28:49.738175 23135 raft_consensus.cc:2229] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260924 14:28:49.740177 24344 consensus_peers.cc:487] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.741441 23135 raft_consensus.cc:2258] T 3822638d266742c4908907a61efe8231 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.744316 23135 tablet_replica.cc:330] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.745128 25963 tablet_bootstrap.cc:492] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:49.746763 25959 raft_consensus.cc:348] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:49.747831 25959 raft_consensus.cc:729] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:49.748603 25959 consensus_queue.cc:260] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:49.745162 23135 raft_consensus.cc:2229] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.750874 23135 raft_consensus.cc:2258] T 3a57dc14600c450485f596239b30df0d P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.755353 25959 ts_tablet_manager.cc:1428] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260924 14:28:49.757244 23135 tablet_replica.cc:330] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.758212 23135 raft_consensus.cc:2229] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.759178 23135 raft_consensus.cc:2258] T e22099296aa6419a924560e9fbe81cfd P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.758145 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-e4aee388a7474ac1bb46b761831b809e received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:49.760238 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-e4aee388a7474ac1bb46b761831b809e on tablet e4aee388a7474ac1bb46b761831b809e with peer b8062fa80b5f415e991884a029861641
W20260924 14:28:49.763204 24418 consensus_peers.cc:487] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.764814 25959 ts_tablet_manager.cc:927] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:49.766278 23135 tablet_replica.cc:330] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
W20260924 14:28:49.767221 24344 consensus_peers.cc:487] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.767266 23135 raft_consensus.cc:2229] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:49.778795 23135 raft_consensus.cc:2258] T 870bbbde2a2f4c76bb558d8c43d7587b P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.781373 23135 tablet_replica.cc:330] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.782263 23135 raft_consensus.cc:2229] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.782809 23135 raft_consensus.cc:2258] T 3ca317d21c5f40f69bc0c4ad009f6e94 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:49.783058 24979 consensus_peers.cc:487] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:49.794252 24344 consensus_peers.cc:487] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.796650 23135 tablet_replica.cc:330] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.797645 23135 raft_consensus.cc:2229] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.798200 23135 raft_consensus.cc:2258] T 638ffc5970704a53a26967c81af23e52 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:49.798381 24979 consensus_peers.cc:487] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.801065 23135 tablet_replica.cc:330] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.801998 23135 raft_consensus.cc:2229] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.802577 23135 raft_consensus.cc:2258] T 8fefeab5a75749a198aa72b89e7e2a53 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.805195 23135 tablet_replica.cc:330] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.806038 23135 raft_consensus.cc:2229] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.806593 23135 raft_consensus.cc:2258] T 252120ce7355481189a4a8c3659997ab P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.809666 23135 tablet_replica.cc:330] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.810501 23135 raft_consensus.cc:2229] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:49.811537 23135 raft_consensus.cc:2258] T bb1ed21b7eb4447b93b9cd4b579d4031 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.814889 23135 tablet_replica.cc:330] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.818166 25959 tablet_copy_client.cc:316] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:49.820133 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet 8b62b316cc05443990141d24402b1f4a from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:49.820710 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet 8b62b316cc05443990141d24402b1f4a from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-8b62b316cc05443990141d24402b1f4a
I20260924 14:28:49.828343 25011 raft_consensus.cc:1203] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
W20260924 14:28:49.842664 24418 consensus_peers.cc:487] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.833554 23135 raft_consensus.cc:2229] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.844635 23135 raft_consensus.cc:2258] T 93b3f9be3c4846a8b75e5e42099e6dbc P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:49.847095 24979 consensus_peers.cc:487] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.847034 23135 tablet_replica.cc:330] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.851135 23135 raft_consensus.cc:2229] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.847014 24462 tablet_copy_source_session.cc:205] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:49.851791 23135 raft_consensus.cc:2258] T ce753b9bac814f17a869f4f8d66deca6 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.854905 23135 tablet_replica.cc:330] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.860941 25959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b62b316cc05443990141d24402b1f4a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:49.866164 24344 consensus_peers.cc:487] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.873325 23135 raft_consensus.cc:2229] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.875092 23135 raft_consensus.cc:2258] T 8b62b316cc05443990141d24402b1f4a P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:49.886579 24979 consensus_peers.cc:487] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.881363 23135 tablet_replica.cc:330] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.888913 23135 raft_consensus.cc:2229] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.889586 23135 raft_consensus.cc:2258] T fcf594a126774d39a57f354d4fc7d338 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.892192 23135 tablet_replica.cc:330] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.893002 23135 raft_consensus.cc:2229] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260924 14:28:49.895605 24344 consensus_peers.cc:487] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.896330 23135 raft_consensus.cc:2258] T d5b55145a82545d9b769ea9d04bf1d99 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.900918 23135 tablet_replica.cc:330] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
W20260924 14:28:49.900933 24979 consensus_peers.cc:487] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.906751 23135 raft_consensus.cc:2229] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:49.908416 23135 raft_consensus.cc:2258] T 4bf23ec7c761435d8352db2e38996e74 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.911003 23135 tablet_replica.cc:330] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.911793 23135 raft_consensus.cc:2229] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.912384 23135 raft_consensus.cc:2258] T b18bcddd405d4bbd9d39081d45429923 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.923622 23135 tablet_replica.cc:330] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.924885 23135 raft_consensus.cc:2229] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260924 14:28:49.925287 24418 consensus_peers.cc:487] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.925907 23135 raft_consensus.cc:2258] T 7514385d8e4a45e196a8b870b01c1c76 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.930711 23135 tablet_replica.cc:330] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.933480 25963 tablet_bootstrap.cc:492] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260924 14:28:49.933607 24344 consensus_peers.cc:487] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.935359 23135 raft_consensus.cc:2229] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260924 14:28:49.933449 24418 consensus_peers.cc:487] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.935899 25963 tablet_bootstrap.cc:492] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:49.936813 25963 ts_tablet_manager.cc:1397] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.192s	user 0.028s	sys 0.016s
I20260924 14:28:49.936961 23135 raft_consensus.cc:2258] T 2c4bc71d12c14157982c98b4c7ca505c P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.940289 25963 raft_consensus.cc:348] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:49.940937 23135 tablet_replica.cc:330] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
W20260924 14:28:49.941061 24418 consensus_peers.cc:487] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.941290 25963 raft_consensus.cc:729] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:49.941969 23135 raft_consensus.cc:2229] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:49.942041 25963 consensus_queue.cc:260] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:49.943164 23135 raft_consensus.cc:2258] T 7959d21c60de4745bebfd367d2367714 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.945343 25963 ts_tablet_manager.cc:1428] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:28:49.946210 23135 tablet_replica.cc:330] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.947304 23135 raft_consensus.cc:2229] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.947933 23135 raft_consensus.cc:2258] T e4aee388a7474ac1bb46b761831b809e P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.948310 24387 tablet_copy_service.cc:342] P 61d316eb5dfe405c95019383bbebecaa: Request end of tablet copy session b8062fa80b5f415e991884a029861641-d5b55145a82545d9b769ea9d04bf1d99 received from {username='slave'} at 127.0.0.1:57284
I20260924 14:28:49.948781 24387 tablet_copy_service.cc:433] P 61d316eb5dfe405c95019383bbebecaa: ending tablet copy session b8062fa80b5f415e991884a029861641-d5b55145a82545d9b769ea9d04bf1d99 on tablet d5b55145a82545d9b769ea9d04bf1d99 with peer b8062fa80b5f415e991884a029861641
I20260924 14:28:49.951028 23135 tablet_replica.cc:330] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.952152 23135 raft_consensus.cc:2229] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260924 14:28:49.952306 24979 consensus_peers.cc:487] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.952843 23135 raft_consensus.cc:2258] T 6971512f7f9a4265bb19996677d61320 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.956629 23135 tablet_replica.cc:330] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.957578 23135 raft_consensus.cc:2229] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:28:49.959074 23135 raft_consensus.cc:2258] T c9065d0aec064841b81b70e65f668aac P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.974876 23135 tablet_replica.cc:330] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.975886 23135 raft_consensus.cc:2229] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:49.976924 23135 raft_consensus.cc:2258] T c775016b57f24851b9c52bd2d708b4ef P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.979533 23135 tablet_replica.cc:330] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.980366 25959 tablet_copy_client.cc:791] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:49.980381 23135 raft_consensus.cc:2229] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:49.981138 25959 tablet_copy_client.cc:655] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:49.981809 23135 raft_consensus.cc:2258] T dd02146c57f9474fba0afbcc145a7a2c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.984786 23135 tablet_replica.cc:330] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.986792 23135 raft_consensus.cc:2229] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.987365 23135 raft_consensus.cc:2258] T 5be2b72b568242d3855180c905e78d71 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.990557 23135 tablet_replica.cc:330] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.991518 23135 raft_consensus.cc:2229] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.992113 23135 raft_consensus.cc:2258] T d7a8256035984c7fb87f7e9212b21dc7 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:49.994815 23135 tablet_replica.cc:330] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.995652 23135 raft_consensus.cc:2229] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:49.996220 23135 raft_consensus.cc:2258] T f6bde1e6051745469ca811e5940bd8f9 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:49.997789 24344 consensus_peers.cc:487] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:49.998706 23135 tablet_replica.cc:330] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:49.999362 25959 tablet_copy_client.cc:531] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:49.999713 23135 raft_consensus.cc:2229] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:50.000727 23135 raft_consensus.cc:2258] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.003253 23135 tablet_replica.cc:330] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.004079 23135 raft_consensus.cc:2229] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 5 LEADER]: Raft consensus shutting down.
I20260924 14:28:50.005084 23135 raft_consensus.cc:2258] T ce4efbdf1a8c482facd972a93dfea77f P cafc2c5ad4df4224a5ccc4893de70144 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.012238 23135 tablet_replica.cc:330] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.013080 23135 raft_consensus.cc:2229] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:50.013705 23135 raft_consensus.cc:2258] T 2bdd9163fb2d44068b0f93f1bbfd7387 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.017143 25959 tablet_bootstrap.cc:492] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: Bootstrap starting.
I20260924 14:28:50.019793 23135 tablet_replica.cc:330] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.020608 23135 raft_consensus.cc:2229] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:50.029837 23135 raft_consensus.cc:2258] T dfe06d46f9904520a8709a2b3824f11c P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:50.032150 24344 consensus_peers.cc:487] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:50.034097 23135 tablet_replica.cc:330] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.035055 23135 raft_consensus.cc:2229] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:50.035682 23135 raft_consensus.cc:2258] T fb4aa8a7302f42b688f37d4fdb892b67 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.038653 23135 tablet_replica.cc:330] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.039498 23135 raft_consensus.cc:2229] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:50.040530 23135 raft_consensus.cc:2258] T a2f5e6c143534a47b388c7cd35bd1f4f P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.043125 23135 tablet_replica.cc:330] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.043926 23135 raft_consensus.cc:2229] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:50.045037 23135 raft_consensus.cc:2258] T b53acab36f9a42838efe3526f4aed03a P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.047982 23135 tablet_replica.cc:330] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.048766 23135 raft_consensus.cc:2229] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:50.049271 23135 raft_consensus.cc:2258] T 568b498fb63e4638b92d175b8d122c0e P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:50.050344 24418 consensus_peers.cc:487] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:50.051847 23135 tablet_replica.cc:330] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.052708 23135 raft_consensus.cc:2229] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:50.053759 23135 raft_consensus.cc:2258] T cbae9b0be0e4421eab738bd8fd45a1e1 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.056942 23135 tablet_replica.cc:330] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.057715 23135 raft_consensus.cc:2229] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:50.058265 23135 raft_consensus.cc:2258] T 86057fee00d44a4c9607d990f2782813 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.060959 23135 tablet_replica.cc:330] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.061751 23135 raft_consensus.cc:2229] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:50.062321 23135 raft_consensus.cc:2258] T 317bae4ebac443c8a2fada29c389ab79 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.064973 23135 tablet_replica.cc:330] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.065748 23135 raft_consensus.cc:2229] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:28:50.066697 23135 raft_consensus.cc:2258] T 095719203855447b9091cc7846d566de P cafc2c5ad4df4224a5ccc4893de70144 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.068913 23135 tablet_replica.cc:330] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.069615 23135 raft_consensus.cc:2229] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:50.070122 23135 raft_consensus.cc:2258] T 6fb42513147f483f912eaec2c1151c50 P cafc2c5ad4df4224a5ccc4893de70144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.072196 23135 tablet_replica.cc:330] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.072775 23135 raft_consensus.cc:2229] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:50.073457 23135 raft_consensus.cc:2258] T 3a06a0e1988d4a53b9590e40ca1481b0 P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.075637 23135 tablet_replica.cc:330] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144: stopping tablet replica
I20260924 14:28:50.077055 23135 raft_consensus.cc:2229] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:50.077615 23135 raft_consensus.cc:2258] T b0b642240b2a48ff984338b3604080ef P cafc2c5ad4df4224a5ccc4893de70144 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:50.108415 25959 tablet_bootstrap.cc:492] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:50.109333 25959 tablet_bootstrap.cc:492] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:50.110023 25959 ts_tablet_manager.cc:1397] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.093s	user 0.023s	sys 0.010s
I20260924 14:28:50.113091 25959 raft_consensus.cc:348] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:50.114110 25959 raft_consensus.cc:729] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:50.114796 25959 consensus_queue.cc:260] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:50.121711 25959 ts_tablet_manager.cc:1428] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260924 14:28:50.123600 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-8b62b316cc05443990141d24402b1f4a received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:50.124071 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-8b62b316cc05443990141d24402b1f4a on tablet 8b62b316cc05443990141d24402b1f4a with peer b8062fa80b5f415e991884a029861641
W20260924 14:28:50.309562 25957 ts_tablet_manager.cc:1005] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Tablet Copy: Unable to fetch data from remote peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Unable to download WAL segment with seq. number 1: unable to fetch data from remote: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:50.310122 25957 tablet_replica.cc:330] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:50.310436 25957 raft_consensus.cc:2229] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641 [term 3 LEARNER]: Raft consensus shutting down.
I20260924 14:28:50.310807 25957 raft_consensus.cc:2258] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641 [term 3 LEARNER]: Raft consensus is shut down!
I20260924 14:28:50.318209 25008 raft_consensus.cc:1203] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
W20260924 14:28:50.322455 25957 tablet_copy_client.cc:1116] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:50.323949 25957 tablet_metadata.cc:476] failed to load DataDirGroup from superblock: Already present: tried to load directory group for tablet c9065d0aec064841b81b70e65f668aac but one is already registered
I20260924 14:28:50.324397 25957 ts_tablet_manager.cc:1905] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260924 14:28:50.333523 25866 raft_consensus.cc:1050] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:50.336331 25866 consensus_queue.cc:237] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:50.366564 24445 raft_consensus.cc:1261] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 14:28:50.369621 25939 consensus_queue.cc:1035] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260924 14:28:50.390442 24344 consensus_peers.cc:487] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:50.395130 25957 ts_tablet_manager.cc:1918] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.0
I20260924 14:28:50.395700 25957 log.cc:1196] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Deleting WAL directory at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1790260019751168-23135-0/minicluster-data/ts-3-root/wals/c9065d0aec064841b81b70e65f668aac
I20260924 14:28:50.409155 25011 raft_consensus.cc:1261] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 14:28:50.420171 25910 consensus_queue.cc:1035] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:28:50.476475 25008 raft_consensus.cc:1203] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260924 14:28:50.490705 25910 raft_consensus.cc:2932] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:50.510294 25011 raft_consensus.cc:2932] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:50.513438 24445 raft_consensus.cc:2932] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:50.551756 24198 catalog_manager.cc:5495] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa reported cstate change: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:50.585868 24370 consensus_queue.cc:237] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:50.595911 25011 raft_consensus.cc:1261] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:50.597720 24445 raft_consensus.cc:1261] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:50.597743 25939 consensus_queue.cc:1035] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:28:50.600575 25939 consensus_queue.cc:1035] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:50.606446 25910 raft_consensus.cc:2932] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:50.610345 24452 raft_consensus.cc:2932] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:50.615471 25011 raft_consensus.cc:2932] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:50.619074 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet e4aee388a7474ac1bb46b761831b809e with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:50.628573 24198 catalog_manager.cc:5495] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
W20260924 14:28:50.674364 24188 catalog_manager.cc:4584] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e4aee388a7474ac1bb46b761831b809e: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:50.863236 25910 raft_consensus.cc:482] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:50.864784 25910 raft_consensus.cc:504] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:50.868943 25910 leader_election.cc:288] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:50.879510 25939 raft_consensus.cc:1050] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:50.881357 25955 raft_consensus.cc:1050] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1: attempting to promote NON_VOTER b8062fa80b5f415e991884a029861641 to VOTER
I20260924 14:28:50.885972 25011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e838ce343bad4d42aa930f68fe4a4b2c" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:50.893865 25862 raft_consensus.cc:482] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:50.894830 25862 raft_consensus.cc:504] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:50.895777 25966 raft_consensus.cc:482] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:50.896678 25966 raft_consensus.cc:504] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:50.897653 25862 leader_election.cc:288] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:50.899523 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870bbbde2a2f4c76bb558d8c43d7587b" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:50.899753 25966 leader_election.cc:288] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:50.890028 25939 consensus_queue.cc:237] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
W20260924 14:28:50.905893 24344 leader_election.cc:334] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:50.906517 24344 leader_election.cc:302] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:50.908164 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
W20260924 14:28:50.910382 24979 leader_election.cc:334] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:50.908222 25910 raft_consensus.cc:2726] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:28:50.910923 24979 leader_election.cc:302] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:50.913102 25862 raft_consensus.cc:2726] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:28:50.916510 24979 leader_election.cc:334] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:50.917028 24979 leader_election.cc:302] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:50.919126 25966 raft_consensus.cc:2726] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:28:50.920969 24445 raft_consensus.cc:1261] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:50.920481 25955 consensus_queue.cc:237] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:50.923079 25910 consensus_queue.cc:1035] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260924 14:28:50.930891 24344 consensus_peers.cc:487] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:50.941799 25011 raft_consensus.cc:1261] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641 [term 1 LEARNER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260924 14:28:50.951113 25981 consensus_queue.cc:1035] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:28:50.972430 25862 raft_consensus.cc:482] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:50.973294 25862 raft_consensus.cc:504] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:50.976509 25910 raft_consensus.cc:482] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:50.977458 25862 leader_election.cc:288] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:50.978091 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e838ce343bad4d42aa930f68fe4a4b2c" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:50.978147 25910 raft_consensus.cc:504] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:50.978942 24370 raft_consensus.cc:2454] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 1.
I20260924 14:28:50.981614 24978 leader_election.cc:302] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:50.989280 25966 raft_consensus.cc:2781] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:50.989838 25966 raft_consensus.cc:482] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:50.990322 25966 raft_consensus.cc:3037] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:50.996842 24370 raft_consensus.cc:1261] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 14:28:50.997895 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3667aba281046d3b974da9065022529" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:50.999015 24979 leader_election.cc:334] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.001389 25938 raft_consensus.cc:482] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.002301 25938 raft_consensus.cc:504] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.005882 25938 leader_election.cc:288] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.011075 25008 raft_consensus.cc:1261] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641 [term 2 LEARNER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260924 14:28:51.007879 25954 consensus_queue.cc:1035] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260924 14:28:50.998358 25910 leader_election.cc:288] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:51.020356 25008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:51.021836 25952 raft_consensus.cc:482] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.024852 25952 raft_consensus.cc:504] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:51.022492 25910 raft_consensus.cc:482] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.023494 25956 consensus_queue.cc:1035] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260924 14:28:51.020838 24418 consensus_peers.cc:487] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:51.031611 25862 raft_consensus.cc:482] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.032552 25862 raft_consensus.cc:504] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.031829 25938 raft_consensus.cc:2932] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:51.039047 25862 leader_election.cc:288] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
W20260924 14:28:51.042387 24344 leader_election.cc:334] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.042906 24344 leader_election.cc:302] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.050804 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:51.052378 25981 raft_consensus.cc:2726] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:28:51.060786 24979 leader_election.cc:334] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.053434 25008 raft_consensus.cc:2932] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
W20260924 14:28:51.068079 24344 leader_election.cc:334] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.068770 24344 leader_election.cc:302] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.070540 25981 raft_consensus.cc:2726] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:28:51.075596 25952 leader_election.cc:288] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.070549 25910 raft_consensus.cc:504] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:51.079670 24418 leader_election.cc:334] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.085479 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:51.086704 25976 raft_consensus.cc:482] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.087698 25976 raft_consensus.cc:504] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.088619 25910 leader_election.cc:288] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:51.090631 25976 leader_election.cc:288] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:51.091847 25954 raft_consensus.cc:482] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.093487 25954 raft_consensus.cc:504] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.092774 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd02146c57f9474fba0afbcc145a7a2c" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
W20260924 14:28:51.106011 24418 leader_election.cc:334] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.107422 25939 raft_consensus.cc:482] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.108510 25939 raft_consensus.cc:504] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
W20260924 14:28:51.110003 24344 leader_election.cc:334] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.110538 24344 leader_election.cc:302] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.115229 25910 raft_consensus.cc:2726] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:28:51.147269 25954 leader_election.cc:288] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
W20260924 14:28:51.157140 24979 leader_election.cc:334] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.162070 24979 leader_election.cc:302] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.161828 24198 catalog_manager.cc:5495] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641 reported cstate change: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:51.185675 25952 raft_consensus.cc:482] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.173514 25976 raft_consensus.cc:2726] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:28:51.186471 25952 raft_consensus.cc:504] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.188495 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3667aba281046d3b974da9065022529" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:51.189162 24376 raft_consensus.cc:2454] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:51.189477 25976 raft_consensus.cc:482] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.190112 25976 raft_consensus.cc:504] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.190459 24417 leader_election.cc:302] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.187179 25939 leader_election.cc:288] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.191509 25939 raft_consensus.cc:482] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.191874 25993 raft_consensus.cc:2781] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.192329 25952 leader_election.cc:288] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.192157 25939 raft_consensus.cc:504] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.192335 25993 raft_consensus.cc:482] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.193606 25993 raft_consensus.cc:3037] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.194483 25939 leader_election.cc:288] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.195901 25966 raft_consensus.cc:504] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.197786 25952 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.200497 25976 leader_election.cc:288] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.200309 25952 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
W20260924 14:28:51.207382 23135 thread.cc:536] Waited for 1000ms trying to join with diag-logger (tid 24324)
W20260924 14:28:51.213363 24418 leader_election.cc:334] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.215826 24344 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 100079us
I20260924 14:28:51.230465 25011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:51.234956 25976 raft_consensus.cc:482] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.234201 24451 raft_consensus.cc:2932] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } attrs { replace: true } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:51.235785 25976 raft_consensus.cc:504] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.238987 25976 leader_election.cc:288] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.240137 25976 raft_consensus.cc:482] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.240926 25976 raft_consensus.cc:504] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.220232 25952 leader_election.cc:288] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.250223 25966 leader_election.cc:288] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.252099 25976 leader_election.cc:288] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:51.255815 25952 raft_consensus.cc:482] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.258239 25952 raft_consensus.cc:504] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:51.260238 24979 leader_election.cc:334] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.260838 24979 leader_election.cc:334] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.263221 24418 leader_election.cc:334] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.266249 25011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:51.266901 25011 raft_consensus.cc:2454] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
W20260924 14:28:51.270526 24979 leader_election.cc:334] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.271160 24979 leader_election.cc:334] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.274737 25952 leader_election.cc:288] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.274948 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:51.276113 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:51.276381 25011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870bbbde2a2f4c76bb558d8c43d7587b" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:51.276783 24370 raft_consensus.cc:2454] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 1.
I20260924 14:28:51.277073 25011 raft_consensus.cc:2454] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:51.278789 25999 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.279381 25008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe06d46f9904520a8709a2b3824f11c" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:51.279335 25999 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.279732 25011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
W20260924 14:28:51.280615 24418 leader_election.cc:334] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.280678 25011 raft_consensus.cc:2454] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 3.
I20260924 14:28:51.281155 24418 leader_election.cc:302] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.281205 24978 leader_election.cc:302] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.282716 25999 leader_election.cc:288] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:51.283020 25955 raft_consensus.cc:2726] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:28:51.289964 26001 raft_consensus.cc:2726] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:28:51.270040 25993 raft_consensus.cc:504] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:51.289240 24344 leader_election.cc:302] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:51.292559 26002 raft_consensus.cc:2781] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260924 14:28:51.292708 24979 leader_election.cc:334] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.293121 26002 raft_consensus.cc:482] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.293586 26002 raft_consensus.cc:3037] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.294389 24978 leader_election.cc:302] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.295073 26001 raft_consensus.cc:2781] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.295521 26001 raft_consensus.cc:482] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.296017 26001 raft_consensus.cc:3037] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.299186 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:51.299978 24370 raft_consensus.cc:2454] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 1.
I20260924 14:28:51.298754 25955 raft_consensus.cc:2932] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
W20260924 14:28:51.300700 24344 leader_election.cc:334] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.301330 24344 leader_election.cc:302] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.302778 25993 leader_election.cc:288] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.303655 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e838ce343bad4d42aa930f68fe4a4b2c" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:51.304265 24376 raft_consensus.cc:3037] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.305990 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:51.306679 24452 raft_consensus.cc:2454] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 3.
I20260924 14:28:51.312278 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:51.312945 24418 leader_election.cc:302] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.314374 25954 raft_consensus.cc:2781] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.314540 24418 leader_election.cc:302] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.314810 25954 raft_consensus.cc:482] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.315289 25954 raft_consensus.cc:3037] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.315763 25956 raft_consensus.cc:2726] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:28:51.316145 24418 leader_election.cc:302] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.316924 25956 raft_consensus.cc:2781] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:28:51.317298 25956 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.317648 25956 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:28:51.319686 25938 raft_consensus.cc:2726] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:28:51.319908 24344 leader_election.cc:334] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.318945 26002 raft_consensus.cc:504] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.323700 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3667aba281046d3b974da9065022529" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:51.324359 24370 raft_consensus.cc:3037] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.324723 24376 raft_consensus.cc:2454] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
W20260924 14:28:51.328040 24418 leader_election.cc:334] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.328174 24978 leader_election.cc:302] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.325493 26002 leader_election.cc:288] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.329180 25998 raft_consensus.cc:2781] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:28:51.340400 25998 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.340812 25998 raft_consensus.cc:3037] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:28:51.348731 25956 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.333070 25954 raft_consensus.cc:504] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.352026 25952 raft_consensus.cc:482] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.339344 26001 raft_consensus.cc:504] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.352862 25952 raft_consensus.cc:504] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:51.329532 24978 leader_election.cc:302] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.354748 26001 leader_election.cc:288] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.356272 25954 leader_election.cc:288] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.356410 25999 raft_consensus.cc:2726] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:28:51.356473 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:51.359184 24376 raft_consensus.cc:2454] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 3.
I20260924 14:28:51.360528 24417 leader_election.cc:302] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:51.361714 25010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:51.362125 25993 raft_consensus.cc:2781] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:28:51.362530 25010 raft_consensus.cc:2379] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 61d316eb5dfe405c95019383bbebecaa in current term 2: Already voted for candidate b8062fa80b5f415e991884a029861641 in this term.
I20260924 14:28:51.362557 25993 raft_consensus.cc:482] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.361913 25011 raft_consensus.cc:2932] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:51.363174 25993 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:28:51.356359 24344 leader_election.cc:334] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.357067 25998 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.379359 26003 raft_consensus.cc:482] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.379910 26001 raft_consensus.cc:482] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.380003 26003 raft_consensus.cc:504] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.380820 25998 leader_election.cc:288] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 4 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:51.380704 26001 raft_consensus.cc:504] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.381713 24344 leader_election.cc:302] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.382551 26003 leader_election.cc:288] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.382673 25938 raft_consensus.cc:2726] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:28:51.382922 26001 leader_election.cc:288] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:51.349825 24370 raft_consensus.cc:2454] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:51.386624 24978 leader_election.cc:302] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.386919 25956 leader_election.cc:288] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.375151 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1ed21b7eb4447b93b9cd4b579d4031" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:51.388054 24978 leader_election.cc:302] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.388973 25998 raft_consensus.cc:2781] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.389428 25998 raft_consensus.cc:482] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.389540 26000 raft_consensus.cc:482] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.389922 25998 raft_consensus.cc:3037] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.390189 26000 raft_consensus.cc:504] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:51.392345 24344 leader_election.cc:334] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.393106 24376 raft_consensus.cc:2379] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b8062fa80b5f415e991884a029861641 in current term 2: Already voted for candidate 61d316eb5dfe405c95019383bbebecaa in this term.
I20260924 14:28:51.393831 25976 raft_consensus.cc:2781] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:51.394572 25976 raft_consensus.cc:686] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:51.395788 26000 leader_election.cc:288] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:51.395955 25976 consensus_queue.cc:237] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.398659 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:51.399412 24452 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 5.
W20260924 14:28:51.400879 24418 leader_election.cc:334] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.403132 24343 leader_election.cc:302] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:51.404112 24417 leader_election.cc:302] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.404362 24452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:51.409178 24452 raft_consensus.cc:2379] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b8062fa80b5f415e991884a029861641 in current term 4: Already voted for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in this term.
I20260924 14:28:51.405622 26000 raft_consensus.cc:2781] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:28:51.406781 25010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:51.410363 26000 raft_consensus.cc:482] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 5 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.410513 25010 raft_consensus.cc:2454] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:51.410874 26000 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:28:51.407196 24979 leader_election.cc:334] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.409013 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe06d46f9904520a8709a2b3824f11c" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" is_pre_election: true
I20260924 14:28:51.412132 24451 raft_consensus.cc:2454] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 1.
I20260924 14:28:51.401783 25952 leader_election.cc:288] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
W20260924 14:28:51.412734 24344 leader_election.cc:334] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.418885 24418 leader_election.cc:334] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.421504 25999 raft_consensus.cc:482] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.425165 25999 raft_consensus.cc:504] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.423462 24978 leader_election.cc:302] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
W20260924 14:28:51.423564 24979 leader_election.cc:334] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.426877 24979 leader_election.cc:334] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.424463 24344 leader_election.cc:302] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.425993 25954 raft_consensus.cc:482] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.425880 24370 raft_consensus.cc:2932] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:51.427371 25952 raft_consensus.cc:2781] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:51.429210 25966 raft_consensus.cc:2726] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:28:51.433398 25938 raft_consensus.cc:2781] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.441023 26002 raft_consensus.cc:482] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.445730 26002 raft_consensus.cc:504] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.448596 26002 leader_election.cc:288] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.440984 25955 raft_consensus.cc:482] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.449298 25938 raft_consensus.cc:482] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.449903 25938 raft_consensus.cc:3037] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.450105 25955 raft_consensus.cc:504] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.453578 25999 leader_election.cc:288] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.432122 25954 raft_consensus.cc:504] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.453660 25955 leader_election.cc:288] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.457345 25954 leader_election.cc:288] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.422116 26000 raft_consensus.cc:504] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:51.409336 25993 raft_consensus.cc:504] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:51.475272 26003 raft_consensus.cc:482] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.479162 26003 raft_consensus.cc:504] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.482136 26003 leader_election.cc:288] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.483126 24200 catalog_manager.cc:5495] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 3 to 4, b8062fa80b5f415e991884a029861641 (127.22.151.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:51.487329 26000 leader_election.cc:288] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 6 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:51.466286 25998 raft_consensus.cc:504] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.492946 25998 leader_election.cc:288] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.489193 25993 leader_election.cc:288] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
W20260924 14:28:51.496284 24978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 63085us
I20260924 14:28:51.503226 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7959d21c60de4745bebfd367d2367714" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:51.503970 24374 raft_consensus.cc:3037] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.504858 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:51.505771 24372 raft_consensus.cc:2454] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 1.
I20260924 14:28:51.507228 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4efbdf1a8c482facd972a93dfea77f" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:51.508193 24448 raft_consensus.cc:3037] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:28:51.512934 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 78809.7us
I20260924 14:28:51.519570 25952 raft_consensus.cc:686] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:51.520752 25952 consensus_queue.cc:237] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
W20260924 14:28:51.523941 24417 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:42533, user_credentials={real_user=slave}} blocked reactor thread for 92388.8us
W20260924 14:28:51.527664 24344 leader_election.cc:334] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.527937 24978 leader_election.cc:302] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.530176 26011 raft_consensus.cc:2726] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:28:51.530550 24978 leader_election.cc:302] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.530731 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9065d0aec064841b81b70e65f668aac" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:51.531680 24372 raft_consensus.cc:3037] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:28:51.532785 25011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870bbbde2a2f4c76bb558d8c43d7587b" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:51.533073 24448 raft_consensus.cc:2454] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 6.
I20260924 14:28:51.533607 25011 raft_consensus.cc:3037] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.534313 24343 leader_election.cc:302] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:51.535171 26002 raft_consensus.cc:2781] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:28:51.535666 26002 raft_consensus.cc:686] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 6 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:51.536571 26002 consensus_queue.cc:237] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } }
I20260924 14:28:51.540177 25010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:51.539239 25862 raft_consensus.cc:2781] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.540946 25010 raft_consensus.cc:2454] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:51.541055 25862 raft_consensus.cc:482] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.541483 25862 raft_consensus.cc:3037] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.542163 25998 raft_consensus.cc:482] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.542735 25998 raft_consensus.cc:504] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.541320 25008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "095719203855447b9091cc7846d566de" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:51.543886 25008 raft_consensus.cc:2379] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in current term 4: Already voted for candidate b8062fa80b5f415e991884a029861641 in this term.
I20260924 14:28:51.543130 24200 catalog_manager.cc:5495] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:51.549893 25011 raft_consensus.cc:2454] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:51.551049 25998 leader_election.cc:288] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
W20260924 14:28:51.552841 24979 leader_election.cc:334] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.555068 24979 leader_election.cc:334] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.563591 25938 raft_consensus.cc:504] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.566277 25938 leader_election.cc:288] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
W20260924 14:28:51.573721 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 54408.9us
W20260924 14:28:51.577490 24418 leader_election.cc:334] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.582405 25006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd02146c57f9474fba0afbcc145a7a2c" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:51.582629 25008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b53acab36f9a42838efe3526f4aed03a" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:51.583202 25006 raft_consensus.cc:2454] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
I20260924 14:28:51.583339 25008 raft_consensus.cc:2454] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 1.
I20260924 14:28:51.585206 25012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641" is_pre_election: true
I20260924 14:28:51.571904 24448 consensus_queue.cc:237] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.585836 25012 raft_consensus.cc:2454] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 1.
W20260924 14:28:51.586556 24979 leader_election.cc:334] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.589210 24374 raft_consensus.cc:2454] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:51.590708 25862 raft_consensus.cc:504] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.602344 24371 raft_consensus.cc:1261] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260924 14:28:51.595697 24418 leader_election.cc:334] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.603122 24418 leader_election.cc:334] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.604882 25952 consensus_queue.cc:1035] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:28:51.562897 24344 leader_election.cc:302] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.607954 26002 raft_consensus.cc:2781] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:28:51.608320 26002 raft_consensus.cc:482] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.608713 26002 raft_consensus.cc:3037] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:51.602377 24372 raft_consensus.cc:2454] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 4.
I20260924 14:28:51.602428 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b53acab36f9a42838efe3526f4aed03a" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa" is_pre_election: true
I20260924 14:28:51.611845 24374 raft_consensus.cc:2454] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 1.
W20260924 14:28:51.615803 24344 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 53006.7us
I20260924 14:28:51.615298 24978 leader_election.cc:302] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.617022 26011 raft_consensus.cc:2781] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.617458 26011 raft_consensus.cc:482] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.617954 26011 raft_consensus.cc:3037] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.618508 24417 leader_election.cc:302] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: 
I20260924 14:28:51.619801 25955 raft_consensus.cc:2781] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:28:51.620385 25955 raft_consensus.cc:686] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
W20260924 14:28:51.621616 24418 leader_election.cc:334] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.621718 25955 consensus_queue.cc:237] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:51.621059 25952 raft_consensus.cc:2932] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:51.626474 25862 leader_election.cc:288] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:51.622179 24200 catalog_manager.cc:5495] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 5 to 6, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 6 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:51.644629 26011 raft_consensus.cc:504] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.648811 26011 leader_election.cc:288] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.652516 26002 raft_consensus.cc:504] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.642047 24201 catalog_manager.cc:5495] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:51.636544 24374 raft_consensus.cc:2932] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:51.661484 24187 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 8b62b316cc05443990141d24402b1f4a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260924 14:28:51.638787 24978 leader_election.cc:302] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.639216 24344 leader_election.cc:302] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:51.664263 25981 raft_consensus.cc:2781] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.664678 25981 raft_consensus.cc:482] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.665062 25981 raft_consensus.cc:3037] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.665187 24978 leader_election.cc:302] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.668164 25862 raft_consensus.cc:2781] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:51.668648 25862 raft_consensus.cc:686] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:51.669224 24448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe06d46f9904520a8709a2b3824f11c" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1"
I20260924 14:28:51.669902 24448 raft_consensus.cc:3037] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.669536 25862 consensus_queue.cc:237] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.672194 25999 raft_consensus.cc:2781] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.672622 25999 raft_consensus.cc:482] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.672966 25999 raft_consensus.cc:3037] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.635308 24418 leader_election.cc:302] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1; no voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.678736 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f5e6c143534a47b388c7cd35bd1f4f" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:51.679209 26015 raft_consensus.cc:2726] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:28:51.679347 24372 raft_consensus.cc:3037] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:28:51.703977 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 68763.7us
I20260924 14:28:51.704771 24418 leader_election.cc:302] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.655289 26002 leader_election.cc:288] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 3 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
W20260924 14:28:51.632514 24979 leader_election.cc:334] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.710363 24979 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 77884.6us
W20260924 14:28:51.709390 24344 leader_election.cc:334] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.712997 24344 leader_election.cc:334] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.714566 25012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf23ec7c761435d8352db2e38996e74" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:51.715255 25012 raft_consensus.cc:3037] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:28:51.717619 24344 leader_election.cc:334] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.706699 26015 raft_consensus.cc:2781] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:51.720419 26015 raft_consensus.cc:686] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:51.720369 25011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c775016b57f24851b9c52bd2d708b4ef" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:51.721148 25011 raft_consensus.cc:3037] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:28:51.721505 26015 consensus_queue.cc:237] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.718382 24372 raft_consensus.cc:2454] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:51.727588 24978 leader_election.cc:302] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: 
I20260924 14:28:51.728770 25998 raft_consensus.cc:2781] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:51.729452 25998 raft_consensus.cc:686] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:51.730453 25998 consensus_queue.cc:237] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.739950 25011 raft_consensus.cc:2454] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 3.
W20260924 14:28:51.729480 24979 leader_election.cc:334] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.706341 24418 leader_election.cc:302] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
W20260924 14:28:51.749477 24344 leader_election.cc:334] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:51.751792 24418 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:41987, user_credentials={real_user=slave}} blocked reactor thread for 45536.5us
I20260924 14:28:51.752470 24418 leader_election.cc:302] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.754668 25012 raft_consensus.cc:2454] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d316eb5dfe405c95019383bbebecaa in term 2.
I20260924 14:28:51.755151 25955 raft_consensus.cc:2781] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.756031 25955 raft_consensus.cc:482] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.756507 25955 raft_consensus.cc:3037] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.760097 24344 leader_election.cc:302] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.760730 26014 raft_consensus.cc:2781] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:28:51.761142 26014 raft_consensus.cc:482] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cafc2c5ad4df4224a5ccc4893de70144)
I20260924 14:28:51.761601 26014 raft_consensus.cc:3037] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.761730 26003 raft_consensus.cc:2781] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:28:51.762231 26003 raft_consensus.cc:686] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 3 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:51.763088 26003 consensus_queue.cc:237] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.763196 25981 raft_consensus.cc:504] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.765985 25981 leader_election.cc:288] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.768725 25012 raft_consensus.cc:1261] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:51.769299 25999 raft_consensus.cc:504] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.759390 24200 catalog_manager.cc:5495] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:51.772616 24372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b53acab36f9a42838efe3526f4aed03a" candidate_uuid: "b8062fa80b5f415e991884a029861641" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61d316eb5dfe405c95019383bbebecaa"
I20260924 14:28:51.773422 24372 raft_consensus.cc:2379] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b8062fa80b5f415e991884a029861641 in current term 2: Already voted for candidate 61d316eb5dfe405c95019383bbebecaa in this term.
I20260924 14:28:51.771065 25955 raft_consensus.cc:504] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.776772 25955 leader_election.cc:288] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
I20260924 14:28:51.785270 26013 consensus_queue.cc:1035] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:28:51.785270 24201 catalog_manager.cc:5495] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:51.787923 25999 leader_election.cc:288] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:51.789182 24448 raft_consensus.cc:2454] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8062fa80b5f415e991884a029861641 in term 2.
I20260924 14:28:51.792760 25006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd02146c57f9474fba0afbcc145a7a2c" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:51.793478 25006 raft_consensus.cc:3037] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:28:51.793653 26014 raft_consensus.cc:504] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.795207 25012 raft_consensus.cc:2932] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } } }
I20260924 14:28:51.799033 24344 leader_election.cc:302] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.800156 26002 raft_consensus.cc:2781] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:51.801578 26002 raft_consensus.cc:686] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Becoming Leader. State: Replica: 61d316eb5dfe405c95019383bbebecaa, State: Running, Role: LEADER
I20260924 14:28:51.803074 26002 consensus_queue.cc:237] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.803428 24978 leader_election.cc:302] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.809350 25999 raft_consensus.cc:2781] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:51.810356 25999 raft_consensus.cc:686] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Becoming Leader. State: Replica: b8062fa80b5f415e991884a029861641, State: Running, Role: LEADER
I20260924 14:28:51.811537 25999 consensus_queue.cc:237] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
W20260924 14:28:51.826396 24344 leader_election.cc:334] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.812108 24200 catalog_manager.cc:5495] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:51.787361 24201 catalog_manager.cc:5495] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 3 to 4, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 4 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:51.815742 26014 leader_election.cc:288] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Requested vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), b8062fa80b5f415e991884a029861641 (127.22.151.196:41987)
W20260924 14:28:51.819226 24418 leader_election.cc:334] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.822371 25006 raft_consensus.cc:2454] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
W20260924 14:28:51.791381 24979 leader_election.cc:334] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.836760 24979 leader_election.cc:302] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.837626 26011 raft_consensus.cc:2726] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260924 14:28:51.838603 24979 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:44507, user_credentials={real_user=slave}} blocked reactor thread for 47309.1us
I20260924 14:28:51.840365 25012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b53acab36f9a42838efe3526f4aed03a" candidate_uuid: "61d316eb5dfe405c95019383bbebecaa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:51.841341 25012 raft_consensus.cc:2379] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 61d316eb5dfe405c95019383bbebecaa in current term 2: Already voted for candidate b8062fa80b5f415e991884a029861641 in this term.
I20260924 14:28:51.849277 24198 catalog_manager.cc:5495] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 2 to 3, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 3 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260924 14:28:51.862469 24188 catalog_manager.cc:4584] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8b62b316cc05443990141d24402b1f4a: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.864423 24344 leader_election.cc:302] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61d316eb5dfe405c95019383bbebecaa; no voters: b8062fa80b5f415e991884a029861641, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.865281 26000 raft_consensus.cc:2726] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:28:51.875780 25006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbae9b0be0e4421eab738bd8fd45a1e1" candidate_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8062fa80b5f415e991884a029861641"
I20260924 14:28:51.878873 25006 raft_consensus.cc:3037] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:28:51.879889 24418 leader_election.cc:334] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:51.880761 24418 leader_election.cc:302] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.884140 24198 catalog_manager.cc:5495] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:51.885610 26015 raft_consensus.cc:2781] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:51.888268 26015 raft_consensus.cc:686] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:51.889405 26015 consensus_queue.cc:237] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:51.902894 24200 catalog_manager.cc:5495] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to b8062fa80b5f415e991884a029861641 (127.22.151.196). New cstate: current_term: 2 leader_uuid: "b8062fa80b5f415e991884a029861641" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260924 14:28:51.908262 24201 catalog_manager.cc:5495] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 61d316eb5dfe405c95019383bbebecaa (127.22.151.194). New cstate: current_term: 2 leader_uuid: "61d316eb5dfe405c95019383bbebecaa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:51.918558 24371 raft_consensus.cc:1261] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:51.921262 26011 consensus_queue.cc:1035] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:51.926678 25006 raft_consensus.cc:2454] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a60ac7d6a0c4795833b917ccdd0c7d1 in term 2.
I20260924 14:28:51.931015 24418 leader_election.cc:302] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, b8062fa80b5f415e991884a029861641; no voters: cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:51.932626 26015 raft_consensus.cc:2781] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:28:51.935056 26015 raft_consensus.cc:686] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 8a60ac7d6a0c4795833b917ccdd0c7d1, State: Running, Role: LEADER
I20260924 14:28:51.941654 24198 catalog_manager.cc:5495] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:51.942821 26015 consensus_queue.cc:237] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
W20260924 14:28:52.010859 24979 consensus_peers.cc:487] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.022996 24448 raft_consensus.cc:1261] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 6 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260924 14:28:52.032001 26002 consensus_queue.cc:1035] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:28:52.045167 24201 catalog_manager.cc:5495] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 reported cstate change: term changed from 1 to 2, leader changed from cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193) to 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:52.050366 24371 raft_consensus.cc:1261] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 4 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260924 14:28:52.056080 24418 consensus_peers.cc:487] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.058141 26014 consensus_queue.cc:1035] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:28:52.070564 24979 consensus_peers.cc:487] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.100199 24418 consensus_peers.cc:487] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.112816 24979 consensus_peers.cc:487] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.113792 24418 consensus_peers.cc:487] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.116092 24418 consensus_peers.cc:487] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.118181 24344 consensus_peers.cc:487] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.118716 24344 consensus_peers.cc:487] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.132647 24418 consensus_peers.cc:487] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer b8062fa80b5f415e991884a029861641 (127.22.151.196:41987): Couldn't send request to peer b8062fa80b5f415e991884a029861641. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.139392 24979 consensus_peers.cc:487] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.166775 24371 raft_consensus.cc:1261] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:52.168015 24372 raft_consensus.cc:1261] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:28:52.169972 26015 consensus_queue.cc:1035] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:28:52.170409 25999 consensus_queue.cc:1035] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:28:52.190148 23135 tablet_server.cc:194] TabletServer@127.22.151.193:0 shutdown complete.
W20260924 14:28:52.190788 24418 consensus_peers.cc:487] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.198489 24344 consensus_peers.cc:487] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.218144 24418 consensus_peers.cc:487] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:28:52.228919 25011 raft_consensus.cc:1261] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260924 14:28:52.230234 26002 consensus_queue.cc:1035] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260924 14:28:52.231549 24418 consensus_peers.cc:487] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.237427 24979 consensus_peers.cc:487] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.242537 25006 raft_consensus.cc:1261] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 61d316eb5dfe405c95019383bbebecaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:52.244177 26002 consensus_queue.cc:1035] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:28:52.255859 24445 raft_consensus.cc:1261] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260924 14:28:52.270221 24979 consensus_peers.cc:487] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:28:52.284188 25012 raft_consensus.cc:1261] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:52.285095 25862 consensus_queue.cc:1035] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:28:52.286000 26013 consensus_queue.cc:1035] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:28:52.310921 24344 consensus_peers.cc:487] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.311656 24344 consensus_peers.cc:487] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.312048 24418 consensus_peers.cc:487] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.312135 24344 consensus_peers.cc:487] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.314348 24372 raft_consensus.cc:1261] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Refusing update from remote peer b8062fa80b5f415e991884a029861641: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:52.316586 25998 consensus_queue.cc:1035] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260924 14:28:52.326030 24418 consensus_peers.cc:487] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.337095 24344 consensus_peers.cc:487] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.353152 24979 consensus_peers.cc:487] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.356777 24979 consensus_peers.cc:487] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.375370 24418 consensus_peers.cc:487] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.378959 24344 consensus_peers.cc:487] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:28:52.399927 23135 tablet_server.cc:178] TabletServer@127.22.151.194:0 shutting down...
W20260924 14:28:52.404379 24344 consensus_peers.cc:487] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.406234 24418 consensus_peers.cc:487] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.412650 24978 consensus_peers.cc:487] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.418059 24979 consensus_peers.cc:487] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.418094 24248 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:28:52.419225 24248 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:28:52.425395 24978 consensus_peers.cc:487] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.432401 24978 consensus_peers.cc:487] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.443024 24978 consensus_peers.cc:487] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.445351 24418 consensus_peers.cc:487] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.445915 24418 consensus_peers.cc:487] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:28:52.450423 25011 raft_consensus.cc:1261] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:52.451612 25952 consensus_queue.cc:1035] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:28:52.452749 24978 consensus_peers.cc:487] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.464145 25012 raft_consensus.cc:1261] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Refusing update from remote peer 8a60ac7d6a0c4795833b917ccdd0c7d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260924 14:28:52.467401 26014 consensus_queue.cc:1035] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:28:52.472076 24978 consensus_peers.cc:487] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.473539 24978 consensus_peers.cc:487] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.481323 24417 consensus_peers.cc:487] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.498569 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:28:52.500277 23135 tablet_replica.cc:330] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.501565 23135 raft_consensus.cc:2229] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.502508 23135 raft_consensus.cc:2258] T b9e83b406c1944efb04163f032a3b4f6 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.505472 23135 tablet_replica.cc:330] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.506417 23135 raft_consensus.cc:2229] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.507242 23135 raft_consensus.cc:2258] T e838ce343bad4d42aa930f68fe4a4b2c P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.510212 23135 tablet_replica.cc:330] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
W20260924 14:28:52.510421 24417 consensus_peers.cc:487] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.512413 23135 raft_consensus.cc:2229] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.513150 23135 raft_consensus.cc:2258] T 234eb8cdbf4c4e7599789194a0032c00 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:52.515012 24979 consensus_peers.cc:487] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:28:52.516000 23135 tablet_replica.cc:330] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.516868 23135 raft_consensus.cc:2229] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.517592 23135 raft_consensus.cc:2258] T c2e4953bd67e40f5823697e67d029f52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.520252 23135 tablet_replica.cc:330] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.520967 23135 raft_consensus.cc:2229] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.521413 23135 raft_consensus.cc:2258] T f3667aba281046d3b974da9065022529 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.523772 23135 tablet_replica.cc:330] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.524677 23135 raft_consensus.cc:2229] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.525307 23135 raft_consensus.cc:2258] T 252120ce7355481189a4a8c3659997ab P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.527803 23135 tablet_replica.cc:330] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.528460 23135 raft_consensus.cc:2229] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.529289 23135 raft_consensus.cc:2258] T bb1ed21b7eb4447b93b9cd4b579d4031 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:52.530920 24344 consensus_peers.cc:487] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:28:52.531774 23135 tablet_replica.cc:330] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.532526 23135 raft_consensus.cc:2229] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.533007 23135 raft_consensus.cc:2258] T 0e065e0c774647b1a9b127accaebd70a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.535293 23135 tablet_replica.cc:330] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.536144 23135 raft_consensus.cc:2229] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.537056 23135 raft_consensus.cc:2258] T fcf594a126774d39a57f354d4fc7d338 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.539588 23135 tablet_replica.cc:330] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
W20260924 14:28:52.540062 24979 consensus_peers.cc:487] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:28:52.540376 23135 raft_consensus.cc:2229] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.541850 23135 raft_consensus.cc:2258] T d5b55145a82545d9b769ea9d04bf1d99 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:52.543002 24417 consensus_peers.cc:487] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.543919 23135 tablet_replica.cc:330] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.544570 23135 raft_consensus.cc:2229] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.545343 23135 raft_consensus.cc:2258] T 5caac6840188451a8e325a37b8ea0e3c P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.547098 23135 tablet_replica.cc:330] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.547808 23135 raft_consensus.cc:2229] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.548643 23135 raft_consensus.cc:2258] T 8fefeab5a75749a198aa72b89e7e2a53 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.550391 23135 tablet_replica.cc:330] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.550989 23135 raft_consensus.cc:2229] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.552106 23135 raft_consensus.cc:2258] T 8df40ee44e21496dbc7ef5dd5e624432 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.554709 23135 tablet_replica.cc:330] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.555527 23135 raft_consensus.cc:2229] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.555984 23135 raft_consensus.cc:2258] T 3ca317d21c5f40f69bc0c4ad009f6e94 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.557991 23135 tablet_replica.cc:330] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.558596 23135 raft_consensus.cc:2229] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.559326 23135 raft_consensus.cc:2258] T a2f5e6c143534a47b388c7cd35bd1f4f P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.562031 23135 tablet_replica.cc:330] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.562862 23135 raft_consensus.cc:2229] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Raft consensus shutting down.
W20260924 14:28:52.562893 24344 consensus_peers.cc:487] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:28:52.564045 23135 raft_consensus.cc:2258] T 6971512f7f9a4265bb19996677d61320 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.566653 23135 tablet_replica.cc:330] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.567481 23135 raft_consensus.cc:2229] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 4 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.568149 23135 raft_consensus.cc:2258] T c9065d0aec064841b81b70e65f668aac P 61d316eb5dfe405c95019383bbebecaa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.571004 23135 tablet_replica.cc:330] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.572053 23135 raft_consensus.cc:2229] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.573385 23135 raft_consensus.cc:2258] T 4bf23ec7c761435d8352db2e38996e74 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.575884 23135 tablet_replica.cc:330] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.576453 23135 raft_consensus.cc:2229] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.577035 23135 raft_consensus.cc:2258] T b18bcddd405d4bbd9d39081d45429923 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.579083 23135 tablet_replica.cc:330] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.579725 23135 raft_consensus.cc:2229] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.580158 23135 raft_consensus.cc:2258] T 37e78577921b4cf18418d98b6f6a30a1 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.581863 23135 tablet_replica.cc:330] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.582443 23135 raft_consensus.cc:2229] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.583331 23135 raft_consensus.cc:2258] T 317bae4ebac443c8a2fada29c389ab79 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.585219 23135 tablet_replica.cc:330] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.585961 23135 raft_consensus.cc:2229] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.586511 23135 pending_rounds.cc:62] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa: Trying to abort 1 pending ops.
I20260924 14:28:52.586722 23135 pending_rounds.cc:68] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa: Aborting op as it isn't in flight: id { term: 2 index: 5 } timestamp: 7332905499452874752 op_type: NO_OP noop_request { }
I20260924 14:28:52.587246 23135 raft_consensus.cc:2866] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260924 14:28:52.587599 23135 raft_consensus.cc:2258] T 7959d21c60de4745bebfd367d2367714 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:52.589793 24417 consensus_peers.cc:487] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.590667 23135 tablet_replica.cc:330] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.591223 23135 raft_consensus.cc:2229] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.592032 23135 raft_consensus.cc:2258] T f6bde1e6051745469ca811e5940bd8f9 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.594313 23135 tablet_replica.cc:330] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.594909 23135 raft_consensus.cc:2229] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 3 LEADER]: Raft consensus shutting down.
W20260924 14:28:52.595193 24417 consensus_peers.cc:487] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.595870 23135 raft_consensus.cc:2258] T c775016b57f24851b9c52bd2d708b4ef P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.597983 23135 tablet_replica.cc:330] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.598485 23135 raft_consensus.cc:2229] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.599035 23135 raft_consensus.cc:2258] T 8b62b316cc05443990141d24402b1f4a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.601330 23135 tablet_replica.cc:330] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.602010 23135 raft_consensus.cc:2229] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.602908 23135 raft_consensus.cc:2258] T e4aee388a7474ac1bb46b761831b809e P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.605172 23135 tablet_replica.cc:330] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.605849 23135 raft_consensus.cc:2229] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.606784 23135 raft_consensus.cc:2258] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.609252 23135 tablet_replica.cc:330] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.609802 23135 raft_consensus.cc:2229] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.610260 23135 raft_consensus.cc:2258] T e79eded71d354d4095b73f2f7f7153e5 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.612121 23135 tablet_replica.cc:330] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.612653 23135 raft_consensus.cc:2229] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.613361 23135 raft_consensus.cc:2258] T 5be2b72b568242d3855180c905e78d71 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.615049 23135 tablet_replica.cc:330] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.615738 23135 raft_consensus.cc:2229] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.616262 23135 raft_consensus.cc:2258] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.618295 23135 tablet_replica.cc:330] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.618623 26043 ts_tablet_manager.cc:927] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Initiating tablet copy from peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
I20260924 14:28:52.618835 23135 raft_consensus.cc:2229] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.619412 23135 raft_consensus.cc:2258] T d7a8256035984c7fb87f7e9212b21dc7 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.620555 26043 tablet_copy_client.cc:316] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: tablet copy: Beginning tablet copy session from remote peer at address 127.22.151.195:35781
I20260924 14:28:52.621690 23135 tablet_replica.cc:330] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.622435 24462 tablet_copy_service.cc:140] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Received BeginTabletCopySession request for tablet c9065d0aec064841b81b70e65f668aac from peer b8062fa80b5f415e991884a029861641 ({username='slave'} at 127.0.0.1:57850)
I20260924 14:28:52.622609 23135 raft_consensus.cc:2229] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.623054 24462 tablet_copy_service.cc:161] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Beginning new tablet copy session on tablet c9065d0aec064841b81b70e65f668aac from peer b8062fa80b5f415e991884a029861641 at {username='slave'} at 127.0.0.1:57850: session id = b8062fa80b5f415e991884a029861641-c9065d0aec064841b81b70e65f668aac
I20260924 14:28:52.623204 23135 raft_consensus.cc:2258] T 4995099d467c4203bc2872f519b99950 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.625815 23135 tablet_replica.cc:330] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.626593 23135 raft_consensus.cc:2229] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.627131 23135 raft_consensus.cc:2258] T 5e48f04a45264e7d8be03ae56ab10560 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.629566 23135 tablet_replica.cc:330] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.630263 23135 raft_consensus.cc:2229] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
W20260924 14:28:52.630698 24417 consensus_peers.cc:487] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.630785 23135 raft_consensus.cc:2258] T 2bdd9163fb2d44068b0f93f1bbfd7387 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.633422 23135 tablet_replica.cc:330] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
W20260924 14:28:52.633400 24978 consensus_peers.cc:487] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.634274 24462 tablet_copy_source_session.cc:205] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 14:28:52.634529 23135 raft_consensus.cc:2229] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 6 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.635507 23135 raft_consensus.cc:2258] T ce4efbdf1a8c482facd972a93dfea77f P 61d316eb5dfe405c95019383bbebecaa [term 6 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:52.635949 24979 consensus_peers.cc:487] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:28:52.637301 26043 ts_tablet_manager.cc:1905] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260924 14:28:52.638067 23135 tablet_replica.cc:330] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.638752 23135 raft_consensus.cc:2229] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.639621 23135 raft_consensus.cc:2258] T 3a57dc14600c450485f596239b30df0d P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.641511 23135 tablet_replica.cc:330] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.642037 23135 raft_consensus.cc:2229] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.642776 23135 raft_consensus.cc:2258] T 3550e2841c1f41939833b5588fb35875 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.644639 23135 tablet_replica.cc:330] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.645157 23135 raft_consensus.cc:2229] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.645651 23135 raft_consensus.cc:2258] T 86057fee00d44a4c9607d990f2782813 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.648025 23135 tablet_replica.cc:330] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.648557 23135 raft_consensus.cc:2229] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.649097 23135 raft_consensus.cc:2258] T fb4aa8a7302f42b688f37d4fdb892b67 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.651144 23135 tablet_replica.cc:330] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.651813 26043 ts_tablet_manager.cc:1918] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.0
I20260924 14:28:52.651834 23135 raft_consensus.cc:2229] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.652693 23135 raft_consensus.cc:2258] T bd5890410f3f4f22a1f26e09cc824bb2 P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.653012 26043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9065d0aec064841b81b70e65f668aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:52.655040 23135 tablet_replica.cc:330] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.655800 23135 raft_consensus.cc:2229] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.656805 23135 raft_consensus.cc:2258] T 3a06a0e1988d4a53b9590e40ca1481b0 P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:52.658020 24417 consensus_peers.cc:487] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.659134 23135 tablet_replica.cc:330] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.662334 23135 raft_consensus.cc:2229] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.662943 23135 raft_consensus.cc:2258] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:52.663792 24418 consensus_peers.cc:487] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:28:52.665479 23135 tablet_replica.cc:330] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.665885 26043 tablet_copy_client.cc:791] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 0 data blocks...
I20260924 14:28:52.666266 23135 raft_consensus.cc:2229] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.666431 26043 tablet_copy_client.cc:655] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: tablet copy: Starting download of 1 WAL segments...
I20260924 14:28:52.666816 23135 raft_consensus.cc:2258] T 638ffc5970704a53a26967c81af23e52 P 61d316eb5dfe405c95019383bbebecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.669179 23135 tablet_replica.cc:330] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.669699 23135 raft_consensus.cc:2229] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.670229 23135 raft_consensus.cc:2258] T bec046486fdc464290fafd6e4978632e P 61d316eb5dfe405c95019383bbebecaa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.670528 26043 tablet_copy_client.cc:531] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 14:28:52.672226 23135 tablet_replica.cc:330] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.672827 23135 raft_consensus.cc:2229] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.673305 23135 raft_consensus.cc:2258] T b0b642240b2a48ff984338b3604080ef P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.675809 23135 tablet_replica.cc:330] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa: stopping tablet replica
I20260924 14:28:52.676355 23135 raft_consensus.cc:2229] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.676986 23135 raft_consensus.cc:2258] T b53acab36f9a42838efe3526f4aed03a P 61d316eb5dfe405c95019383bbebecaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.679092 26043 tablet_bootstrap.cc:492] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Bootstrap starting.
W20260924 14:28:52.703438 24978 consensus_peers.cc:487] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.705353 24417 consensus_peers.cc:487] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.706684 26043 tablet_bootstrap.cc:492] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:28:52.708361 26043 tablet_bootstrap.cc:492] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Bootstrap complete.
I20260924 14:28:52.709164 26043 ts_tablet_manager.cc:1397] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.009s
W20260924 14:28:52.710443 24417 consensus_peers.cc:487] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.712950 26043 raft_consensus.cc:348] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
I20260924 14:28:52.714553 26043 raft_consensus.cc:729] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: b8062fa80b5f415e991884a029861641, State: Initialized, Role: LEARNER
I20260924 14:28:52.715257 26043 consensus_queue.cc:260] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } attrs { replace: true } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: NON_VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: true } }
W20260924 14:28:52.715866 24978 consensus_peers.cc:487] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.717396 26043 ts_tablet_manager.cc:1428] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260924 14:28:52.720130 24462 tablet_copy_service.cc:342] P 8a60ac7d6a0c4795833b917ccdd0c7d1: Request end of tablet copy session b8062fa80b5f415e991884a029861641-c9065d0aec064841b81b70e65f668aac received from {username='slave'} at 127.0.0.1:57850
I20260924 14:28:52.720534 24462 tablet_copy_service.cc:433] P 8a60ac7d6a0c4795833b917ccdd0c7d1: ending tablet copy session b8062fa80b5f415e991884a029861641-c9065d0aec064841b81b70e65f668aac on tablet c9065d0aec064841b81b70e65f668aac with peer b8062fa80b5f415e991884a029861641
W20260924 14:28:52.794752 24417 consensus_peers.cc:487] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:42533: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.798870 23135 tablet_server.cc:194] TabletServer@127.22.151.194:0 shutdown complete.
W20260924 14:28:52.803711 24417 consensus_peers.cc:487] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:42533: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.813490 24418 consensus_peers.cc:487] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Couldn't send request to peer cafc2c5ad4df4224a5ccc4893de70144. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:28:52.830271 24417 consensus_peers.cc:487] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:42533: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.839901 24978 consensus_peers.cc:487] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:42533: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:52.844437 24978 consensus_peers.cc:487] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:42533: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.890420 23135 tablet_server.cc:178] TabletServer@127.22.151.195:0 shutting down...
I20260924 14:28:52.942204 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:28:52.943264 23135 tablet_replica.cc:330] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.943985 23135 raft_consensus.cc:2229] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.944881 23135 raft_consensus.cc:2258] T c2e4953bd67e40f5823697e67d029f52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.946951 23135 tablet_replica.cc:330] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.947669 23135 raft_consensus.cc:2229] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.948455 23135 raft_consensus.cc:2258] T 234eb8cdbf4c4e7599789194a0032c00 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.950156 23135 tablet_replica.cc:330] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.950708 23135 raft_consensus.cc:2229] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.951545 23135 raft_consensus.cc:2258] T 3822638d266742c4908907a61efe8231 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.953696 23135 tablet_replica.cc:330] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.954447 23135 raft_consensus.cc:2229] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.954912 23135 raft_consensus.cc:2258] T b9e83b406c1944efb04163f032a3b4f6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.957417 23135 tablet_replica.cc:330] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.958135 23135 raft_consensus.cc:2229] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.959139 23135 raft_consensus.cc:2258] T e22099296aa6419a924560e9fbe81cfd P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.961557 23135 tablet_replica.cc:330] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.962277 23135 raft_consensus.cc:2229] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.963260 23135 raft_consensus.cc:2258] T f3667aba281046d3b974da9065022529 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:52.964833 24978 consensus_peers.cc:487] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:42533: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.966178 23135 tablet_replica.cc:330] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.967056 23135 raft_consensus.cc:2229] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.967855 23135 raft_consensus.cc:2258] T 0e065e0c774647b1a9b127accaebd70a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.970485 23135 tablet_replica.cc:330] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.971303 23135 raft_consensus.cc:2229] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.971856 23135 raft_consensus.cc:2258] T f6bde1e6051745469ca811e5940bd8f9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.974400 23135 tablet_replica.cc:330] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.975208 23135 raft_consensus.cc:2229] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.975793 23135 raft_consensus.cc:2258] T fcf594a126774d39a57f354d4fc7d338 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.978365 23135 tablet_replica.cc:330] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.979142 23135 raft_consensus.cc:2229] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.979833 23135 raft_consensus.cc:2258] T d5b55145a82545d9b769ea9d04bf1d99 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.982278 23135 tablet_replica.cc:330] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.983144 23135 raft_consensus.cc:2229] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.984233 23135 raft_consensus.cc:2258] T 870bbbde2a2f4c76bb558d8c43d7587b P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.987150 23135 tablet_replica.cc:330] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.987977 23135 raft_consensus.cc:2229] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.988976 23135 raft_consensus.cc:2258] T 252120ce7355481189a4a8c3659997ab P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:52.989457 24978 consensus_peers.cc:487] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 -> Peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Couldn't send request to peer 61d316eb5dfe405c95019383bbebecaa. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:42533: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:52.991523 23135 tablet_replica.cc:330] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.992377 23135 raft_consensus.cc:2229] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:52.992972 23135 raft_consensus.cc:2258] T ce753b9bac814f17a869f4f8d66deca6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.995666 23135 tablet_replica.cc:330] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:52.996447 23135 raft_consensus.cc:2229] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:52.997429 23135 raft_consensus.cc:2258] T 8b62b316cc05443990141d24402b1f4a P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:52.999548 23135 tablet_replica.cc:330] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.000089 23135 raft_consensus.cc:2229] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.000602 23135 raft_consensus.cc:2258] T 5caac6840188451a8e325a37b8ea0e3c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.002825 23135 tablet_replica.cc:330] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.003726 23135 raft_consensus.cc:2229] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.003768 25862 raft_consensus.cc:482] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:28:53.004503 24978 consensus_peers.cc:487] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:53.004334 25862 raft_consensus.cc:504] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:53.004761 23135 raft_consensus.cc:2258] T 8df40ee44e21496dbc7ef5dd5e624432 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.006147 25862 leader_election.cc:288] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:53.007236 23135 tablet_replica.cc:330] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.008213 23135 raft_consensus.cc:2229] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.009219 23135 raft_consensus.cc:2258] T 7514385d8e4a45e196a8b870b01c1c76 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:53.010550 24978 leader_election.cc:334] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Network error: Client connection negotiation failed: client connection to 127.22.151.194:42533: connect: Connection refused (error 111)
W20260924 14:28:53.011040 24979 leader_election.cc:334] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:53.011438 24979 leader_election.cc:302] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:53.011659 23135 tablet_replica.cc:330] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.012334 23135 raft_consensus.cc:2229] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.012431 25862 raft_consensus.cc:2726] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:28:53.013006 23135 raft_consensus.cc:2258] T 37e78577921b4cf18418d98b6f6a30a1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.014935 23135 tablet_replica.cc:330] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.015578 23135 raft_consensus.cc:2229] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.015995 23135 raft_consensus.cc:2258] T 8fefeab5a75749a198aa72b89e7e2a53 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.017784 23135 tablet_replica.cc:330] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.018321 23135 raft_consensus.cc:2229] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.018884 23135 raft_consensus.cc:2258] T 93b3f9be3c4846a8b75e5e42099e6dbc P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.020934 23135 tablet_replica.cc:330] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.021674 23135 raft_consensus.cc:2229] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.022128 23135 raft_consensus.cc:2258] T 6971512f7f9a4265bb19996677d61320 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.024498 23135 tablet_replica.cc:330] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.025450 23135 raft_consensus.cc:2229] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.028661 23135 raft_consensus.cc:2258] T c9065d0aec064841b81b70e65f668aac P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.032164 23135 tablet_replica.cc:330] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.032876 23135 raft_consensus.cc:2229] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.033445 23135 raft_consensus.cc:2258] T 317bae4ebac443c8a2fada29c389ab79 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.035950 23135 tablet_replica.cc:330] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.036743 23135 raft_consensus.cc:2229] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.037209 23135 raft_consensus.cc:2258] T 2c4bc71d12c14157982c98b4c7ca505c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.039623 23135 tablet_replica.cc:330] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.040344 23135 raft_consensus.cc:2229] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.041427 23135 raft_consensus.cc:2258] T dd02146c57f9474fba0afbcc145a7a2c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.043370 23135 tablet_replica.cc:330] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.043926 23135 raft_consensus.cc:2229] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.044466 23135 raft_consensus.cc:2258] T e4aee388a7474ac1bb46b761831b809e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.046095 23135 tablet_replica.cc:330] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.046566 23135 raft_consensus.cc:2229] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.047463 23135 raft_consensus.cc:2258] T 5e48f04a45264e7d8be03ae56ab10560 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.050061 23135 tablet_replica.cc:330] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.050863 23135 raft_consensus.cc:2229] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.051357 23135 raft_consensus.cc:2258] T 5be2b72b568242d3855180c905e78d71 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.053308 23135 tablet_replica.cc:330] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.053840 23135 raft_consensus.cc:2229] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.054388 23135 raft_consensus.cc:2258] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.056311 23135 tablet_replica.cc:330] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.056972 23135 raft_consensus.cc:2229] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.057982 23135 raft_consensus.cc:2258] T 4995099d467c4203bc2872f519b99950 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.060526 23135 tablet_replica.cc:330] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.061249 23135 raft_consensus.cc:2229] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.061755 23135 raft_consensus.cc:2258] T e79eded71d354d4095b73f2f7f7153e5 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.064025 23135 tablet_replica.cc:330] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.064718 23135 raft_consensus.cc:2229] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.065249 23135 raft_consensus.cc:2258] T 0d8ea1ccd8b24bb3a8afb3e6f673c774 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:53.065275 24978 consensus_peers.cc:487] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:53.068053 23135 tablet_replica.cc:330] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.068845 23135 raft_consensus.cc:2229] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.069414 23135 raft_consensus.cc:2258] T ce4efbdf1a8c482facd972a93dfea77f P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.071942 23135 tablet_replica.cc:330] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.072423 23135 raft_consensus.cc:2229] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 LEADER]: Raft consensus shutting down.
W20260924 14:28:53.072937 24188 catalog_manager.cc:4584] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e4aee388a7474ac1bb46b761831b809e: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:53.073185 23135 raft_consensus.cc:2258] T 568b498fb63e4638b92d175b8d122c0e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.075666 23135 tablet_replica.cc:330] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.076380 23135 raft_consensus.cc:2229] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.077551 23135 raft_consensus.cc:2258] T cbae9b0be0e4421eab738bd8fd45a1e1 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.080341 23135 tablet_replica.cc:330] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.081064 23135 raft_consensus.cc:2229] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.081568 23135 raft_consensus.cc:2258] T 095719203855447b9091cc7846d566de P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.083915 23135 tablet_replica.cc:330] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.084596 23135 raft_consensus.cc:2229] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.085528 23135 raft_consensus.cc:2258] T 6fb42513147f483f912eaec2c1151c50 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.088249 23135 tablet_replica.cc:330] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.088933 23135 raft_consensus.cc:2229] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.089864 23135 raft_consensus.cc:2258] T b0b642240b2a48ff984338b3604080ef P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.092401 23135 tablet_replica.cc:330] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.093091 23135 raft_consensus.cc:2229] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.093742 23135 raft_consensus.cc:2258] T bd5890410f3f4f22a1f26e09cc824bb2 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.096112 23135 tablet_replica.cc:330] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.096603 23135 raft_consensus.cc:2229] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.097321 23135 raft_consensus.cc:2258] T 638ffc5970704a53a26967c81af23e52 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.098928 23135 tablet_replica.cc:330] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.099426 23135 raft_consensus.cc:2229] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.099960 23135 raft_consensus.cc:2258] T bec046486fdc464290fafd6e4978632e P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.101804 23135 tablet_replica.cc:330] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.102293 23135 raft_consensus.cc:2229] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.103089 23135 raft_consensus.cc:2258] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.105022 23135 tablet_replica.cc:330] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.105911 23135 raft_consensus.cc:2229] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.106750 23135 raft_consensus.cc:2258] T dfe06d46f9904520a8709a2b3824f11c P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.109429 23135 tablet_replica.cc:330] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.109949 23135 raft_consensus.cc:2229] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.110688 23135 raft_consensus.cc:2258] T 3550e2841c1f41939833b5588fb35875 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.112607 23135 tablet_replica.cc:330] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1: stopping tablet replica
I20260924 14:28:53.113068 23135 raft_consensus.cc:2229] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.113909 23135 raft_consensus.cc:2258] T fb4aa8a7302f42b688f37d4fdb892b67 P 8a60ac7d6a0c4795833b917ccdd0c7d1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:53.143024 24978 consensus_peers.cc:487] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:53.189953 23135 tablet_server.cc:194] TabletServer@127.22.151.195:0 shutdown complete.
W20260924 14:28:53.190387 24978 consensus_peers.cc:487] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:53.191766 24978 consensus_peers.cc:487] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:53.222925 24978 consensus_peers.cc:487] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:53.229593 25998 raft_consensus.cc:482] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:53.230428 25998 raft_consensus.cc:504] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "8a60ac7d6a0c4795833b917ccdd0c7d1" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 35781 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:53.232409 25998 leader_election.cc:288] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781)
W20260924 14:28:53.236326 24979 leader_election.cc:334] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:53.237550 24978 leader_election.cc:334] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111)
I20260924 14:28:53.238003 24978 leader_election.cc:302] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 8a60ac7d6a0c4795833b917ccdd0c7d1, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:53.238824 25998 raft_consensus.cc:2726] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:28:53.256798 24978 consensus_peers.cc:487] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:53.267560 24978 consensus_peers.cc:487] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:53.284598 23135 tablet_server.cc:178] TabletServer@127.22.151.196:0 shutting down...
W20260924 14:28:53.288837 24978 consensus_peers.cc:487] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:28:53.305961 24978 consensus_peers.cc:487] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 -> Peer 8a60ac7d6a0c4795833b917ccdd0c7d1 (127.22.151.195:35781): Couldn't send request to peer 8a60ac7d6a0c4795833b917ccdd0c7d1. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:53.329270 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:28:53.330473 23135 tablet_replica.cc:330] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.331358 23135 raft_consensus.cc:2229] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641 [term 4 LEARNER]: Raft consensus shutting down.
I20260924 14:28:53.331957 23135 raft_consensus.cc:2258] T c9065d0aec064841b81b70e65f668aac P b8062fa80b5f415e991884a029861641 [term 4 LEARNER]: Raft consensus is shut down!
I20260924 14:28:53.334192 23135 tablet_replica.cc:330] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.334882 23135 raft_consensus.cc:2229] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.335610 23135 raft_consensus.cc:2258] T 8b62b316cc05443990141d24402b1f4a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.337594 23135 tablet_replica.cc:330] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.338399 23135 raft_consensus.cc:2229] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.338918 23135 raft_consensus.cc:2258] T d5b55145a82545d9b769ea9d04bf1d99 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.340935 23135 tablet_replica.cc:330] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.341653 23135 raft_consensus.cc:2229] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.342832 23135 raft_consensus.cc:2258] T a2f5e6c143534a47b388c7cd35bd1f4f P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.344933 23135 tablet_replica.cc:330] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.345595 23135 raft_consensus.cc:2229] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.346205 23135 raft_consensus.cc:2258] T 5e48f04a45264e7d8be03ae56ab10560 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.348408 23135 tablet_replica.cc:330] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.349099 23135 raft_consensus.cc:2229] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.349553 23135 raft_consensus.cc:2258] T 3a06a0e1988d4a53b9590e40ca1481b0 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.351300 23135 tablet_replica.cc:330] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.351850 23135 raft_consensus.cc:2229] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.352366 23135 raft_consensus.cc:2258] T 870bbbde2a2f4c76bb558d8c43d7587b P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.354516 23135 tablet_replica.cc:330] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.355144 23135 raft_consensus.cc:2229] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.355665 23135 raft_consensus.cc:2258] T 3a57dc14600c450485f596239b30df0d P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.357434 23135 tablet_replica.cc:330] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.357955 23135 raft_consensus.cc:2229] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.358400 23135 raft_consensus.cc:2258] T 3550e2841c1f41939833b5588fb35875 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.360222 23135 tablet_replica.cc:330] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.360752 23135 raft_consensus.cc:2229] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.361260 23135 raft_consensus.cc:2258] T c2e4953bd67e40f5823697e67d029f52 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.362852 23135 tablet_replica.cc:330] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.363370 23135 raft_consensus.cc:2229] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.363929 23135 raft_consensus.cc:2258] T bb1ed21b7eb4447b93b9cd4b579d4031 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.365646 23135 tablet_replica.cc:330] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.366153 23135 raft_consensus.cc:2229] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.366643 23135 raft_consensus.cc:2258] T 8df40ee44e21496dbc7ef5dd5e624432 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.368561 23135 tablet_replica.cc:330] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.369067 23135 raft_consensus.cc:2229] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.369709 25998 raft_consensus.cc:482] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:53.369964 23135 raft_consensus.cc:2258] T 93b3f9be3c4846a8b75e5e42099e6dbc P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.370237 25998 raft_consensus.cc:504] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cafc2c5ad4df4224a5ccc4893de70144" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 44507 } } peers { permanent_uuid: "61d316eb5dfe405c95019383bbebecaa" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 42533 } } peers { permanent_uuid: "b8062fa80b5f415e991884a029861641" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 41987 } attrs { promote: false } }
I20260924 14:28:53.372042 25998 leader_election.cc:288] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507), 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533)
I20260924 14:28:53.372113 23135 tablet_replica.cc:330] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.373047 23135 raft_consensus.cc:2229] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.374089 23135 raft_consensus.cc:2258] T e838ce343bad4d42aa930f68fe4a4b2c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:53.375682 24979 leader_election.cc:334] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:53.376245 24978 leader_election.cc:334] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 61d316eb5dfe405c95019383bbebecaa (127.22.151.194:42533): Network error: Client connection negotiation failed: client connection to 127.22.151.194:42533: connect: Connection refused (error 111)
I20260924 14:28:53.376518 23135 tablet_replica.cc:330] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.376658 24978 leader_election.cc:302] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8062fa80b5f415e991884a029861641; no voters: 61d316eb5dfe405c95019383bbebecaa, cafc2c5ad4df4224a5ccc4893de70144
I20260924 14:28:53.377277 23135 raft_consensus.cc:2229] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.377334 25998 raft_consensus.cc:2726] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:28:53.378183 23135 raft_consensus.cc:2258] T bd5890410f3f4f22a1f26e09cc824bb2 P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.380359 23135 tablet_replica.cc:330] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.380981 23135 raft_consensus.cc:2229] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.381937 23135 raft_consensus.cc:2258] T bec046486fdc464290fafd6e4978632e P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.383931 23135 tablet_replica.cc:330] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.384440 23135 raft_consensus.cc:2229] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.385234 23135 raft_consensus.cc:2258] T dfe06d46f9904520a8709a2b3824f11c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.386912 23135 tablet_replica.cc:330] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.387404 23135 raft_consensus.cc:2229] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.388264 23135 raft_consensus.cc:2258] T 3ca317d21c5f40f69bc0c4ad009f6e94 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.390386 23135 tablet_replica.cc:330] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.390908 23135 raft_consensus.cc:2229] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.391922 23135 raft_consensus.cc:2258] T 0e065e0c774647b1a9b127accaebd70a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.393684 23135 tablet_replica.cc:330] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.394176 23135 raft_consensus.cc:2229] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.394586 23135 raft_consensus.cc:2258] T e22099296aa6419a924560e9fbe81cfd P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.396456 23135 tablet_replica.cc:330] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.396958 23135 raft_consensus.cc:2229] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.397694 23135 raft_consensus.cc:2258] T 86057fee00d44a4c9607d990f2782813 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.399513 23135 tablet_replica.cc:330] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.400018 23135 raft_consensus.cc:2229] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.400435 23135 raft_consensus.cc:2258] T 9aa9c52e97d0444b8ec17719ea5a4ac6 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.402416 23135 tablet_replica.cc:330] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.403383 23135 raft_consensus.cc:2229] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.404327 23135 raft_consensus.cc:2258] T c9aebcc9d3ed438daf87d68bf2b8f6a9 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.406721 23135 tablet_replica.cc:330] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.407289 23135 raft_consensus.cc:2229] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.407862 23135 raft_consensus.cc:2258] T 7514385d8e4a45e196a8b870b01c1c76 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.410460 23135 tablet_replica.cc:330] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.411223 23135 raft_consensus.cc:2229] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.412353 23135 raft_consensus.cc:2258] T d7a8256035984c7fb87f7e9212b21dc7 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.414444 23135 tablet_replica.cc:330] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.415028 23135 raft_consensus.cc:2229] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.415501 23135 raft_consensus.cc:2258] T 4995099d467c4203bc2872f519b99950 P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.418256 23135 tablet_replica.cc:330] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.418918 23135 raft_consensus.cc:2229] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.419624 23135 raft_consensus.cc:2258] T 4bf23ec7c761435d8352db2e38996e74 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.421504 23135 tablet_replica.cc:330] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.422113 23135 raft_consensus.cc:2229] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.422869 23135 raft_consensus.cc:2258] T 37e78577921b4cf18418d98b6f6a30a1 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.425379 23135 tablet_replica.cc:330] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.426134 23135 raft_consensus.cc:2229] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.427047 23135 raft_consensus.cc:2258] T b18bcddd405d4bbd9d39081d45429923 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.429579 23135 tablet_replica.cc:330] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.430351 23135 raft_consensus.cc:2229] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.431013 23135 raft_consensus.cc:2258] T 3822638d266742c4908907a61efe8231 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.433889 23135 tablet_replica.cc:330] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.434770 23135 raft_consensus.cc:2229] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.435492 23135 raft_consensus.cc:2258] T dd02146c57f9474fba0afbcc145a7a2c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.439014 23135 tablet_replica.cc:330] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.439761 23135 raft_consensus.cc:2229] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.440688 23135 raft_consensus.cc:2258] T 2bdd9163fb2d44068b0f93f1bbfd7387 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.442687 23135 tablet_replica.cc:330] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.443298 23135 raft_consensus.cc:2229] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.443907 23135 raft_consensus.cc:2258] T e4aee388a7474ac1bb46b761831b809e P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.446065 23135 tablet_replica.cc:330] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.446619 23135 raft_consensus.cc:2229] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.447609 23135 raft_consensus.cc:2258] T 7959d21c60de4745bebfd367d2367714 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.449457 23135 tablet_replica.cc:330] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.450019 23135 raft_consensus.cc:2229] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.450438 23135 raft_consensus.cc:2258] T c775016b57f24851b9c52bd2d708b4ef P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.452275 23135 tablet_replica.cc:330] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.452811 23135 raft_consensus.cc:2229] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.453554 23135 raft_consensus.cc:2258] T e79eded71d354d4095b73f2f7f7153e5 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.455489 23135 tablet_replica.cc:330] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.456043 23135 raft_consensus.cc:2229] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.456530 23135 raft_consensus.cc:2258] T 5caac6840188451a8e325a37b8ea0e3c P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.458799 23135 tablet_replica.cc:330] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.459430 23135 raft_consensus.cc:2229] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.460049 23135 raft_consensus.cc:2258] T 6fb42513147f483f912eaec2c1151c50 P b8062fa80b5f415e991884a029861641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.461992 23135 tablet_replica.cc:330] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.462502 23135 raft_consensus.cc:2229] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.462980 23135 raft_consensus.cc:2258] T 095719203855447b9091cc7846d566de P b8062fa80b5f415e991884a029861641 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.465051 23135 tablet_replica.cc:330] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.465586 23135 raft_consensus.cc:2229] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.467056 23135 raft_consensus.cc:2258] T ce753b9bac814f17a869f4f8d66deca6 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.469128 23135 tablet_replica.cc:330] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.469642 23135 raft_consensus.cc:2229] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:28:53.470409 23135 raft_consensus.cc:2258] T 2c4bc71d12c14157982c98b4c7ca505c P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.472728 23135 tablet_replica.cc:330] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.473304 23135 raft_consensus.cc:2229] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.473884 23135 raft_consensus.cc:2258] T cbae9b0be0e4421eab738bd8fd45a1e1 P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.476596 23135 tablet_replica.cc:330] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.477231 23135 raft_consensus.cc:2229] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.478210 23135 raft_consensus.cc:2258] T b53acab36f9a42838efe3526f4aed03a P b8062fa80b5f415e991884a029861641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:53.480324 23135 tablet_replica.cc:330] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641: stopping tablet replica
I20260924 14:28:53.480767 23135 raft_consensus.cc:2229] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:28:53.481192 23135 raft_consensus.cc:2258] T 568b498fb63e4638b92d175b8d122c0e P b8062fa80b5f415e991884a029861641 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260924 14:28:53.485108 24248 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111)
W20260924 14:28:53.489703 24248 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111)
W20260924 14:28:53.490018 24248 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.22.151.195:35781: connect: Connection refused (error 111)
I20260924 14:28:53.571027 23135 tablet_server.cc:194] TabletServer@127.22.151.196:0 shutdown complete.
I20260924 14:28:53.629557 23135 master.cc:537] Master@127.22.151.254:36341 shutting down...
W20260924 14:28:54.126073 24188 catalog_manager.cc:4584] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8b62b316cc05443990141d24402b1f4a: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
W20260924 14:28:55.142038 24188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111) [suppressed 347 similar messages]
W20260924 14:28:55.145223 24188 catalog_manager.cc:4584] TS cafc2c5ad4df4224a5ccc4893de70144 (127.22.151.193:44507): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e4aee388a7474ac1bb46b761831b809e: Network error: Client connection negotiation failed: client connection to 127.22.151.193:44507: connect: Connection refused (error 111)
I20260924 14:28:55.508543 23135 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:28:55.509258 23135 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:28:55.509613 23135 tablet_replica.cc:330] T 00000000000000000000000000000000 P 77786dfd8b794984995b91ede36bc1b6: stopping tablet replica
I20260924 14:28:55.541931 23135 master.cc:552] Master@127.22.151.254:36341 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (55373 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260924 14:28:55.637400 23135 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.22.151.254:43789
I20260924 14:28:55.638556 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:28:55.644781 26057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:55.644779 26056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:55.645969 26059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:55.646284 23135 server_base.cc:1034] running on GCE node
I20260924 14:28:55.648051 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:28:55.648231 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:28:55.648412 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260135648383 us; error 0 us; skew 500 ppm
I20260924 14:28:55.648947 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:28:55.651372 23135 webserver.cc:460] Webserver started at http://127.22.151.254:40781/ using document root <none> and password file <none>
I20260924 14:28:55.652194 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:28:55.652375 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:28:55.652679 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:28:55.653801 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/master-0-root/instance:
uuid: "277acd3330ed4b40b2cea74b4e602a13"
format_stamp: "Formatted at 2026-09-24 14:28:55 on dist-test-slave-pm5x"
I20260924 14:28:55.658288 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260924 14:28:55.661681 26064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:55.662623 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260924 14:28:55.662968 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/master-0-root
uuid: "277acd3330ed4b40b2cea74b4e602a13"
format_stamp: "Formatted at 2026-09-24 14:28:55 on dist-test-slave-pm5x"
I20260924 14:28:55.663214 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:28:55.685804 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:28:55.687057 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:28:55.727550 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.254:43789
I20260924 14:28:55.727681 26115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.254:43789 every 8 connection(s)
I20260924 14:28:55.731684 26116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:55.741691 26116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Bootstrap starting.
I20260924 14:28:55.746184 26116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:55.750561 26116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: No bootstrap required, opened a new log
I20260924 14:28:55.752509 26116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277acd3330ed4b40b2cea74b4e602a13" member_type: VOTER }
I20260924 14:28:55.752880 26116 raft_consensus.cc:374] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:55.753053 26116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277acd3330ed4b40b2cea74b4e602a13, State: Initialized, Role: FOLLOWER
I20260924 14:28:55.753612 26116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277acd3330ed4b40b2cea74b4e602a13" member_type: VOTER }
I20260924 14:28:55.754037 26116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 14:28:55.754220 26116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 14:28:55.754420 26116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:55.758550 26116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277acd3330ed4b40b2cea74b4e602a13" member_type: VOTER }
I20260924 14:28:55.759013 26116 leader_election.cc:302] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 277acd3330ed4b40b2cea74b4e602a13; no voters: 
I20260924 14:28:55.760125 26116 leader_election.cc:288] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260924 14:28:55.760629 26119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:55.762145 26119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 1 LEADER]: Becoming Leader. State: Replica: 277acd3330ed4b40b2cea74b4e602a13, State: Running, Role: LEADER
I20260924 14:28:55.762818 26119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277acd3330ed4b40b2cea74b4e602a13" member_type: VOTER }
I20260924 14:28:55.763059 26116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 14:28:55.767580 26120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "277acd3330ed4b40b2cea74b4e602a13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277acd3330ed4b40b2cea74b4e602a13" member_type: VOTER } }
I20260924 14:28:55.768251 26120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [sys.catalog]: This master's current role is: LEADER
I20260924 14:28:55.768366 26121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 277acd3330ed4b40b2cea74b4e602a13. Latest consensus state: current_term: 1 leader_uuid: "277acd3330ed4b40b2cea74b4e602a13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277acd3330ed4b40b2cea74b4e602a13" member_type: VOTER } }
I20260924 14:28:55.768951 26121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [sys.catalog]: This master's current role is: LEADER
I20260924 14:28:55.773414 26127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 14:28:55.781333 26127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 14:28:55.782495 23135 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 14:28:55.790748 26127 catalog_manager.cc:1347] Generated new cluster ID: 2ab2e8d8ec554c69936f1cf0bd21669d
I20260924 14:28:55.791102 26127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 14:28:55.817891 26127 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 14:28:55.819623 26127 catalog_manager.cc:1504] Loading token signing keys...
I20260924 14:28:55.834302 26127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Generated new TSK 0
I20260924 14:28:55.835036 26127 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 14:28:55.849910 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:28:55.856261 26137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:55.857213 26138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:55.858611 26140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:55.860006 23135 server_base.cc:1034] running on GCE node
I20260924 14:28:55.861159 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:28:55.861351 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:28:55.861558 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260135861545 us; error 0 us; skew 500 ppm
I20260924 14:28:55.862138 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:28:55.865098 23135 webserver.cc:460] Webserver started at http://127.22.151.193:34861/ using document root <none> and password file <none>
I20260924 14:28:55.865649 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:28:55.865804 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:28:55.866123 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:28:55.867311 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/instance:
uuid: "7e09fbebe29a49d291d23f9d395fe335"
format_stamp: "Formatted at 2026-09-24 14:28:55 on dist-test-slave-pm5x"
I20260924 14:28:55.872292 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260924 14:28:55.876168 26145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:55.877195 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260924 14:28:55.877554 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-0-root
uuid: "7e09fbebe29a49d291d23f9d395fe335"
format_stamp: "Formatted at 2026-09-24 14:28:55 on dist-test-slave-pm5x"
I20260924 14:28:55.877913 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:28:55.905727 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:28:55.907073 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:28:55.908668 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:28:55.911092 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:28:55.911281 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:55.911629 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:28:55.911823 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:55.956456 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.193:38943
I20260924 14:28:55.956591 26207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.193:38943 every 8 connection(s)
I20260924 14:28:55.961928 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:28:55.969856 26212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:55.971113 26213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:55.975126 23135 server_base.cc:1034] running on GCE node
W20260924 14:28:55.975917 26215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:55.977185 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:28:55.977454 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:28:55.977655 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260135977641 us; error 0 us; skew 500 ppm
I20260924 14:28:55.978375 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:28:55.980288 26208 heartbeater.cc:347] Connected to a master server at 127.22.151.254:43789
I20260924 14:28:55.980973 26208 heartbeater.cc:464] Registering TS with master...
I20260924 14:28:55.981565 23135 webserver.cc:460] Webserver started at http://127.22.151.194:43609/ using document root <none> and password file <none>
I20260924 14:28:55.982120 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:28:55.982079 26208 heartbeater.cc:511] Master 127.22.151.254:43789 requested a full tablet report, sending...
I20260924 14:28:55.982553 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:28:55.982867 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:28:55.984161 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/instance:
uuid: "6c22052275034c369b7f7dca086fbd1e"
format_stamp: "Formatted at 2026-09-24 14:28:55 on dist-test-slave-pm5x"
I20260924 14:28:55.984784 26081 ts_manager.cc:194] Registered new tserver with Master: 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:55.987725 26081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55180
I20260924 14:28:55.990504 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260924 14:28:55.994441 26220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:55.995345 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260924 14:28:55.995733 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-1-root
uuid: "6c22052275034c369b7f7dca086fbd1e"
format_stamp: "Formatted at 2026-09-24 14:28:55 on dist-test-slave-pm5x"
I20260924 14:28:55.996192 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:28:56.012447 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:28:56.013878 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:28:56.015393 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:28:56.017803 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:28:56.018042 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:56.018307 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:28:56.018473 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:56.063717 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.194:46433
I20260924 14:28:56.063838 26282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.194:46433 every 8 connection(s)
I20260924 14:28:56.068881 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:28:56.076716 26286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:56.077361 26287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:56.081789 26289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:56.083743 23135 server_base.cc:1034] running on GCE node
I20260924 14:28:56.085000 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:28:56.085253 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:28:56.085436 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260136085422 us; error 0 us; skew 500 ppm
I20260924 14:28:56.086103 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:28:56.090687 23135 webserver.cc:460] Webserver started at http://127.22.151.195:45189/ using document root <none> and password file <none>
I20260924 14:28:56.091352 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:28:56.091694 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:28:56.092018 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:28:56.092916 26283 heartbeater.cc:347] Connected to a master server at 127.22.151.254:43789
I20260924 14:28:56.093350 26283 heartbeater.cc:464] Registering TS with master...
I20260924 14:28:56.093384 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/instance:
uuid: "f59f3234edf84aab852fa989fb754d31"
format_stamp: "Formatted at 2026-09-24 14:28:56 on dist-test-slave-pm5x"
I20260924 14:28:56.094255 26283 heartbeater.cc:511] Master 127.22.151.254:43789 requested a full tablet report, sending...
I20260924 14:28:56.096607 26081 ts_manager.cc:194] Registered new tserver with Master: 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:28:56.098794 26081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55184
I20260924 14:28:56.099362 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260924 14:28:56.103392 26294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:56.104321 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260924 14:28:56.104693 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-2-root
uuid: "f59f3234edf84aab852fa989fb754d31"
format_stamp: "Formatted at 2026-09-24 14:28:56 on dist-test-slave-pm5x"
I20260924 14:28:56.104996 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:28:56.130327 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:28:56.131727 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:28:56.133255 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:28:56.135530 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:28:56.135758 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:56.136013 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:28:56.136183 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:56.179945 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.195:45507
I20260924 14:28:56.180039 26356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.195:45507 every 8 connection(s)
I20260924 14:28:56.185017 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:28:56.193689 26360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:28:56.193936 26361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:56.197507 23135 server_base.cc:1034] running on GCE node
W20260924 14:28:56.198558 26363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:28:56.200071 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:28:56.200342 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:28:56.200575 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260136200557 us; error 0 us; skew 500 ppm
I20260924 14:28:56.200733 26357 heartbeater.cc:347] Connected to a master server at 127.22.151.254:43789
I20260924 14:28:56.201138 26357 heartbeater.cc:464] Registering TS with master...
I20260924 14:28:56.201297 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:28:56.202119 26357 heartbeater.cc:511] Master 127.22.151.254:43789 requested a full tablet report, sending...
I20260924 14:28:56.204450 23135 webserver.cc:460] Webserver started at http://127.22.151.196:33383/ using document root <none> and password file <none>
I20260924 14:28:56.204698 26081 ts_manager.cc:194] Registered new tserver with Master: f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:28:56.205024 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:28:56.205340 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:28:56.205626 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:28:56.206634 26081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55200
I20260924 14:28:56.206928 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-3-root/instance:
uuid: "45ab97b763804767846e9d27a01b561f"
format_stamp: "Formatted at 2026-09-24 14:28:56 on dist-test-slave-pm5x"
I20260924 14:28:56.211992 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260924 14:28:56.215827 26368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:56.216805 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260924 14:28:56.217084 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-3-root
uuid: "45ab97b763804767846e9d27a01b561f"
format_stamp: "Formatted at 2026-09-24 14:28:56 on dist-test-slave-pm5x"
I20260924 14:28:56.217343 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:28:56.228077 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:28:56.229369 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:28:56.230767 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:28:56.233168 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:28:56.233366 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:56.233651 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:28:56.233808 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:28:56.279106 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.196:45583
I20260924 14:28:56.279197 26430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.196:45583 every 8 connection(s)
I20260924 14:28:56.298856 26431 heartbeater.cc:347] Connected to a master server at 127.22.151.254:43789
I20260924 14:28:56.299391 26431 heartbeater.cc:464] Registering TS with master...
I20260924 14:28:56.300211 26431 heartbeater.cc:511] Master 127.22.151.254:43789 requested a full tablet report, sending...
I20260924 14:28:56.302416 26081 ts_manager.cc:194] Registered new tserver with Master: 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:28:56.302999 23135 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.020214957s
I20260924 14:28:56.304842 26081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55204
I20260924 14:28:56.306432 23135 test_util.cc:268] Using random seed: 1803216616
I20260924 14:28:56.334936 26081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55218:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260924 14:28:56.609371 26396 tablet_service.cc:1467] Processing CreateTablet for tablet fbdb392c304c4e15b1b6c10c61a95323 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260924 14:28:56.609887 26395 tablet_service.cc:1467] Processing CreateTablet for tablet 00361946476c4fecbd244a86b9cfb752 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260924 14:28:56.611138 26396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbdb392c304c4e15b1b6c10c61a95323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.612649 26394 tablet_service.cc:1467] Processing CreateTablet for tablet 42cfb7094048467797704ab8ee156f83 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260924 14:28:56.613917 26390 tablet_service.cc:1467] Processing CreateTablet for tablet 4630fa065ede40338b319f0846fdd511 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260924 14:28:56.615111 26392 tablet_service.cc:1467] Processing CreateTablet for tablet a83a2b98291f41d7beed3c1329ba4bc9 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260924 14:28:56.616101 26387 tablet_service.cc:1467] Processing CreateTablet for tablet e81d1761568f40418d9259b74501d677 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260924 14:28:56.616585 26392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a83a2b98291f41d7beed3c1329ba4bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.617549 26387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e81d1761568f40418d9259b74501d677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.618192 26394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42cfb7094048467797704ab8ee156f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.619351 26390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4630fa065ede40338b319f0846fdd511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.619643 26388 tablet_service.cc:1467] Processing CreateTablet for tablet 0fae41ae7fca440da0de40b0ee6973c7 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260924 14:28:56.621037 26388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fae41ae7fca440da0de40b0ee6973c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.623860 26389 tablet_service.cc:1467] Processing CreateTablet for tablet d4d74021f6fc4625982decf68a590eac (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260924 14:28:56.625324 26389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4d74021f6fc4625982decf68a590eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.625334 26248 tablet_service.cc:1467] Processing CreateTablet for tablet 78540b1453ae42188ee3fcd5d9d4cb94 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260924 14:28:56.626829 26248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78540b1453ae42188ee3fcd5d9d4cb94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.628234 26241 tablet_service.cc:1467] Processing CreateTablet for tablet 4630fa065ede40338b319f0846fdd511 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260924 14:28:56.629832 26241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4630fa065ede40338b319f0846fdd511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.630113 26246 tablet_service.cc:1467] Processing CreateTablet for tablet 42cfb7094048467797704ab8ee156f83 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260924 14:28:56.612751 26391 tablet_service.cc:1467] Processing CreateTablet for tablet 9301429ace424dc58cbf9b57301e674c (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260924 14:28:56.632865 26391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9301429ace424dc58cbf9b57301e674c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.632565 26245 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9812df92004da0ba2c451e7b423e66 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260924 14:28:56.634878 26244 tablet_service.cc:1467] Processing CreateTablet for tablet 898441fbc17a42e09c7639e00843680b (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260924 14:28:56.628449 26247 tablet_service.cc:1467] Processing CreateTablet for tablet 00361946476c4fecbd244a86b9cfb752 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260924 14:28:56.635955 26395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00361946476c4fecbd244a86b9cfb752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.637230 26243 tablet_service.cc:1467] Processing CreateTablet for tablet 9301429ace424dc58cbf9b57301e674c (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260924 14:28:56.639640 26242 tablet_service.cc:1467] Processing CreateTablet for tablet 168958697ba24715a127c9c7b7c520ff (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260924 14:28:56.643028 26240 tablet_service.cc:1467] Processing CreateTablet for tablet 0fae41ae7fca440da0de40b0ee6973c7 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260924 14:28:56.645977 26239 tablet_service.cc:1467] Processing CreateTablet for tablet dda86ff6977641a8af47dad8d1d1a049 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260924 14:28:56.610869 26393 tablet_service.cc:1467] Processing CreateTablet for tablet 898441fbc17a42e09c7639e00843680b (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260924 14:28:56.649298 26393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898441fbc17a42e09c7639e00843680b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.651706 26246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42cfb7094048467797704ab8ee156f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.652431 26247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00361946476c4fecbd244a86b9cfb752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.653159 26242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 168958697ba24715a127c9c7b7c520ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.655771 26244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898441fbc17a42e09c7639e00843680b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.656422 26245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9812df92004da0ba2c451e7b423e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.657063 26243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9301429ace424dc58cbf9b57301e674c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.659770 26239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dda86ff6977641a8af47dad8d1d1a049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.664038 26240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fae41ae7fca440da0de40b0ee6973c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.681023 26171 tablet_service.cc:1467] Processing CreateTablet for tablet 00361946476c4fecbd244a86b9cfb752 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260924 14:28:56.682590 26171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00361946476c4fecbd244a86b9cfb752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.682964 26172 tablet_service.cc:1467] Processing CreateTablet for tablet 78540b1453ae42188ee3fcd5d9d4cb94 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260924 14:28:56.684473 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78540b1453ae42188ee3fcd5d9d4cb94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.684870 26170 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9812df92004da0ba2c451e7b423e66 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260924 14:28:56.686489 26173 tablet_service.cc:1467] Processing CreateTablet for tablet fbdb392c304c4e15b1b6c10c61a95323 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260924 14:28:56.687198 26169 tablet_service.cc:1467] Processing CreateTablet for tablet 898441fbc17a42e09c7639e00843680b (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260924 14:28:56.688027 26389 tablet_service.cc:1467] Processing CreateTablet for tablet 62575580f9ac4403b5f531f616fe41fd (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260924 14:28:56.689015 26168 tablet_service.cc:1467] Processing CreateTablet for tablet a83a2b98291f41d7beed3c1329ba4bc9 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260924 14:28:56.690352 26389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62575580f9ac4403b5f531f616fe41fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.691121 26167 tablet_service.cc:1467] Processing CreateTablet for tablet 168958697ba24715a127c9c7b7c520ff (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260924 14:28:56.689018 26166 tablet_service.cc:1467] Processing CreateTablet for tablet 4630fa065ede40338b319f0846fdd511 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260924 14:28:56.691151 26165 tablet_service.cc:1467] Processing CreateTablet for tablet d4d74021f6fc4625982decf68a590eac (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260924 14:28:56.694746 26164 tablet_service.cc:1467] Processing CreateTablet for tablet dda86ff6977641a8af47dad8d1d1a049 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260924 14:28:56.704948 26452 tablet_bootstrap.cc:492] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:56.705660 26248 tablet_service.cc:1467] Processing CreateTablet for tablet e81d1761568f40418d9259b74501d677 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260924 14:28:56.721015 26173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbdb392c304c4e15b1b6c10c61a95323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.724078 26169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898441fbc17a42e09c7639e00843680b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.727893 26170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9812df92004da0ba2c451e7b423e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.730590 26165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4d74021f6fc4625982decf68a590eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.735252 26322 tablet_service.cc:1467] Processing CreateTablet for tablet fbdb392c304c4e15b1b6c10c61a95323 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260924 14:28:56.735864 26168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a83a2b98291f41d7beed3c1329ba4bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.736868 26322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbdb392c304c4e15b1b6c10c61a95323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.737910 26167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 168958697ba24715a127c9c7b7c520ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.737607 26318 tablet_service.cc:1467] Processing CreateTablet for tablet a83a2b98291f41d7beed3c1329ba4bc9 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260924 14:28:56.738833 26164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dda86ff6977641a8af47dad8d1d1a049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.739125 26318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a83a2b98291f41d7beed3c1329ba4bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.738893 26321 tablet_service.cc:1467] Processing CreateTablet for tablet 78540b1453ae42188ee3fcd5d9d4cb94 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260924 14:28:56.740540 26321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78540b1453ae42188ee3fcd5d9d4cb94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.741236 26317 tablet_service.cc:1467] Processing CreateTablet for tablet 9301429ace424dc58cbf9b57301e674c (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260924 14:28:56.742646 26317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9301429ace424dc58cbf9b57301e674c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.742563 26320 tablet_service.cc:1467] Processing CreateTablet for tablet 42cfb7094048467797704ab8ee156f83 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260924 14:28:56.743589 26166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4630fa065ede40338b319f0846fdd511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.744012 26320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42cfb7094048467797704ab8ee156f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.744783 26316 tablet_service.cc:1467] Processing CreateTablet for tablet 168958697ba24715a127c9c7b7c520ff (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260924 14:28:56.746225 26316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 168958697ba24715a127c9c7b7c520ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.746035 26319 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9812df92004da0ba2c451e7b423e66 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260924 14:28:56.747520 26319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9812df92004da0ba2c451e7b423e66. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:56.774473 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:56.800997 26315 tablet_service.cc:1467] Processing CreateTablet for tablet d4d74021f6fc4625982decf68a590eac (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260924 14:28:56.804039 26451 tablet_bootstrap.cc:492] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:56.804888 26314 tablet_service.cc:1467] Processing CreateTablet for tablet 0fae41ae7fca440da0de40b0ee6973c7 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260924 14:28:56.808460 26313 tablet_service.cc:1467] Processing CreateTablet for tablet dda86ff6977641a8af47dad8d1d1a049 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260924 14:28:56.813357 26245 tablet_service.cc:1467] Processing CreateTablet for tablet cf849a0475754232bd6bf5cd41d3821c (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260924 14:28:56.818095 26242 tablet_service.cc:1467] Processing CreateTablet for tablet e912296c0f6f4c4d89bbec1581a46cf9 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260924 14:28:56.824556 26248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e81d1761568f40418d9259b74501d677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.817710 26241 tablet_service.cc:1467] Processing CreateTablet for tablet 28de2d954e094686bc5d3b931d2924ce (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260924 14:28:56.836345 26387 tablet_service.cc:1467] Processing CreateTablet for tablet c6e4a72c4886471c8096037b808a6543 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260924 14:28:56.837785 26387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e4a72c4886471c8096037b808a6543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.853245 26246 tablet_service.cc:1467] Processing CreateTablet for tablet 62575580f9ac4403b5f531f616fe41fd (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260924 14:28:56.855592 26313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dda86ff6977641a8af47dad8d1d1a049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.856802 26388 tablet_service.cc:1467] Processing CreateTablet for tablet cf849a0475754232bd6bf5cd41d3821c (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260924 14:28:56.856396 26451 tablet_bootstrap.cc:654] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:56.875820 26314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fae41ae7fca440da0de40b0ee6973c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.876533 26315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4d74021f6fc4625982decf68a590eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.882651 26395 tablet_service.cc:1467] Processing CreateTablet for tablet 28de2d954e094686bc5d3b931d2924ce (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260924 14:28:56.886286 26389 tablet_service.cc:1467] Processing CreateTablet for tablet a2799f93a20d4fd58d617adf2eda3070 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260924 14:28:56.823976 26396 tablet_service.cc:1467] Processing CreateTablet for tablet 014ae451ea994b5e9926dc80215382e4 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260924 14:28:56.865067 26240 tablet_service.cc:1467] Processing CreateTablet for tablet 968ded8aff90408e9baafab16ae17e14 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260924 14:28:56.898218 26245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf849a0475754232bd6bf5cd41d3821c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.910404 26240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 968ded8aff90408e9baafab16ae17e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.911715 26166 tablet_service.cc:1467] Processing CreateTablet for tablet e81d1761568f40418d9259b74501d677 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260924 14:28:56.913736 26166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e81d1761568f40418d9259b74501d677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:56.914697 26243 tablet_service.cc:1467] Processing CreateTablet for tablet a2799f93a20d4fd58d617adf2eda3070 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260924 14:28:56.977134 26452 tablet_bootstrap.cc:654] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:56.921463 26246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62575580f9ac4403b5f531f616fe41fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.015944 26243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2799f93a20d4fd58d617adf2eda3070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.018175 26241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28de2d954e094686bc5d3b931d2924ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.018486 26322 tablet_service.cc:1467] Processing CreateTablet for tablet cf849a0475754232bd6bf5cd41d3821c (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260924 14:28:57.020066 26322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf849a0475754232bd6bf5cd41d3821c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.014649 26314 tablet_service.cc:1467] Processing CreateTablet for tablet 014ae451ea994b5e9926dc80215382e4 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260924 14:28:56.991745 26208 heartbeater.cc:503] Master 127.22.151.254:43789 was elected leader, sending a full tablet report...
I20260924 14:28:56.993664 26247 tablet_service.cc:1467] Processing CreateTablet for tablet 67038d5a253c4d3e8d78c9fc7446aac8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260924 14:28:56.998148 26239 tablet_service.cc:1467] Processing CreateTablet for tablet 64e0b4058f18400dab940d3832586746 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260924 14:28:56.916460 26453 tablet_bootstrap.cc:492] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:28:57.038254 26242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e912296c0f6f4c4d89bbec1581a46cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.041437 26451 tablet_bootstrap.cc:492] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:57.041998 26451 ts_tablet_manager.cc:1397] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.238s	user 0.011s	sys 0.023s
I20260924 14:28:57.045312 26451 raft_consensus.cc:348] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:57.046291 26451 raft_consensus.cc:374] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:57.046677 26451 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:57.053941 26314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 014ae451ea994b5e9926dc80215382e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.061968 26247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67038d5a253c4d3e8d78c9fc7446aac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.082984 26239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64e0b4058f18400dab940d3832586746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.047767 26451 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:57.084100 26318 tablet_service.cc:1467] Processing CreateTablet for tablet e912296c0f6f4c4d89bbec1581a46cf9 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260924 14:28:57.078321 26244 tablet_service.cc:1467] Processing CreateTablet for tablet 612b21159cac4f00b88b7076d49d4fec (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260924 14:28:57.101981 26315 tablet_service.cc:1467] Processing CreateTablet for tablet 968ded8aff90408e9baafab16ae17e14 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260924 14:28:57.059237 26316 tablet_service.cc:1467] Processing CreateTablet for tablet c6e4a72c4886471c8096037b808a6543 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260924 14:28:57.085629 26318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e912296c0f6f4c4d89bbec1581a46cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.127840 26456 tablet_bootstrap.cc:492] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:28:57.111397 26451 ts_tablet_manager.cc:1428] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.069s	user 0.004s	sys 0.011s
I20260924 14:28:57.120013 26431 heartbeater.cc:503] Master 127.22.151.254:43789 was elected leader, sending a full tablet report...
I20260924 14:28:57.085603 26453 tablet_bootstrap.cc:654] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:57.115974 26452 tablet_bootstrap.cc:492] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:57.282307 26246 tablet_service.cc:1467] Processing CreateTablet for tablet 986260198b71418c831bfc4617da4603 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
W20260924 14:28:57.285054 26432 tablet.cc:2369] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 14:28:57.259410 26320 tablet_service.cc:1467] Processing CreateTablet for tablet d2f9b654ec4542a781bdc872c6ddbdce (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260924 14:28:57.449026 26452 ts_tablet_manager.cc:1397] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.744s	user 0.007s	sys 0.018s
I20260924 14:28:57.289772 26247 tablet_service.cc:1467] Processing CreateTablet for tablet 96f2f08e7f5a416886c9a652e4ad11f9 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260924 14:28:57.303017 26319 tablet_service.cc:1467] Processing CreateTablet for tablet 67038d5a253c4d3e8d78c9fc7446aac8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260924 14:28:57.320034 26321 tablet_service.cc:1467] Processing CreateTablet for tablet 28de2d954e094686bc5d3b931d2924ce (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260924 14:28:57.336958 26165 tablet_service.cc:1467] Processing CreateTablet for tablet e912296c0f6f4c4d89bbec1581a46cf9 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260924 14:28:57.337608 26164 tablet_service.cc:1467] Processing CreateTablet for tablet 62575580f9ac4403b5f531f616fe41fd (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260924 14:28:57.338156 26172 tablet_service.cc:1467] Processing CreateTablet for tablet c6e4a72c4886471c8096037b808a6543 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260924 14:28:57.451514 26165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e912296c0f6f4c4d89bbec1581a46cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.452360 26164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62575580f9ac4403b5f531f616fe41fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.340540 26167 tablet_service.cc:1467] Processing CreateTablet for tablet a2799f93a20d4fd58d617adf2eda3070 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260924 14:28:57.341066 26170 tablet_service.cc:1467] Processing CreateTablet for tablet 64e0b4058f18400dab940d3832586746 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260924 14:28:57.341252 26169 tablet_service.cc:1467] Processing CreateTablet for tablet d2f9b654ec4542a781bdc872c6ddbdce (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260924 14:28:57.453469 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e4a72c4886471c8096037b808a6543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.455101 26392 tablet_service.cc:1467] Processing CreateTablet for tablet 612b21159cac4f00b88b7076d49d4fec (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260924 14:28:57.336958 26168 tablet_service.cc:1467] Processing CreateTablet for tablet 014ae451ea994b5e9926dc80215382e4 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260924 14:28:57.341843 26171 tablet_service.cc:1467] Processing CreateTablet for tablet 968ded8aff90408e9baafab16ae17e14 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260924 14:28:57.361007 26317 tablet_service.cc:1467] Processing CreateTablet for tablet 64e0b4058f18400dab940d3832586746 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260924 14:28:57.362859 26391 tablet_service.cc:1467] Processing CreateTablet for tablet d2f9b654ec4542a781bdc872c6ddbdce (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260924 14:28:57.457883 26167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2799f93a20d4fd58d617adf2eda3070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.458626 26393 tablet_service.cc:1467] Processing CreateTablet for tablet 26229a9b02a646ac80993cd410cd7237 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260924 14:28:57.364009 26283 heartbeater.cc:503] Master 127.22.151.254:43789 was elected leader, sending a full tablet report...
I20260924 14:28:57.460395 26169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f9b654ec4542a781bdc872c6ddbdce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.372876 26240 tablet_service.cc:1467] Processing CreateTablet for tablet bcac7743e33046019152a570ae473ca3 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260924 14:28:57.383711 26245 tablet_service.cc:1467] Processing CreateTablet for tablet 1455727abfd74a9aa50ba77fa9c79771 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260924 14:28:57.464018 26171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 968ded8aff90408e9baafab16ae17e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.384935 26357 heartbeater.cc:503] Master 127.22.151.254:43789 was elected leader, sending a full tablet report...
I20260924 14:28:57.467484 26170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64e0b4058f18400dab940d3832586746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.469772 26394 tablet_service.cc:1467] Processing CreateTablet for tablet 1455727abfd74a9aa50ba77fa9c79771 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260924 14:28:57.472220 26168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 014ae451ea994b5e9926dc80215382e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.474710 26172 tablet_service.cc:1467] Processing CreateTablet for tablet bcac7743e33046019152a570ae473ca3 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260924 14:28:57.476094 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcac7743e33046019152a570ae473ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.477789 26452 raft_consensus.cc:348] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:57.478597 26452 raft_consensus.cc:374] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:57.478889 26452 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:57.475270 26390 tablet_service.cc:1467] Processing CreateTablet for tablet 986260198b71418c831bfc4617da4603 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260924 14:28:57.495261 26172 tablet_service.cc:1467] Processing CreateTablet for tablet 986260198b71418c831bfc4617da4603 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260924 14:28:57.496658 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 986260198b71418c831bfc4617da4603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.479808 26452 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:57.507156 26458 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:57.507671 26458 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:57.512964 26458 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:28:57.390815 26322 tablet_service.cc:1467] Processing CreateTablet for tablet 26229a9b02a646ac80993cd410cd7237 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260924 14:28:57.392294 26243 tablet_service.cc:1467] Processing CreateTablet for tablet 051d85003b754a6e95eaf1d19022c217 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260924 14:28:57.392566 26239 tablet_service.cc:1467] Processing CreateTablet for tablet 70ed8dcd463347619717298c81a42998 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260924 14:28:57.394713 26166 tablet_service.cc:1467] Processing CreateTablet for tablet 612b21159cac4f00b88b7076d49d4fec (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260924 14:28:57.396605 26242 tablet_service.cc:1467] Processing CreateTablet for tablet dad9bf5791f146a4b74218b2dc7e458f (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260924 14:28:57.400309 26244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 612b21159cac4f00b88b7076d49d4fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.419783 26457 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:57.426079 26314 tablet_service.cc:1467] Processing CreateTablet for tablet 1455727abfd74a9aa50ba77fa9c79771 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260924 14:28:57.427482 26248 tablet_service.cc:1467] Processing CreateTablet for tablet 80f626c203394b4f8b85ab089d6aa044 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260924 14:28:57.517584 26457 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:57.428658 26313 tablet_service.cc:1467] Processing CreateTablet for tablet bcac7743e33046019152a570ae473ca3 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260924 14:28:57.520681 26457 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:28:57.437973 26173 tablet_service.cc:1467] Processing CreateTablet for tablet 26229a9b02a646ac80993cd410cd7237 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260924 14:28:57.439132 26315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 968ded8aff90408e9baafab16ae17e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.440651 26389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2799f93a20d4fd58d617adf2eda3070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.523576 26314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1455727abfd74a9aa50ba77fa9c79771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.441913 26451 tablet_bootstrap.cc:492] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:57.441604 26387 tablet_service.cc:1467] Processing CreateTablet for tablet 67038d5a253c4d3e8d78c9fc7446aac8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260924 14:28:57.447399 26241 tablet_service.cc:1467] Processing CreateTablet for tablet 4a92b51c57b441158371bdfdf2d193c1 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260924 14:28:57.516616 26243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 051d85003b754a6e95eaf1d19022c217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.547334 26173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26229a9b02a646ac80993cd410cd7237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.548049 26395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28de2d954e094686bc5d3b931d2924ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.578105 26242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad9bf5791f146a4b74218b2dc7e458f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.560727 26321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28de2d954e094686bc5d3b931d2924ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.591379 26166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 612b21159cac4f00b88b7076d49d4fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.596844 26239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ed8dcd463347619717298c81a42998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.597597 26248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80f626c203394b4f8b85ab089d6aa044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.603471 26246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 986260198b71418c831bfc4617da4603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.607667 26317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64e0b4058f18400dab940d3832586746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.607954 26247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96f2f08e7f5a416886c9a652e4ad11f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.609897 26319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67038d5a253c4d3e8d78c9fc7446aac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.610514 26245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1455727abfd74a9aa50ba77fa9c79771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.611748 26320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f9b654ec4542a781bdc872c6ddbdce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.602238 26387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67038d5a253c4d3e8d78c9fc7446aac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.612548 26316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e4a72c4886471c8096037b808a6543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.627219 26241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a92b51c57b441158371bdfdf2d193c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.631186 26240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcac7743e33046019152a570ae473ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.633128 26313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcac7743e33046019152a570ae473ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.635406 26322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26229a9b02a646ac80993cd410cd7237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.635555 26172 tablet_service.cc:1467] Processing CreateTablet for tablet 9f81913531e84a379f3bc3886ea64e95 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260924 14:28:57.637048 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f81913531e84a379f3bc3886ea64e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.644842 26169 tablet_service.cc:1467] Processing CreateTablet for tablet 051d85003b754a6e95eaf1d19022c217 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260924 14:28:57.646189 26169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 051d85003b754a6e95eaf1d19022c217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.648140 26452 ts_tablet_manager.cc:1428] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.199s	user 0.006s	sys 0.018s
I20260924 14:28:57.650727 26396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 014ae451ea994b5e9926dc80215382e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.651392 26391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f9b654ec4542a781bdc872c6ddbdce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.652037 26393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26229a9b02a646ac80993cd410cd7237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.653775 26390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 986260198b71418c831bfc4617da4603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.655109 26167 tablet_service.cc:1467] Processing CreateTablet for tablet 70ed8dcd463347619717298c81a42998 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260924 14:28:57.656934 26167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ed8dcd463347619717298c81a42998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.657716 26394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1455727abfd74a9aa50ba77fa9c79771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.658378 26388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf849a0475754232bd6bf5cd41d3821c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.669914 26392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 612b21159cac4f00b88b7076d49d4fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.682366 26452 tablet_bootstrap.cc:492] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:57.714622 26246 tablet_service.cc:1467] Processing CreateTablet for tablet a24655c8fcef4b9882a0ab16bfc5bbb8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260924 14:28:57.716073 26246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a24655c8fcef4b9882a0ab16bfc5bbb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.723977 26456 tablet_bootstrap.cc:654] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:57.765295 26242 tablet_service.cc:1467] Processing CreateTablet for tablet 3a94b735387741fca7f83059b64c0917 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260924 14:28:57.766755 26242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a94b735387741fca7f83059b64c0917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.753249 26164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 13) took 1056 ms. Trace:
I20260924 14:28:57.771878 26164 rpcz_store.cc:276] 0924 14:28:56.696948 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.336667 (+639719us) service_pool.cc:227] Handling call
0924 14:28:57.451819 (+115152us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.451940 (+   121us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.512954 (+ 61014us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:57.739026 (+226072us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:57.752329 (+ 13303us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:57.776701 26171 tablet_service.cc:1467] Processing CreateTablet for tablet 00498335d0b847f299d347155be5c437 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260924 14:28:57.777356 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:57.778173 26171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00498335d0b847f299d347155be5c437. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.784781 26242 tablet_service.cc:1467] Processing CreateTablet for tablet 9bb2dfc0ec214f5497b6887c8f98ad9f (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260924 14:28:57.786291 26242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bb2dfc0ec214f5497b6887c8f98ad9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.788161 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:28:57.789930 26221 leader_election.cc:341] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:57.790761 26243 tablet_service.cc:1467] Processing CreateTablet for tablet ededbbb82305459a8fbb9dc1f5731ee8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260924 14:28:57.796269 26243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ededbbb82305459a8fbb9dc1f5731ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.797040 26168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 11) took 1100 ms. Trace:
I20260924 14:28:57.797634 26168 rpcz_store.cc:276] 0924 14:28:56.696295 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.336014 (+639719us) service_pool.cc:227] Handling call
0924 14:28:57.471704 (+135690us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.471820 (+   116us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.604804 (+132984us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:57.754653 (+149849us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:57.796977 (+ 42324us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:57.802203 26168 tablet_service.cc:1467] Processing CreateTablet for tablet 4a92b51c57b441158371bdfdf2d193c1 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260924 14:28:57.803311 26453 tablet_bootstrap.cc:492] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:28:57.803813 26168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a92b51c57b441158371bdfdf2d193c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.803889 26453 ts_tablet_manager.cc:1397] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.888s	user 0.025s	sys 0.010s
I20260924 14:28:57.806286 26164 tablet_service.cc:1467] Processing CreateTablet for tablet a24655c8fcef4b9882a0ab16bfc5bbb8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260924 14:28:57.824340 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:28:57.826023 26221 leader_election.cc:341] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:57.829434 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:28:57.830693 26460 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:57.825747 26322 tablet_service.cc:1467] Processing CreateTablet for tablet 9f81913531e84a379f3bc3886ea64e95 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260924 14:28:57.831784 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:28:57.832724 26322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f81913531e84a379f3bc3886ea64e95. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:57.833335 26369 leader_election.cc:341] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:57.826987 26170 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5083097b524eb584b0af59aace9850 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260924 14:28:57.836403 26242 tablet_service.cc:1467] Processing CreateTablet for tablet e188e3a6a42340858d5f3ada3f5e867d (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260924 14:28:57.836854 26452 tablet_bootstrap.cc:654] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:57.839962 26242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e188e3a6a42340858d5f3ada3f5e867d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.842940 26318 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 12) took 1030 ms. Trace:
I20260924 14:28:57.843416 26318 rpcz_store.cc:276] 0924 14:28:56.813242 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.053067 (+239825us) service_pool.cc:227] Handling call
0924 14:28:57.085044 (+ 31977us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.085137 (+    93us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.735273 (+650136us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:57.842430 (+107157us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:57.842880 (+   450us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:57.877076 26318 tablet_service.cc:1467] Processing CreateTablet for tablet 051d85003b754a6e95eaf1d19022c217 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260924 14:28:57.878531 26318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 051d85003b754a6e95eaf1d19022c217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.848023 26173 tablet_service.cc:1467] Processing CreateTablet for tablet 9bb2dfc0ec214f5497b6887c8f98ad9f (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260924 14:28:57.847096 26319 tablet_service.cc:1467] Processing CreateTablet for tablet 96f2f08e7f5a416886c9a652e4ad11f9 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260924 14:28:57.880666 26319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96f2f08e7f5a416886c9a652e4ad11f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.858286 26242 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad26252e4284fb18db8d64f502c4a10 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260924 14:28:57.859903 26164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a24655c8fcef4b9882a0ab16bfc5bbb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.882182 26394 tablet_service.cc:1467] Processing CreateTablet for tablet 9f81913531e84a379f3bc3886ea64e95 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260924 14:28:57.883978 26173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bb2dfc0ec214f5497b6887c8f98ad9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.828819 26244 tablet_service.cc:1467] Processing CreateTablet for tablet 16dc088d2abf460eb278e44fc4f60620 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260924 14:28:57.886019 26170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c5083097b524eb584b0af59aace9850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.827490 26453 raft_consensus.cc:348] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:57.867601 26451 tablet_bootstrap.cc:654] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:57.869910 26392 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 18) took 1003 ms. Trace:
I20260924 14:28:57.874019 26245 tablet_service.cc:1467] Processing CreateTablet for tablet c4ddc9746f0a41f08d38d9ab46bac5ba (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260924 14:28:57.889402 26246 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed216090b8f4d88a827c18c45fd0422 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260924 14:28:57.917865 26242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad26252e4284fb18db8d64f502c4a10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.918689 26453 raft_consensus.cc:374] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:57.939611 26453 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:28:57.940400 26453 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:57.922508 26247 tablet_service.cc:1467] Processing CreateTablet for tablet 5e6029c377f94de7b21145dc0ac2f5a7 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260924 14:28:57.939224 26244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16dc088d2abf460eb278e44fc4f60620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.932730 26165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 12) took 1236 ms. Trace:
I20260924 14:28:57.947618 26322 tablet_service.cc:1467] Processing CreateTablet for tablet dad9bf5791f146a4b74218b2dc7e458f (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260924 14:28:57.951874 26394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f81913531e84a379f3bc3886ea64e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.952416 26320 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 16) took 1025 ms. Trace:
I20260924 14:28:57.934659 26392 rpcz_store.cc:276] 0924 14:28:56.866111 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.454110 (+587999us) service_pool.cc:227] Handling call
0924 14:28:57.456081 (+  1971us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.456192 (+   111us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.764800 (+308608us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:57.869317 (+104517us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:57.869843 (+   526us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:57.976070 26320 rpcz_store.cc:276] 0924 14:28:56.927089 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.258328 (+331239us) service_pool.cc:227] Handling call
0924 14:28:57.449271 (+190943us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.449371 (+   100us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.708384 (+259013us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:57.754807 (+ 46423us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:57.952337 (+197530us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:57.977630 26392 tablet_service.cc:1467] Processing CreateTablet for tablet 96f2f08e7f5a416886c9a652e4ad11f9 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260924 14:28:57.979079 26392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96f2f08e7f5a416886c9a652e4ad11f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.979343 26320 tablet_service.cc:1467] Processing CreateTablet for tablet 4a92b51c57b441158371bdfdf2d193c1 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260924 14:28:57.953833 26396 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 10) took 1341 ms. Trace:
I20260924 14:28:57.980376 26396 rpcz_store.cc:276] 0924 14:28:56.612095 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:56.822936 (+210841us) service_pool.cc:227] Handling call
0924 14:28:57.197800 (+374864us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2fcd21 00007f09be2da1e3 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2 00007f09b788db01
0924 14:28:57.475699 (+277899us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.475795 (+    96us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.770469 (+294674us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:57.953300 (+182831us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:57.953772 (+   472us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":35606784}
I20260924 14:28:57.982409 26396 tablet_service.cc:1467] Processing CreateTablet for tablet 70ed8dcd463347619717298c81a42998 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260924 14:28:57.955152 26388 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 11) took 1342 ms. Trace:
I20260924 14:28:57.964263 26245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ddc9746f0a41f08d38d9ab46bac5ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.986208 26246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed216090b8f4d88a827c18c45fd0422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.972527 26166 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 19) took 1105 ms. Trace:
I20260924 14:28:57.990489 26166 rpcz_store.cc:276] 0924 14:28:56.867125 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.393793 (+526668us) service_pool.cc:227] Handling call
0924 14:28:57.525101 (+131308us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.544501 (+ 19400us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.815570 (+271069us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:57.971937 (+156367us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:57.972462 (+   525us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.004581 26166 tablet_service.cc:1467] Processing CreateTablet for tablet 80f626c203394b4f8b85ab089d6aa044 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260924 14:28:58.005960 26166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80f626c203394b4f8b85ab089d6aa044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.006687 26390 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 21) took 1132 ms. Trace:
I20260924 14:28:58.008392 26247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e6029c377f94de7b21145dc0ac2f5a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.008466 26317 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 18) took 1080 ms. Trace:
I20260924 14:28:58.010653 26248 tablet_service.cc:1467] Processing CreateTablet for tablet 0756687d99e643c094531124c9438376 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260924 14:28:57.987784 26388 rpcz_store.cc:276] 0924 14:28:56.612561 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:56.848256 (+235695us) service_pool.cc:227] Handling call
0924 14:28:57.181514 (+333258us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63eeb7c 00007f09b63ee73b 00007f09bf2deb09 00007f09be21a730 00007f09be2fcd21 00007f09be2da1e3 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2 00007f09b788db01
0924 14:28:57.475522 (+294008us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.475654 (+   132us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.871908 (+396254us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:57.954609 (+ 82701us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:57.955093 (+   484us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":35621760}
I20260924 14:28:58.018935 26317 rpcz_store.cc:276] 0924 14:28:56.927826 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.360124 (+432298us) service_pool.cc:227] Handling call
0924 14:28:57.456989 (+ 96865us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.457084 (+    95us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.841526 (+384442us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:57.977830 (+136304us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.008074 (+ 30244us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eee82 00007f09bf3078b9 00007f09be21a5d8 00007f09be2de063 00007f09be2d9e2f 00007f09be2da5ed 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2
0924 14:28:58.008403 (+   329us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":32434432}
I20260924 14:28:57.955045 26313 tablet_service.cc:1467] Processing CreateTablet for tablet 00498335d0b847f299d347155be5c437 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260924 14:28:58.025610 26067 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45666.7us
I20260924 14:28:58.028342 26388 tablet_service.cc:1467] Processing CreateTablet for tablet 00498335d0b847f299d347155be5c437 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260924 14:28:58.029682 26317 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5083097b524eb584b0af59aace9850 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260924 14:28:58.038978 26315 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 15) took 1112 ms. Trace:
I20260924 14:28:58.030516 26316 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 13) took 1216 ms. Trace:
I20260924 14:28:58.025122 26396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ed8dcd463347619717298c81a42998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.049398 26248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0756687d99e643c094531124c9438376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:57.957410 26165 rpcz_store.cc:276] 0924 14:28:56.696630 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.336014 (+639384us) service_pool.cc:227] Handling call
0924 14:28:57.450947 (+114933us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.451057 (+   110us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.671856 (+220799us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:57.932201 (+260345us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:57.932666 (+   465us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.045082 26322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad9bf5791f146a4b74218b2dc7e458f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.057655 26456 tablet_bootstrap.cc:492] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:58.057327 26239 tablet_service.cc:1467] Processing CreateTablet for tablet 0eec7b6dbbd94479aa179fa23fc1f403 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260924 14:28:58.062147 26387 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 17) took 1205 ms. Trace:
I20260924 14:28:58.062777 26387 rpcz_store.cc:276] 0924 14:28:56.856970 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.440568 (+583598us) service_pool.cc:227] Handling call
0924 14:28:57.552231 (+111663us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.560983 (+  8752us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.858198 (+297215us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.061644 (+203446us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.062089 (+   445us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.093061 26387 tablet_service.cc:1467] Processing CreateTablet for tablet dad9bf5791f146a4b74218b2dc7e458f (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260924 14:28:58.069284 26239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eec7b6dbbd94479aa179fa23fc1f403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.069603 26456 ts_tablet_manager.cc:1397] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.943s	user 0.020s	sys 0.017s
I20260924 14:28:58.100953 26319 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 23) took 1076 ms. Trace:
I20260924 14:28:58.045325 26316 rpcz_store.cc:276] 0924 14:28:56.830862 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.058264 (+227402us) service_pool.cc:227] Handling call
0924 14:28:57.437232 (+378968us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.437338 (+   106us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.775363 (+338025us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.029959 (+254596us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.030453 (+   494us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.115381 26316 tablet_service.cc:1467] Processing CreateTablet for tablet 3a94b735387741fca7f83059b64c0917 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260924 14:28:58.073717 26240 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49586 (request call id 21) took 1200 ms. Trace:
I20260924 14:28:58.117862 26240 rpcz_store.cc:276] 0924 14:28:56.872881 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.081474 (+208593us) service_pool.cc:227] Handling call
0924 14:28:57.463773 (+382299us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.463898 (+   125us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.869418 (+405520us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.073110 (+203692us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.073661 (+   551us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.115937 26244 tablet_service.cc:1467] Processing CreateTablet for tablet 5581950b69104df485f842c6b78b1047 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260924 14:28:58.073868 26172 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 23) took 1048 ms. Trace:
I20260924 14:28:58.120165 26172 rpcz_store.cc:276] 0924 14:28:57.025535 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.600943 (+575408us) service_pool.cc:227] Handling call
0924 14:28:57.636523 (+ 35580us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.636685 (+   162us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.718966 (+ 82281us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.073374 (+354408us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.073808 (+   434us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.074530 26317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c5083097b524eb584b0af59aace9850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.075755 26168 tablet_service.cc:1467] Processing CreateTablet for tablet f59f0af5dd6048918c6579b295ac5510 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260924 14:28:58.122002 26168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f59f0af5dd6048918c6579b295ac5510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.122396 26167 tablet_service.cc:1467] Processing CreateTablet for tablet 78da914f30dc4c65be5ff03ec099253b (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260924 14:28:58.123716 26167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78da914f30dc4c65be5ff03ec099253b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.123751 26313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00498335d0b847f299d347155be5c437. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.123844 26395 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 14) took 1339 ms. Trace:
I20260924 14:28:58.125541 26320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a92b51c57b441158371bdfdf2d193c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.125977 26392 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 23) took 1112 ms. Trace:
W20260924 14:28:58.126816 26204 debug-util.cc:398] Leaking SignalData structure 0x7b0800331bc0 after lost signal to thread 23138
W20260924 14:28:58.127876 26204 debug-util.cc:398] Leaking SignalData structure 0x7b0800259a40 after lost signal to thread 26115
I20260924 14:28:58.076141 26169 tablet_service.cc:1467] Processing CreateTablet for tablet 16dc088d2abf460eb278e44fc4f60620 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260924 14:28:58.076448 26243 tablet_service.cc:1467] Processing CreateTablet for tablet 97ce0c6b37e04bf89405065d71329eb8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260924 14:28:58.126394 26316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a94b735387741fca7f83059b64c0917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.072535 26241 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa1d9a124214b1089cb23dd48f8d322 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260924 14:28:58.045799 26315 rpcz_store.cc:276] 0924 14:28:56.926623 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.100765 (+174142us) service_pool.cc:227] Handling call
0924 14:28:57.435897 (+335132us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.435982 (+    85us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.793185 (+357203us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.038471 (+245286us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.038916 (+   445us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.125226 26395 rpcz_store.cc:276] 0924 14:28:56.784053 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:56.881555 (+ 97502us) service_pool.cc:227] Handling call
0924 14:28:57.472803 (+591248us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.472944 (+   141us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.043982 (+571038us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.092970 (+ 48988us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.123433 (+ 30463us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eee82 00007f09bf3078b9 00007f09be21a5d8 00007f09be2de063 00007f09be2d9e2f 00007f09be2da5ed 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2
0924 14:28:58.123771 (+   338us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3937664}
I20260924 14:28:58.133072 26315 tablet_service.cc:1467] Processing CreateTablet for tablet 9bb2dfc0ec214f5497b6887c8f98ad9f (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260924 14:28:58.133803 26395 tablet_service.cc:1467] Processing CreateTablet for tablet a24655c8fcef4b9882a0ab16bfc5bbb8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260924 14:28:58.134686 26315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bb2dfc0ec214f5497b6887c8f98ad9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.126364 26392 rpcz_store.cc:276] 0924 14:28:57.013834 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.976628 (+962794us) service_pool.cc:227] Handling call
0924 14:28:57.978556 (+  1928us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.978660 (+   104us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.089564 (+110904us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.122673 (+ 33109us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.125918 (+  3245us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.018031 26390 rpcz_store.cc:276] 0924 14:28:56.873934 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.474315 (+600381us) service_pool.cc:227] Handling call
0924 14:28:57.485051 (+ 10736us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.485164 (+   113us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.878403 (+393239us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.006132 (+127729us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.006627 (+   495us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.138473 26172 tablet_service.cc:1467] Processing CreateTablet for tablet 0756687d99e643c094531124c9438376 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260924 14:28:58.079697 26451 tablet_bootstrap.cc:492] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:58.083822 26388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00498335d0b847f299d347155be5c437. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.140293 26451 ts_tablet_manager.cc:1397] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.699s	user 0.010s	sys 0.015s
I20260924 14:28:58.081981 26452 tablet_bootstrap.cc:492] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:58.088819 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:28:58.068291 26165 tablet_service.cc:1467] Processing CreateTablet for tablet e188e3a6a42340858d5f3ada3f5e867d (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260924 14:28:58.142473 26371 leader_election.cc:341] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:58.143148 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:28:58.143771 26387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad9bf5791f146a4b74218b2dc7e458f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.139880 26390 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5083097b524eb584b0af59aace9850 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260924 14:28:58.086621 26171 tablet_service.cc:1467] Processing CreateTablet for tablet ac352e5066864ac6be3f50d39a066493 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260924 14:28:58.146685 26452 ts_tablet_manager.cc:1397] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.464s	user 0.012s	sys 0.004s
I20260924 14:28:58.102753 26456 raft_consensus.cc:348] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.148818 26456 raft_consensus.cc:374] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:58.149114 26456 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:28:58.149766 26456 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.071952 26471 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:58.098526 26393 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 19) took 1229 ms. Trace:
I20260924 14:28:58.143538 26319 rpcz_store.cc:276] 0924 14:28:57.024373 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.846149 (+821776us) service_pool.cc:227] Handling call
0924 14:28:57.880080 (+ 33931us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.880194 (+   114us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.985318 (+105124us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.071152 (+ 85834us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.100893 (+ 29741us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.151217 26393 rpcz_store.cc:276] 0924 14:28:56.868627 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.457651 (+589024us) service_pool.cc:227] Handling call
0924 14:28:57.491796 (+ 34145us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.491903 (+   107us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.969213 (+477310us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.077564 (+108351us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.098455 (+ 20891us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":84,"spinlock_wait_cycles":3328}
I20260924 14:28:58.110133 26164 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed216090b8f4d88a827c18c45fd0422 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260924 14:28:58.151228 26471 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.156198 26471 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:28:58.153149 26393 tablet_service.cc:1467] Processing CreateTablet for tablet 3a94b735387741fca7f83059b64c0917 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260924 14:28:58.159698 26173 tablet_service.cc:1467] Processing CreateTablet for tablet 0eec7b6dbbd94479aa179fa23fc1f403 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260924 14:28:58.153149 26319 tablet_service.cc:1467] Processing CreateTablet for tablet ac352e5066864ac6be3f50d39a066493 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260924 14:28:58.162165 26319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac352e5066864ac6be3f50d39a066493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.045572 26314 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 20) took 1115 ms. Trace:
I20260924 14:28:58.163893 26314 rpcz_store.cc:276] 0924 14:28:56.955679 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.425131 (+469452us) service_pool.cc:227] Handling call
0924 14:28:57.518015 (+ 92884us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.518116 (+   101us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.881337 (+363221us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.042984 (+161647us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.044123 (+  1139us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.098526 26391 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 16) took 1242 ms. Trace:
I20260924 14:28:58.165074 26391 rpcz_store.cc:276] 0924 14:28:56.856625 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.361934 (+505309us) service_pool.cc:227] Handling call
0924 14:28:57.475956 (+114022us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.476061 (+   105us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.851007 (+374946us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.077914 (+226907us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.098455 (+ 20541us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":17536}
I20260924 14:28:58.166929 26391 tablet_service.cc:1467] Processing CreateTablet for tablet 80f626c203394b4f8b85ab089d6aa044 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260924 14:28:58.173754 26317 tablet_service.cc:1467] Processing CreateTablet for tablet ededbbb82305459a8fbb9dc1f5731ee8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260924 14:28:58.075079 26321 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 14) took 1235 ms. Trace:
I20260924 14:28:58.179737 26321 rpcz_store.cc:276] 0924 14:28:56.839938 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.094541 (+254603us) service_pool.cc:227] Handling call
0924 14:28:57.450927 (+356386us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.451013 (+    86us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:57.833727 (+382714us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.039358 (+205631us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.075016 (+ 35658us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.182339 26321 tablet_service.cc:1467] Processing CreateTablet for tablet 16dc088d2abf460eb278e44fc4f60620 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
W20260924 14:28:58.131397 26204 debug-util.cc:398] Leaking SignalData structure 0x7b08002d1280 after lost signal to thread 26430
W20260924 14:28:58.183924 26204 debug-util.cc:398] Leaking SignalData structure 0x7b0800300c80 after lost signal to thread 26448
I20260924 14:28:58.184497 26390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c5083097b524eb584b0af59aace9850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.185226 26393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a94b735387741fca7f83059b64c0917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.185946 26391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80f626c203394b4f8b85ab089d6aa044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.139786 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0756687d99e643c094531124c9438376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.188488 26165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e188e3a6a42340858d5f3ada3f5e867d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.191777 26169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16dc088d2abf460eb278e44fc4f60620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.193039 26452 raft_consensus.cc:348] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:58.109578 26389 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 15) took 1267 ms. Trace:
I20260924 14:28:58.216504 26389 rpcz_store.cc:276] 0924 14:28:56.841826 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:56.885081 (+ 43255us) service_pool.cc:227] Handling call
0924 14:28:57.394918 (+509837us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2fcd21 00007f09be2da1e3 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2 00007f09b788db01
0924 14:28:57.440193 (+ 45275us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.440284 (+    91us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.019313 (+579029us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.108023 (+ 88710us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.109357 (+  1334us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":37079808}
I20260924 14:28:58.104502 26170 tablet_service.cc:1467] Processing CreateTablet for tablet c4ddc9746f0a41f08d38d9ab46bac5ba (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260924 14:28:58.195828 26171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac352e5066864ac6be3f50d39a066493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.196235 26321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16dc088d2abf460eb278e44fc4f60620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.198611 26395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a24655c8fcef4b9882a0ab16bfc5bbb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.223901 26170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ddc9746f0a41f08d38d9ab46bac5ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.219204 26389 tablet_service.cc:1467] Processing CreateTablet for tablet e188e3a6a42340858d5f3ada3f5e867d (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260924 14:28:58.228726 26389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e188e3a6a42340858d5f3ada3f5e867d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.230746 26173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eec7b6dbbd94479aa179fa23fc1f403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.233860 26164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed216090b8f4d88a827c18c45fd0422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.096684 26453 ts_tablet_manager.cc:1428] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.292s	user 0.018s	sys 0.008s
I20260924 14:28:58.143433 26451 raft_consensus.cc:348] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.208333 26243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ce0c6b37e04bf89405065d71329eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.238744 26241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa1d9a124214b1089cb23dd48f8d322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.113673 26394 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 22) took 1230 ms. Trace:
I20260924 14:28:58.239358 26394 rpcz_store.cc:276] 0924 14:28:56.882795 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.881139 (+998344us) service_pool.cc:227] Handling call
0924 14:28:57.883278 (+  2139us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.883400 (+   122us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.083874 (+200474us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.113171 (+ 29297us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.113614 (+   443us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.241658 26394 tablet_service.cc:1467] Processing CreateTablet for tablet f59f0af5dd6048918c6579b295ac5510 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260924 14:28:58.242899 26394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f59f0af5dd6048918c6579b295ac5510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.244295 26453 tablet_bootstrap.cc:492] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:28:58.244041 26396 tablet_service.cc:1467] Processing CreateTablet for tablet ededbbb82305459a8fbb9dc1f5731ee8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260924 14:28:58.245368 26396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ededbbb82305459a8fbb9dc1f5731ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.245982 26388 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad26252e4284fb18db8d64f502c4a10 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260924 14:28:58.247723 26451 raft_consensus.cc:374] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:58.248307 26451 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:58.249332 26451 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.208846 26392 tablet_service.cc:1467] Processing CreateTablet for tablet ac352e5066864ac6be3f50d39a066493 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260924 14:28:58.210319 26318 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 24) took 1001 ms. Trace:
I20260924 14:28:58.195091 26314 tablet_service.cc:1467] Processing CreateTablet for tablet f59f0af5dd6048918c6579b295ac5510 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260924 14:28:58.259029 26315 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 30) took 1048 ms. Trace:
I20260924 14:28:58.261433 26452 raft_consensus.cc:374] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:58.264213 26244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5581950b69104df485f842c6b78b1047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.247313 26388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad26252e4284fb18db8d64f502c4a10. 1 dirs total, 0 dirs full, 0 dirs failed
W20260924 14:28:58.276633 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 134317us
I20260924 14:28:58.296264 26168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 35) took 1076 ms. Trace:
I20260924 14:28:58.269615 26318 rpcz_store.cc:276] 0924 14:28:57.209042 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.876169 (+667127us) service_pool.cc:227] Handling call
0924 14:28:57.877936 (+  1767us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.878090 (+   154us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.085186 (+207096us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.209810 (+124624us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.210258 (+   448us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.306878 26318 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad26252e4284fb18db8d64f502c4a10 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260924 14:28:58.303339 26168 rpcz_store.cc:276] 0924 14:28:57.219971 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.074865 (+854894us) service_pool.cc:227] Handling call
0924 14:28:58.121499 (+ 46634us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.121584 (+    85us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.197412 (+ 75828us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.295667 (+ 98255us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.296195 (+   528us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.317035 26320 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 27) took 1107 ms. Trace:
I20260924 14:28:58.317523 26168 tablet_service.cc:1467] Processing CreateTablet for tablet f272a708f6ca4bd8bc3a3c4925abad6d (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260924 14:28:58.319118 26168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f272a708f6ca4bd8bc3a3c4925abad6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.267076 26313 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 25) took 1057 ms. Trace:
I20260924 14:28:58.277001 26166 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 31) took 1071 ms. Trace:
I20260924 14:28:58.276885 26474 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:58.301626 26452 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:58.317448 26320 rpcz_store.cc:276] 0924 14:28:57.210074 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.978383 (+768309us) service_pool.cc:227] Handling call
0924 14:28:58.017042 (+ 38659us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2fcd21 00007f09be2da1e3 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2 00007f09b788db01
0924 14:28:58.017204 (+   162us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.018762 (+  1558us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.181606 (+162844us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.316464 (+134858us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.316966 (+   502us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":27648896}
I20260924 14:28:58.330099 26453 tablet_bootstrap.cc:654] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:58.324420 26317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ededbbb82305459a8fbb9dc1f5731ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.330917 26166 rpcz_store.cc:276] 0924 14:28:57.205589 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.003606 (+798017us) service_pool.cc:227] Handling call
0924 14:28:58.005443 (+  1837us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.005547 (+   104us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.114124 (+108577us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.276427 (+162303us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.276940 (+   513us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.350517 26320 tablet_service.cc:1467] Processing CreateTablet for tablet c4ddc9746f0a41f08d38d9ab46bac5ba (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260924 14:28:58.355916 26165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 34) took 1139 ms. Trace:
I20260924 14:28:58.365032 26392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac352e5066864ac6be3f50d39a066493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.285712 26315 rpcz_store.cc:276] 0924 14:28:57.211046 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.132178 (+921132us) service_pool.cc:227] Handling call
0924 14:28:58.134217 (+  2039us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.134337 (+   120us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.198423 (+ 64086us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.258423 (+ 60000us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.258961 (+   538us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.364167 26387 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 26) took 1100 ms. Trace:
I20260924 14:28:58.324084 26313 rpcz_store.cc:276] 0924 14:28:57.209413 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.936686 (+727273us) service_pool.cc:227] Handling call
0924 14:28:58.035477 (+ 98791us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.035571 (+    94us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.162821 (+127250us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.266494 (+103673us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.267010 (+   516us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.389446 26319 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 31) took 1175 ms. Trace:
I20260924 14:28:58.367944 26165 rpcz_store.cc:276] 0924 14:28:57.217019 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.067333 (+850314us) service_pool.cc:227] Handling call
0924 14:28:58.141527 (+ 74194us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.141622 (+    95us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.225096 (+ 83474us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.336532 (+111436us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.355848 (+ 19316us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5593}
I20260924 14:28:58.383162 26314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f59f0af5dd6048918c6579b295ac5510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.384882 26322 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 26) took 1175 ms. Trace:
I20260924 14:28:58.414606 26456 ts_tablet_manager.cc:1428] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.315s	user 0.010s	sys 0.025s
I20260924 14:28:58.414904 26313 tablet_service.cc:1467] Processing CreateTablet for tablet 5e6029c377f94de7b21145dc0ac2f5a7 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260924 14:28:58.382607 26452 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:58.416047 26322 rpcz_store.cc:276] 0924 14:28:57.209717 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:57.946639 (+736922us) service_pool.cc:227] Handling call
0924 14:28:57.992155 (+ 45516us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:57.999140 (+  6985us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.220773 (+221633us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.290422 (+ 69649us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.364501 (+ 74079us) spinlock_profiling.cc:243] Waited 9.95 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eee82 00007f09bf3078b9 00007f09be21a5d8 00007f09be2de063 00007f09be2d9e2f 00007f09be2da5ed 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2
0924 14:28:58.382509 (+ 18008us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":22878592}
I20260924 14:28:58.417848 26244 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49586 (request call id 43) took 1015 ms. Trace:
I20260924 14:28:58.418412 26322 tablet_service.cc:1467] Processing CreateTablet for tablet 0756687d99e643c094531124c9438376 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260924 14:28:58.419023 26166 tablet_service.cc:1467] Processing CreateTablet for tablet 97ce0c6b37e04bf89405065d71329eb8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260924 14:28:58.420740 26166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ce0c6b37e04bf89405065d71329eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.422402 26167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 38) took 1192 ms. Trace:
I20260924 14:28:58.423799 26318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad26252e4284fb18db8d64f502c4a10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.395248 26319 rpcz_store.cc:276] 0924 14:28:57.213984 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.152292 (+938308us) service_pool.cc:227] Handling call
0924 14:28:58.161707 (+  9415us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.161797 (+    90us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.313629 (+151832us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.381041 (+ 67412us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.388984 (+  7943us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eee82 00007f09bf3078b9 00007f09be21a5d8 00007f09be2de063 00007f09be2d9e2f 00007f09be2da5ed 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2
0924 14:28:58.389378 (+   394us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10856,"spinlock_wait_cycles":4013312}
I20260924 14:28:58.427774 26319 tablet_service.cc:1467] Processing CreateTablet for tablet f272a708f6ca4bd8bc3a3c4925abad6d (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260924 14:28:58.432632 26451 ts_tablet_manager.cc:1428] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.292s	user 0.026s	sys 0.017s
I20260924 14:28:58.435251 26315 tablet_service.cc:1467] Processing CreateTablet for tablet 78da914f30dc4c65be5ff03ec099253b (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260924 14:28:58.438054 26169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 33) took 1222 ms. Trace:
I20260924 14:28:58.422835 26167 rpcz_store.cc:276] 0924 14:28:57.230025 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.121461 (+891436us) service_pool.cc:227] Handling call
0924 14:28:58.123224 (+  1763us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.123343 (+   119us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.291580 (+168237us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.421761 (+130181us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.422336 (+   575us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.418272 26244 rpcz_store.cc:276] 0924 14:28:57.402185 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.110672 (+708487us) service_pool.cc:227] Handling call
0924 14:28:58.120721 (+ 10049us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.208703 (+ 87982us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.330755 (+122052us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.417294 (+ 86539us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.417787 (+   493us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.444686 26167 tablet_service.cc:1467] Processing CreateTablet for tablet 5581950b69104df485f842c6b78b1047 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260924 14:28:58.446213 26167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5581950b69104df485f842c6b78b1047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.441497 26316 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 29) took 1230 ms. Trace:
I20260924 14:28:58.447638 26316 rpcz_store.cc:276] 0924 14:28:57.210714 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.114463 (+903749us) service_pool.cc:227] Handling call
0924 14:28:58.116217 (+  1754us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.116306 (+    89us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.159637 (+ 43331us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.377466 (+217829us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.441422 (+ 63956us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.449020 26172 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 39) took 1216 ms. Trace:
I20260924 14:28:58.449951 26169 rpcz_store.cc:276] 0924 14:28:57.216022 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.050903 (+834881us) service_pool.cc:227] Handling call
0924 14:28:58.139680 (+ 88777us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.139788 (+   108us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.310014 (+170226us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.423145 (+113131us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.437993 (+ 14848us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.464538 26165 tablet_service.cc:1467] Processing CreateTablet for tablet 951fc36414c641b7a4bf7b6b3e6dc3f2 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260924 14:28:58.466269 26165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 951fc36414c641b7a4bf7b6b3e6dc3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.467775 26321 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 33) took 1250 ms. Trace:
I20260924 14:28:58.468542 26321 rpcz_store.cc:276] 0924 14:28:57.217710 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.180957 (+963247us) service_pool.cc:227] Handling call
0924 14:28:58.195569 (+ 14612us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.195721 (+   152us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.417795 (+222074us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.466084 (+ 48289us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.467700 (+  1616us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.450151 26316 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa1d9a124214b1089cb23dd48f8d322 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260924 14:28:58.449460 26172 rpcz_store.cc:276] 0924 14:28:57.232659 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.137588 (+904929us) service_pool.cc:227] Handling call
0924 14:28:58.139284 (+  1696us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.139402 (+   118us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.208035 (+ 68633us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.448093 (+240058us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.448958 (+   865us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.455238 26320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ddc9746f0a41f08d38d9ab46bac5ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.474422 26313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e6029c377f94de7b21145dc0ac2f5a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.394053 26387 rpcz_store.cc:276] 0924 14:28:57.263684 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.092075 (+828391us) service_pool.cc:227] Handling call
0924 14:28:58.115646 (+ 23571us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.115762 (+   116us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.225099 (+109337us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.300038 (+ 74939us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.348702 (+ 48664us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.476847 26321 tablet_service.cc:1467] Processing CreateTablet for tablet 97ce0c6b37e04bf89405065d71329eb8 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260924 14:28:58.478680 26387 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed216090b8f4d88a827c18c45fd0422 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260924 14:28:58.456075 26393 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 29) took 1191 ms. Trace:
I20260924 14:28:58.479592 26393 rpcz_store.cc:276] 0924 14:28:57.264805 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.152307 (+887502us) service_pool.cc:227] Handling call
0924 14:28:58.157629 (+  5322us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.157718 (+    89us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.311156 (+153438us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.431074 (+119918us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.456000 (+ 24926us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.480152 26387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed216090b8f4d88a827c18c45fd0422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.457293 26451 tablet_bootstrap.cc:492] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:58.459762 26456 tablet_bootstrap.cc:492] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:28:58.482535 26318 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 35) took 1261 ms. Trace:
I20260924 14:28:58.482939 26318 rpcz_store.cc:276] 0924 14:28:57.221316 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.305916 (+1084600us) service_pool.cc:227] Handling call
0924 14:28:58.307829 (+  1913us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.307943 (+   114us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.441008 (+133065us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.482010 (+ 41002us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.482472 (+   462us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.484251 26321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ce0c6b37e04bf89405065d71329eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.460521 26391 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 30) took 1195 ms. Trace:
I20260924 14:28:58.460835 26475 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:58.487838 26475 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.490551 26475 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:58.462960 26171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 32) took 1247 ms. Trace:
I20260924 14:28:58.491636 26171 rpcz_store.cc:276] 0924 14:28:57.215651 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.050771 (+835120us) service_pool.cc:227] Handling call
0924 14:28:58.182525 (+131754us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.182645 (+   120us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.262585 (+ 79940us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.462408 (+199823us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.462898 (+   490us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.492683 26391 rpcz_store.cc:276] 0924 14:28:57.265148 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.166037 (+900889us) service_pool.cc:227] Handling call
0924 14:28:58.167867 (+  1830us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.167970 (+   103us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.256421 (+ 88451us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.441149 (+184728us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.460458 (+ 19309us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.494580 26391 tablet_service.cc:1467] Processing CreateTablet for tablet 78da914f30dc4c65be5ff03ec099253b (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260924 14:28:58.500711 26393 tablet_service.cc:1467] Processing CreateTablet for tablet 5e6029c377f94de7b21145dc0ac2f5a7 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260924 14:28:58.501930 26316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa1d9a124214b1089cb23dd48f8d322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.509755 26322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0756687d99e643c094531124c9438376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.504354 26453 tablet_bootstrap.cc:492] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:28:58.515802 26319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f272a708f6ca4bd8bc3a3c4925abad6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.516000 26453 ts_tablet_manager.cc:1397] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.272s	user 0.016s	sys 0.024s
I20260924 14:28:58.505218 26391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78da914f30dc4c65be5ff03ec099253b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.519804 26315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78da914f30dc4c65be5ff03ec099253b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.523815 26393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e6029c377f94de7b21145dc0ac2f5a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.524931 26482 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:58.526317 26170 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 36) took 1303 ms. Trace:
I20260924 14:28:58.528359 26452 ts_tablet_manager.cc:1428] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.373s	user 0.012s	sys 0.004s
I20260924 14:28:58.531961 26456 tablet_bootstrap.cc:654] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:58.534617 26388 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 35) took 1267 ms. Trace:
I20260924 14:28:58.590932 26451 tablet_bootstrap.cc:654] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:58.519356 26453 raft_consensus.cc:348] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.593269 26452 tablet_bootstrap.cc:492] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:58.506454 26318 tablet_service.cc:1467] Processing CreateTablet for tablet 951fc36414c641b7a4bf7b6b3e6dc3f2 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260924 14:28:58.596632 26388 rpcz_store.cc:276] 0924 14:28:57.269209 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.245073 (+975864us) service_pool.cc:227] Handling call
0924 14:28:58.246835 (+  1762us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.246954 (+   119us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.446907 (+199953us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.533824 (+ 86917us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.534530 (+   706us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.613659 26318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 951fc36414c641b7a4bf7b6b3e6dc3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.614218 26388 tablet_service.cc:1467] Processing CreateTablet for tablet 0eec7b6dbbd94479aa179fa23fc1f403 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260924 14:28:58.615518 26388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eec7b6dbbd94479aa179fa23fc1f403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.618323 26396 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 32) took 1352 ms. Trace:
I20260924 14:28:58.602180 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:28:58.612574 26479 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:58.618893 26396 rpcz_store.cc:276] 0924 14:28:57.265843 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.212807 (+946964us) service_pool.cc:227] Handling call
0924 14:28:58.244913 (+ 32106us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.245007 (+    94us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.476279 (+231272us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.617641 (+141362us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.618246 (+   605us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.609669 26453 raft_consensus.cc:374] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:58.627286 26479 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:28:58.642948 26148 leader_election.cc:341] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:58.645015 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:28:58.646595 26395 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 27) took 1382 ms. Trace:
I20260924 14:28:58.647081 26395 rpcz_store.cc:276] 0924 14:28:57.264024 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.132984 (+868960us) service_pool.cc:227] Handling call
0924 14:28:58.134694 (+  1710us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.134798 (+   104us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.354047 (+219249us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.642848 (+288801us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.643579 (+   731us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.649838 26395 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa1d9a124214b1089cb23dd48f8d322 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260924 14:28:58.650982 26320 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 36) took 1425 ms. Trace:
I20260924 14:28:58.651237 26395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa1d9a124214b1089cb23dd48f8d322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.651408 26320 rpcz_store.cc:276] 0924 14:28:57.225153 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.349161 (+1124008us) service_pool.cc:227] Handling call
0924 14:28:58.383051 (+ 33890us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.392020 (+  8969us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.498971 (+106951us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.650439 (+151468us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.650914 (+   475us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.636210 26167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 43) took 1387 ms. Trace:
I20260924 14:28:58.635802 26453 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:28:58.655503 26167 rpcz_store.cc:276] 0924 14:28:57.246500 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.443157 (+1196657us) service_pool.cc:227] Handling call
0924 14:28:58.445718 (+  2561us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.445823 (+   105us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.485715 (+ 39892us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.595317 (+109602us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.633645 (+ 38328us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.656198 26453 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.590549 26482 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:58.586474 26170 rpcz_store.cc:276] 0924 14:28:57.223347 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.079271 (+855924us) service_pool.cc:227] Handling call
0924 14:28:58.219891 (+140620us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.219998 (+   107us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.423391 (+203393us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.525756 (+102365us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.526254 (+   498us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.662333 26390 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 28) took 1398 ms. Trace:
I20260924 14:28:58.662748 26390 rpcz_store.cc:276] 0924 14:28:57.264374 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.138933 (+874559us) service_pool.cc:227] Handling call
0924 14:28:58.146760 (+  7827us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.147386 (+   626us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.513812 (+366426us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.661840 (+148028us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.662274 (+   434us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.665858 26479 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:58.663425 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:28:58.666889 26332 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:28:58.670881 26394 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 34) took 1404 ms. Trace:
I20260924 14:28:58.671482 26396 tablet_service.cc:1467] Processing CreateTablet for tablet f272a708f6ca4bd8bc3a3c4925abad6d (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260924 14:28:58.663322 26317 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 32) took 1446 ms. Trace:
I20260924 14:28:58.672329 26322 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 39) took 1435 ms. Trace:
I20260924 14:28:58.672652 26317 rpcz_store.cc:276] 0924 14:28:57.217218 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.172642 (+955424us) service_pool.cc:227] Handling call
0924 14:28:58.207734 (+ 35092us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.207846 (+   112us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.521643 (+313797us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.662798 (+141155us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.663263 (+   465us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.675114 26396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f272a708f6ca4bd8bc3a3c4925abad6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.672782 26322 rpcz_store.cc:276] 0924 14:28:57.237417 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.417262 (+1179845us) service_pool.cc:227] Handling call
0924 14:28:58.480702 (+ 63440us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2fcd21 00007f09be2da1e3 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2 00007f09b788db01
0924 14:28:58.480830 (+   128us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.480929 (+    99us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.534707 (+ 53778us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.671817 (+137110us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.672268 (+   451us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":140008832}
I20260924 14:28:58.706399 26313 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 38) took 1469 ms. Trace:
I20260924 14:28:58.701722 26164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 37) took 1475 ms. Trace:
I20260924 14:28:58.671341 26394 rpcz_store.cc:276] 0924 14:28:57.266487 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.240729 (+974242us) service_pool.cc:227] Handling call
0924 14:28:58.242427 (+  1698us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.242539 (+   112us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.468664 (+226125us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.670267 (+201603us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.670820 (+   553us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.718665 26164 rpcz_store.cc:276] 0924 14:28:57.226775 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.109145 (+882370us) service_pool.cc:227] Handling call
0924 14:28:58.154069 (+ 44924us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.154172 (+   103us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.470022 (+315850us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.698162 (+228140us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.701658 (+  3496us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:28:58.679245 26149 leader_election.cc:341] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:58.698916 26482 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:28:58.698315 26321 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 42) took 1453 ms. Trace:
I20260924 14:28:58.719918 26149 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:28:58.698621 26452 tablet_bootstrap.cc:654] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:58.682128 26456 tablet_bootstrap.cc:492] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:58.700563 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:28:58.683981 26315 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 37) took 1455 ms. Trace:
I20260924 14:28:58.720238 26321 rpcz_store.cc:276] 0924 14:28:57.245001 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.470280 (+1225279us) service_pool.cc:227] Handling call
0924 14:28:58.478436 (+  8156us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.478569 (+   133us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.644858 (+166289us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.697804 (+ 52946us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.698253 (+   449us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.680626 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:58.692160 26389 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 33) took 1426 ms. Trace:
I20260924 14:28:58.682142 26168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 41) took 1442 ms. Trace:
I20260924 14:28:58.690932 26391 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 37) took 1419 ms. Trace:
I20260924 14:28:58.703576 26166 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 42) took 1459 ms. Trace:
I20260924 14:28:58.735339 26166 rpcz_store.cc:276] 0924 14:28:57.243760 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.377387 (+1133627us) service_pool.cc:227] Handling call
0924 14:28:58.420232 (+ 42845us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.420336 (+   104us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.652377 (+232041us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.695188 (+ 42811us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.703514 (+  8326us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.738605 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:28:58.739567 26406 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:28:58.730186 26391 rpcz_store.cc:276] 0924 14:28:57.271963 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.493613 (+1221650us) service_pool.cc:227] Handling call
0924 14:28:58.495419 (+  1806us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.504835 (+  9416us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.655708 (+150873us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.690392 (+ 34684us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.690853 (+   461us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.738003 26387 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 36) took 1466 ms. Trace:
I20260924 14:28:58.744112 26387 rpcz_store.cc:276] 0924 14:28:57.271399 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.477793 (+1206394us) service_pool.cc:227] Handling call
0924 14:28:58.479682 (+  1889us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.479790 (+   108us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.701839 (+222049us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.737428 (+ 35589us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.737933 (+   505us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.742697 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:28:58.733513 26221 leader_election.cc:341] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:58.749426 26475 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:58.750236 26475 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:28:58.750346 26221 leader_election.cc:341] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:58.750846 26475 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:58.728490 26456 ts_tablet_manager.cc:1397] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.269s	user 0.023s	sys 0.007s
I20260924 14:28:58.751013 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:28:58.752100 26482 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:58.708312 26390 tablet_service.cc:1467] Processing CreateTablet for tablet 5581950b69104df485f842c6b78b1047 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260924 14:28:58.730727 26389 rpcz_store.cc:276] 0924 14:28:57.266158 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.218126 (+951968us) service_pool.cc:227] Handling call
0924 14:28:58.228206 (+ 10080us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.228311 (+   105us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.444239 (+215928us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.691626 (+247387us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.692102 (+   476us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.730985 26168 rpcz_store.cc:276] 0924 14:28:57.239408 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.315912 (+1076504us) service_pool.cc:227] Handling call
0924 14:28:58.318609 (+  2697us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.318708 (+    99us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.512507 (+193799us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.634319 (+121812us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.682080 (+ 47761us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.727129 26315 rpcz_store.cc:276] 0924 14:28:57.228249 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.410928 (+1182679us) service_pool.cc:227] Handling call
0924 14:28:58.472403 (+ 61475us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2fcd21 00007f09be2da1e3 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2 00007f09b788db01
0924 14:28:58.472545 (+   142us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.472641 (+    96us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.597466 (+124825us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.659004 (+ 61538us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.683923 (+ 24919us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":81649536}
I20260924 14:28:58.708592 26319 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 40) took 1470 ms. Trace:
I20260924 14:28:58.721571 26313 rpcz_store.cc:276] 0924 14:28:57.237011 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.413835 (+1176824us) service_pool.cc:227] Handling call
0924 14:28:58.457784 (+ 43949us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.458057 (+   273us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.635472 (+177415us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.705870 (+ 70398us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.706324 (+   454us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":49920}
I20260924 14:28:58.763504 26319 rpcz_store.cc:276] 0924 14:28:57.237811 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.426658 (+1188847us) service_pool.cc:227] Handling call
0924 14:28:58.479776 (+ 53118us) spinlock_profiling.cc:243] Waited 50.7 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2fcd21 00007f09be2da1e3 00007f09be28b869 00007f09b788dc44 00007f09b788dbd2 00007f09b788db01
0924 14:28:58.479984 (+   208us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.480071 (+    87us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.648261 (+168190us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.708081 (+ 59820us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.708532 (+   451us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":116568832}
I20260924 14:28:58.755309 26456 raft_consensus.cc:348] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:58.766299 26456 raft_consensus.cc:374] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:58.766619 26456 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:28:58.767335 26456 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:58.770155 26390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5581950b69104df485f842c6b78b1047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.771711 26173 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 40) took 1536 ms. Trace:
I20260924 14:28:58.772837 26241 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49586 (request call id 41) took 1371 ms. Trace:
I20260924 14:28:58.772131 26173 rpcz_store.cc:276] 0924 14:28:57.235686 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.158787 (+923101us) service_pool.cc:227] Handling call
0924 14:28:58.160575 (+  1788us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.160695 (+   120us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.469206 (+308511us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.771078 (+301872us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.771652 (+   574us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.773229 26241 rpcz_store.cc:276] 0924 14:28:57.401632 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.070830 (+669198us) service_pool.cc:227] Handling call
0924 14:28:58.208838 (+138008us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.208931 (+    93us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.441301 (+232370us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.772221 (+330920us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.772733 (+   512us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.776551 26453 ts_tablet_manager.cc:1428] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.260s	user 0.018s	sys 0.001s
W20260924 14:28:58.780295 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:58.790799 26392 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 31) took 1525 ms. Trace:
I20260924 14:28:58.793241 26388 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 39) took 1519 ms. Trace:
I20260924 14:28:58.802142 26490 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:58.808214 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:28:58.808537 26395 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 41) took 1533 ms. Trace:
I20260924 14:28:58.797778 26394 tablet_service.cc:1467] Processing CreateTablet for tablet 951fc36414c641b7a4bf7b6b3e6dc3f2 (DEFAULT_TABLE table=test-workload [id=759a156a2bae4526a57e566a2f7bf465]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260924 14:28:58.809232 26479 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:58.809070 26395 rpcz_store.cc:276] 0924 14:28:57.275136 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.648883 (+1373747us) service_pool.cc:227] Handling call
0924 14:28:58.650762 (+  1879us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.650849 (+    87us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.734631 (+ 83782us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.806567 (+ 71936us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.808476 (+  1909us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.809782 26479 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:58.810298 26479 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:58.808674 26314 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 34) took 1590 ms. Trace:
I20260924 14:28:58.811345 26394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 951fc36414c641b7a4bf7b6b3e6dc3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:28:58.800848 26392 rpcz_store.cc:276] 0924 14:28:57.265487 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.207857 (+942370us) service_pool.cc:227] Handling call
0924 14:28:58.300889 (+ 93032us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.311559 (+ 10670us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.654969 (+343410us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.790096 (+135127us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.790733 (+   637us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.811669 26314 rpcz_store.cc:276] 0924 14:28:57.218471 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.193891 (+975420us) service_pool.cc:227] Handling call
0924 14:28:58.298054 (+104163us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.298172 (+   118us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.522679 (+224507us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.806569 (+283890us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.808617 (+  2048us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.807571 26475 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.815708 26318 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 43) took 1567 ms. Trace:
I20260924 14:28:58.804692 26388 rpcz_store.cc:276] 0924 14:28:57.273776 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.613281 (+1339505us) service_pool.cc:227] Handling call
0924 14:28:58.615045 (+  1764us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.615140 (+    95us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.682660 (+ 67520us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.792722 (+110062us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.793181 (+   459us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.817476 26393 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 38) took 1544 ms. Trace:
I20260924 14:28:58.817912 26479 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.818507 26316 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50644 (request call id 41) took 1576 ms. Trace:
I20260924 14:28:58.813680 26451 tablet_bootstrap.cc:492] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:58.807686 26453 tablet_bootstrap.cc:492] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
W20260924 14:28:58.807924 26149 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 128722us
I20260924 14:28:58.822899 26452 tablet_bootstrap.cc:492] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:58.823719 26456 ts_tablet_manager.cc:1428] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.072s	user 0.009s	sys 0.000s
I20260924 14:28:58.870644 26452 ts_tablet_manager.cc:1397] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.278s	user 0.017s	sys 0.021s
I20260924 14:28:58.871562 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:58.872399 26182 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:28:58.872633 26456 tablet_bootstrap.cc:492] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:28:58.874178 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:28:58.874750 26182 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:58.874639 26452 raft_consensus.cc:348] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:58.875522 26452 raft_consensus.cc:374] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:58.875841 26452 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:58.870090 26318 rpcz_store.cc:276] 0924 14:28:57.248262 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.505540 (+1257278us) service_pool.cc:227] Handling call
0924 14:28:58.613154 (+107614us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.613257 (+   103us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.705938 (+ 92681us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.815174 (+109236us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.815650 (+   476us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.877954 26452 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:58.880936 26452 ts_tablet_manager.cc:1428] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260924 14:28:58.882318 26452 tablet_bootstrap.cc:492] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:58.825755 26165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50086 (request call id 44) took 1576 ms. Trace:
I20260924 14:28:58.885133 26165 rpcz_store.cc:276] 0924 14:28:57.249299 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.463620 (+1214321us) service_pool.cc:227] Handling call
0924 14:28:58.465775 (+  2155us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.465887 (+   112us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.685634 (+219747us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.825059 (+139425us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.825659 (+   600us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.886533 26182 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:28:58.888651 26452 tablet_bootstrap.cc:654] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:58.888828 26295 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:28:58.889958 26494 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:58.838637 26396 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 40) took 1564 ms. Trace:
I20260924 14:28:58.896907 26390 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 42) took 1621 ms. Trace:
I20260924 14:28:58.825438 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:28:58.896993 26396 rpcz_store.cc:276] 0924 14:28:57.274718 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.640458 (+1365740us) service_pool.cc:227] Handling call
0924 14:28:58.674571 (+ 34113us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.674668 (+    97us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.794537 (+119869us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.831207 (+ 36670us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.838555 (+  7348us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.824381 26475 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:58.897475 26390 rpcz_store.cc:276] 0924 14:28:57.275547 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.664207 (+1388660us) service_pool.cc:227] Handling call
0924 14:28:58.769654 (+105447us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.769757 (+   103us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.845896 (+ 76139us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.883240 (+ 37344us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.896849 (+ 13609us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.904592 26406 raft_consensus.cc:2379] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 1: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:28:58.832726 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:58.908039 26494 raft_consensus.cc:686] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:28:58.908985 26183 raft_consensus.cc:2379] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 1: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:28:58.909353 26494 consensus_queue.cc:237] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.900741 26479 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:58.897187 26393 rpcz_store.cc:276] 0924 14:28:57.273331 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.499804 (+1226473us) service_pool.cc:227] Handling call
0924 14:28:58.511604 (+ 11800us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.511721 (+   117us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.698747 (+187026us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.816838 (+118091us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.817413 (+   575us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.870990 26451 ts_tablet_manager.cc:1397] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.414s	user 0.012s	sys 0.015s
I20260924 14:28:58.902428 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:28:58.903494 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:28:58.898551 26456 tablet_bootstrap.cc:654] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:58.913064 26182 raft_consensus.cc:2379] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 1: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:28:58.914186 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:28:58.915611 26479 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 14:28:58.918059 26316 rpcz_store.cc:276] 0924 14:28:57.241976 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.449152 (+1207176us) service_pool.cc:227] Handling call
0924 14:28:58.500689 (+ 51537us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.501433 (+   744us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.709240 (+207807us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.807087 (+ 97847us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.818423 (+ 11336us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2432}
I20260924 14:28:58.919265 26452 tablet_bootstrap.cc:492] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:58.919961 26452 ts_tablet_manager.cc:1397] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.014s
I20260924 14:28:58.917352 26451 raft_consensus.cc:348] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.922834 26452 raft_consensus.cc:348] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:58.923790 26452 raft_consensus.cc:374] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:58.924131 26452 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:58.925028 26452 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:58.923116 26451 raft_consensus.cc:374] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:58.926841 26451 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:58.927630 26080 catalog_manager.cc:5495] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:58.926544 26453 tablet_bootstrap.cc:654] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:58.935475 26451 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.937033 26452 ts_tablet_manager.cc:1428] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260924 14:28:58.940920 26456 tablet_bootstrap.cc:492] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:58.941524 26456 ts_tablet_manager.cc:1397] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.007s
I20260924 14:28:58.942724 26451 ts_tablet_manager.cc:1428] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.003s
I20260924 14:28:58.943581 26452 tablet_bootstrap.cc:492] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:58.943917 26451 tablet_bootstrap.cc:492] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:58.944702 26456 raft_consensus.cc:348] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.945586 26456 raft_consensus.cc:374] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:58.946117 26456 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:28:58.952790 26451 tablet_bootstrap.cc:654] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:58.954439 26452 tablet_bootstrap.cc:654] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:58.955829 26456 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:58.958755 26394 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46090 (request call id 43) took 1682 ms. Trace:
I20260924 14:28:58.959206 26394 rpcz_store.cc:276] 0924 14:28:57.275958 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:28:58.727640 (+1451682us) service_pool.cc:227] Handling call
0924 14:28:58.810858 (+ 83218us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0924 14:28:58.810968 (+   110us) ts_tablet_manager.cc:676] Creating new metadata...
0924 14:28:58.930012 (+119044us) tablet_metadata.cc:641] Metadata flushed
0924 14:28:58.958146 (+ 28134us) tablet_replica.cc:205] Creating consensus instance
0924 14:28:58.958691 (+   545us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:28:58.962013 26456 ts_tablet_manager.cc:1428] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.005s
I20260924 14:28:58.963160 26456 tablet_bootstrap.cc:492] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:28:58.971151 26452 tablet_bootstrap.cc:492] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:58.972218 26452 ts_tablet_manager.cc:1397] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.006s
I20260924 14:28:58.976642 26452 raft_consensus.cc:348] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:58.977705 26452 raft_consensus.cc:374] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:58.978157 26452 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:58.979858 26453 tablet_bootstrap.cc:492] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:28:58.979640 26452 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:58.979892 26456 tablet_bootstrap.cc:654] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:58.980499 26453 ts_tablet_manager.cc:1397] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.175s	user 0.007s	sys 0.044s
I20260924 14:28:58.995131 26451 tablet_bootstrap.cc:492] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:58.996165 26451 ts_tablet_manager.cc:1397] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.007s
I20260924 14:28:58.998490 26452 ts_tablet_manager.cc:1428] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260924 14:28:58.990566 26453 raft_consensus.cc:348] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:58.999735 26453 raft_consensus.cc:374] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.000192 26452 tablet_bootstrap.cc:492] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:59.000133 26453 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.001793 26453 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.012027 26453 ts_tablet_manager.cc:1428] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260924 14:28:59.011720 26451 raft_consensus.cc:348] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.014266 26479 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.014631 26451 raft_consensus.cc:374] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.015005 26451 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.014775 26479 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.015813 26451 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.019068 26482 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.018926 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:28:59.019539 26479 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:59.019565 26482 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.020781 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:59.022051 26482 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:28:59.022368 26369 leader_election.cc:341] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.019658 26332 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:28:59.024374 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:28:59.026180 26221 leader_election.cc:341] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.026840 26453 tablet_bootstrap.cc:492] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:28:59.028167 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:28:59.029994 26452 tablet_bootstrap.cc:654] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.032797 26479 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.034116 26479 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.037773 26479 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.035027 26456 tablet_bootstrap.cc:492] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:59.040136 26456 ts_tablet_manager.cc:1397] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.077s	user 0.018s	sys 0.014s
I20260924 14:28:59.048113 26456 raft_consensus.cc:348] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.049527 26456 raft_consensus.cc:374] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.050058 26456 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.051131 26456 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.053886 26456 ts_tablet_manager.cc:1428] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.002s
I20260924 14:28:59.055189 26456 tablet_bootstrap.cc:492] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:28:59.033744 26453 tablet_bootstrap.cc:654] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.062034 26456 tablet_bootstrap.cc:654] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.048275 26479 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.066248 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:28:59.067772 26452 tablet_bootstrap.cc:492] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
W20260924 14:28:59.067724 26369 leader_election.cc:341] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.068356 26452 ts_tablet_manager.cc:1397] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.068s	user 0.016s	sys 0.003s
I20260924 14:28:59.068429 26479 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:59.068753 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:28:59.069640 26331 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.071955 26456 tablet_bootstrap.cc:492] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:59.072644 26456 ts_tablet_manager.cc:1397] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.009s
I20260924 14:28:59.075960 26456 raft_consensus.cc:348] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.076968 26456 raft_consensus.cc:374] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.077371 26456 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.077170 26452 raft_consensus.cc:348] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.078116 26452 raft_consensus.cc:374] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.078507 26456 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.080156 26331 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:28:59.081168 26453 tablet_bootstrap.cc:492] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:28:59.081413 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:28:59.081748 26453 ts_tablet_manager.cc:1397] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.064s	user 0.007s	sys 0.005s
I20260924 14:28:59.082659 26479 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:59.083164 26479 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:28:59.078438 26452 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.084084 26479 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.085801 26456 ts_tablet_manager.cc:1428] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260924 14:28:59.085556 26452 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.087033 26456 tablet_bootstrap.cc:492] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:28:59.089531 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:28:59.085068 26453 raft_consensus.cc:348] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.090932 26453 raft_consensus.cc:374] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.091063 26452 ts_tablet_manager.cc:1428] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.002s
I20260924 14:28:59.092139 26492 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:28:59.092471 26224 leader_election.cc:341] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.092995 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:28:59.092764 26492 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.093828 26482 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:59.096449 26452 tablet_bootstrap.cc:492] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:59.096499 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:28:59.098567 26149 leader_election.cc:341] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.100808 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:28:59.103705 26451 ts_tablet_manager.cc:1428] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.107s	user 0.021s	sys 0.022s
I20260924 14:28:59.101811 26258 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:28:59.106632 26451 tablet_bootstrap.cc:492] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:59.107331 26148 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:28:59.108525 26490 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.108937 26490 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.109105 26456 tablet_bootstrap.cc:654] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.109318 26490 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.103739 26452 tablet_bootstrap.cc:654] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.112950 26492 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:28:59.091326 26453 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.118623 26081 catalog_manager.cc:5495] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:59.120438 26453 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.134366 26453 ts_tablet_manager.cc:1428] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
I20260924 14:28:59.121731 26490 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.138024 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:28:59.139689 26149 leader_election.cc:341] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.142081 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:28:59.142772 26258 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.149029 26494 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.149569 26494 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.151518 26494 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:28:59.152422 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:59.154237 26494 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:28:59.154505 26295 leader_election.cc:341] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.154739 26494 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.157295 26494 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:28:59.158384 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:28:59.159173 26406 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:28:59.149086 26451 tablet_bootstrap.cc:654] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.160473 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:28:59.161446 26494 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.161921 26494 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.162321 26494 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.172878 26453 tablet_bootstrap.cc:492] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:28:59.173471 26490 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:28:59.174077 26494 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.174688 26258 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:28:59.179838 26148 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:28:59.179950 26494 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:28:59.182899 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:28:59.183748 26451 tablet_bootstrap.cc:492] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:59.183939 26490 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:59.184438 26451 ts_tablet_manager.cc:1397] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.078s	user 0.012s	sys 0.005s
I20260924 14:28:59.185824 26490 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:28:59.183760 26406 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.189698 26490 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.190300 26452 tablet_bootstrap.cc:492] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:59.190896 26452 ts_tablet_manager.cc:1397] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.095s	user 0.006s	sys 0.040s
I20260924 14:28:59.194931 26452 raft_consensus.cc:348] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.195732 26452 raft_consensus.cc:374] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.195775 26456 tablet_bootstrap.cc:492] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:59.196421 26456 ts_tablet_manager.cc:1397] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.110s	user 0.024s	sys 0.022s
I20260924 14:28:59.188594 26451 raft_consensus.cc:348] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.197937 26451 raft_consensus.cc:374] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.198311 26451 raft_consensus.cc:729] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.197455 26406 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:28:59.199304 26456 raft_consensus.cc:348] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.199818 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:28:59.200126 26456 raft_consensus.cc:374] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.200430 26456 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.201138 26494 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:59.201257 26452 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.201761 26494 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:28:59.202620 26494 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.202804 26452 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.205539 26452 ts_tablet_manager.cc:1428] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260924 14:28:59.206647 26451 consensus_queue.cc:260] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.210454 26451 ts_tablet_manager.cc:1428] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260924 14:28:59.214018 26451 tablet_bootstrap.cc:492] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:59.220285 26452 tablet_bootstrap.cc:492] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:59.221757 26451 tablet_bootstrap.cc:654] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.224172 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:28:59.224978 26257 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:28:59.226132 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:28:59.226806 26452 tablet_bootstrap.cc:654] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.201189 26456 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.226805 26256 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.228965 26456 ts_tablet_manager.cc:1428] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260924 14:28:59.229929 26456 tablet_bootstrap.cc:492] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:28:59.231436 26079 catalog_manager.cc:5495] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:59.233135 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:28:59.233923 26258 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:28:59.235344 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:28:59.235952 26456 tablet_bootstrap.cc:654] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.236701 26495 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.237097 26495 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.237519 26495 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.249933 26456 tablet_bootstrap.cc:492] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:59.250458 26456 ts_tablet_manager.cc:1397] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.005s
I20260924 14:28:59.252933 26456 raft_consensus.cc:348] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.253512 26456 raft_consensus.cc:374] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.253930 26456 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.255640 26451 tablet_bootstrap.cc:492] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:59.256176 26451 ts_tablet_manager.cc:1397] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.011s
I20260924 14:28:59.256080 26456 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.258991 26456 ts_tablet_manager.cc:1428] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260924 14:28:59.258996 26451 raft_consensus.cc:348] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.259881 26451 raft_consensus.cc:374] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.260178 26451 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.260293 26456 tablet_bootstrap.cc:492] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:28:59.260987 26451 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.263486 26451 ts_tablet_manager.cc:1428] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260924 14:28:59.264458 26451 tablet_bootstrap.cc:492] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:59.268306 26456 tablet_bootstrap.cc:654] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.270781 26451 tablet_bootstrap.cc:654] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.282004 26256 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:28:59.290020 26451 tablet_bootstrap.cc:492] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:59.299911 26451 ts_tablet_manager.cc:1397] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.001s
I20260924 14:28:59.289584 26079 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:59.297533 26494 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.303395 26451 raft_consensus.cc:348] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.303660 26494 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.304237 26451 raft_consensus.cc:374] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.304778 26451 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.305088 26456 tablet_bootstrap.cc:492] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:59.305578 26456 ts_tablet_manager.cc:1397] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.028s
I20260924 14:28:59.305989 26494 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:28:59.307236 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:28:59.308327 26456 raft_consensus.cc:348] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.212560 26453 tablet_bootstrap.cc:654] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.291381 26495 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.305641 26451 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.310743 26495 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:28:59.312636 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:28:59.313300 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:28:59.313809 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:28:59.314543 26256 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
W20260924 14:28:59.315029 26295 leader_election.cc:341] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:28:59.308691 26298 leader_election.cc:341] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.313335 26258 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.318742 26452 tablet_bootstrap.cc:492] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:59.319276 26452 ts_tablet_manager.cc:1397] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.099s	user 0.030s	sys 0.009s
I20260924 14:28:59.321430 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:28:59.321954 26482 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.322104 26452 raft_consensus.cc:348] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.322417 26495 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.322849 26452 raft_consensus.cc:374] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.322466 26482 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.323146 26452 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.322956 26495 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.323729 26495 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.323858 26452 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.329670 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:59.330226 26258 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:28:59.331614 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:28:59.332343 26221 leader_election.cc:341] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.332448 26494 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:59.333266 26494 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:28:59.334254 26494 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.336537 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:28:59.334594 26482 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:28:59.343943 26475 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.344489 26475 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.344995 26451 ts_tablet_manager.cc:1428] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260924 14:28:59.346622 26475 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:59.348230 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:28:59.350114 26298 leader_election.cc:341] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.350556 26456 raft_consensus.cc:374] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.350858 26456 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.348232 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:28:59.353595 26295 leader_election.cc:341] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.354053 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:28:59.355299 26451 tablet_bootstrap.cc:492] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:59.355384 26475 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:59.356729 26456 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.357487 26490 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.361107 26490 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.365578 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:28:59.365988 26482 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.367188 26451 tablet_bootstrap.cc:654] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.366659 26482 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.376492 26452 ts_tablet_manager.cc:1428] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.057s	user 0.026s	sys 0.025s
I20260924 14:28:59.377388 26482 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:59.379050 26453 tablet_bootstrap.cc:492] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:28:59.379766 26453 ts_tablet_manager.cc:1397] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.207s	user 0.014s	sys 0.007s
I20260924 14:28:59.379922 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:59.380627 26182 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:28:59.382470 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:28:59.383190 26406 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:28:59.383930 26453 raft_consensus.cc:348] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.384691 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:28:59.385090 26452 tablet_bootstrap.cc:492] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:59.385121 26453 raft_consensus.cc:374] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.385703 26453 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.363238 26490 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:28:59.368686 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:28:59.369695 26331 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:28:59.368685 26495 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.389115 26495 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:28:59.391285 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:28:59.391681 26452 tablet_bootstrap.cc:654] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.392297 26525 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.392691 26525 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.393028 26525 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.393563 26482 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.393940 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:28:59.371061 26456 ts_tablet_manager.cc:1428] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.000s
W20260924 14:28:59.395282 26298 leader_election.cc:341] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.396103 26456 tablet_bootstrap.cc:492] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:28:59.395288 26081 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:59.392846 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:28:59.399035 26258 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:59.367877 26149 leader_election.cc:341] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.393937 26482 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:28:59.399773 26148 leader_election.cc:341] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.399967 26482 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.400686 26148 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:28:59.386122 26494 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.401510 26490 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:59.401978 26494 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.397991 26495 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:59.404069 26494 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:59.405395 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:59.406219 26525 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.408504 26525 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:59.409104 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:28:59.409336 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:28:59.409793 26256 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:28:59.410239 26406 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.409825 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:28:59.411180 26183 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.414734 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:28:59.415688 26494 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.415935 26456 tablet_bootstrap.cc:654] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.416121 26494 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.416489 26494 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.419884 26183 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:28:59.421234 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:28:59.423638 26525 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:59.425298 26494 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.425827 26453 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.426894 26182 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:28:59.428475 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:28:59.429100 26256 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.434232 26453 ts_tablet_manager.cc:1428] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.001s
I20260924 14:28:59.435345 26453 tablet_bootstrap.cc:492] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:28:59.437681 26494 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:59.439235 26479 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.439803 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:28:59.439930 26479 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.440562 26182 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.442217 26495 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:59.442543 26479 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:28:59.443318 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:59.445740 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:28:59.446640 26479 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:28:59.445758 26369 leader_election.cc:341] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.447537 26479 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:28:59.448340 26371 leader_election.cc:341] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.448787 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:28:59.449818 26507 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:59.451256 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:59.453167 26406 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
W20260924 14:28:59.453272 26369 leader_election.cc:341] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.455716 26479 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:28:59.456277 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:28:59.458241 26371 leader_election.cc:341] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.458737 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:28:59.460134 26479 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:28:59.425298 26482 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.465718 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:28:59.467470 26221 leader_election.cc:341] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.474771 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:28:59.475620 26331 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.485674 26331 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:28:59.488492 26256 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:28:59.489919 26456 tablet_bootstrap.cc:492] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:59.490648 26456 ts_tablet_manager.cc:1397] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.095s	user 0.012s	sys 0.000s
I20260924 14:28:59.490588 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:28:59.491744 26475 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:59.492229 26475 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:28:59.488503 26258 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:28:59.493100 26475 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.494091 26456 raft_consensus.cc:348] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.494820 26456 raft_consensus.cc:374] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.495093 26456 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.489828 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:28:59.498018 26456 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.500205 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:28:59.501156 26479 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.501804 26494 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:59.502789 26494 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:28:59.503746 26494 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.485757 26525 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:28:59.486295 26482 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:28:59.534533 26531 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:59.535910 26525 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.561106 26451 tablet_bootstrap.cc:492] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:59.561910 26451 ts_tablet_manager.cc:1397] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.207s	user 0.037s	sys 0.053s
I20260924 14:28:59.501673 26479 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.567999 26456 ts_tablet_manager.cc:1428] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.077s	user 0.011s	sys 0.010s
I20260924 14:28:59.569125 26456 tablet_bootstrap.cc:492] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
W20260924 14:28:59.569645 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:59.566515 26451 raft_consensus.cc:348] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.570320 26451 raft_consensus.cc:374] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.571717 26451 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.571511 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:28:59.572479 26258 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:28:59.572820 26182 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:28:59.573302 26451 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.574095 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:28:59.575139 26507 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.575577 26507 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.575959 26507 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.578933 26453 tablet_bootstrap.cc:654] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.580482 26531 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:28:59.581390 26531 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.584283 26451 ts_tablet_manager.cc:1428] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
W20260924 14:28:59.584770 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 84689.7us
I20260924 14:28:59.585585 26451 tablet_bootstrap.cc:492] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:59.586539 26507 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.587951 26479 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:28:59.588575 26452 tablet_bootstrap.cc:492] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:59.589119 26452 ts_tablet_manager.cc:1397] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.204s	user 0.050s	sys 0.042s
I20260924 14:28:59.600028 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:28:59.600778 26258 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.604214 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:28:59.604966 26331 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:28:59.610886 26451 tablet_bootstrap.cc:654] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.612041 26081 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:59.612968 26258 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:28:59.614511 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:28:59.618577 26479 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:59.620154 26451 tablet_bootstrap.cc:492] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:59.620644 26451 ts_tablet_manager.cc:1397] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.001s
I20260924 14:28:59.620294 26479 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:28:59.623131 26479 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.624553 26452 raft_consensus.cc:348] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.627164 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:28:59.633652 26453 tablet_bootstrap.cc:492] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:28:59.634011 26452 raft_consensus.cc:374] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.634716 26475 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:59.635138 26452 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.635982 26078 catalog_manager.cc:5495] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:59.640883 26494 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:59.628473 26451 raft_consensus.cc:348] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.634102 26331 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.650857 26452 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.653230 26452 ts_tablet_manager.cc:1428] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.000s
I20260924 14:28:59.655207 26456 tablet_bootstrap.cc:654] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.658619 26452 tablet_bootstrap.cc:492] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:59.634296 26453 ts_tablet_manager.cc:1397] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.199s	user 0.015s	sys 0.011s
I20260924 14:28:59.628404 26507 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:28:59.630023 26544 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260924 14:28:59.669698 26453 raft_consensus.cc:348] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.670497 26453 raft_consensus.cc:374] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.670789 26453 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.671573 26451 raft_consensus.cc:374] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.671880 26451 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.672487 26451 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.675103 26451 ts_tablet_manager.cc:1428] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.002s
I20260924 14:28:59.676054 26451 tablet_bootstrap.cc:492] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:59.678332 26452 tablet_bootstrap.cc:654] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.666180 26078 catalog_manager.cc:5495] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:28:59.685561 26081 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:59.687945 26456 tablet_bootstrap.cc:492] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:59.679750 26453 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.688459 26456 ts_tablet_manager.cc:1397] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.120s	user 0.014s	sys 0.009s
I20260924 14:28:59.692170 26451 tablet_bootstrap.cc:654] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.695760 26453 ts_tablet_manager.cc:1428] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260924 14:28:59.696892 26453 tablet_bootstrap.cc:492] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:28:59.689751 26331 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:28:59.709394 26453 tablet_bootstrap.cc:654] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.721315 26452 tablet_bootstrap.cc:492] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:59.722091 26452 ts_tablet_manager.cc:1397] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.064s	user 0.016s	sys 0.000s
I20260924 14:28:59.724779 26453 tablet_bootstrap.cc:492] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:28:59.725435 26453 ts_tablet_manager.cc:1397] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.029s	user 0.005s	sys 0.007s
I20260924 14:28:59.726701 26452 raft_consensus.cc:348] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.727860 26452 raft_consensus.cc:374] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.728291 26452 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.729386 26452 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.736284 26452 ts_tablet_manager.cc:1428] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260924 14:28:59.737485 26452 tablet_bootstrap.cc:492] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:59.737655 26453 raft_consensus.cc:348] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.738647 26453 raft_consensus.cc:374] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.738986 26453 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.739861 26453 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.744066 26452 tablet_bootstrap.cc:654] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.747399 26453 ts_tablet_manager.cc:1428] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260924 14:28:59.748595 26453 tablet_bootstrap.cc:492] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:28:59.753255 26451 tablet_bootstrap.cc:492] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:59.753803 26451 ts_tablet_manager.cc:1397] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.078s	user 0.014s	sys 0.016s
I20260924 14:28:59.756126 26456 raft_consensus.cc:348] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.757165 26456 raft_consensus.cc:374] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.757507 26456 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.758319 26456 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.769742 26451 raft_consensus.cc:348] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.770601 26451 raft_consensus.cc:374] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.770881 26451 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.771822 26451 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.773871 26451 ts_tablet_manager.cc:1428] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260924 14:28:59.775319 26451 tablet_bootstrap.cc:492] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:59.779668 26452 tablet_bootstrap.cc:492] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:59.780164 26452 ts_tablet_manager.cc:1397] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.043s	user 0.029s	sys 0.007s
I20260924 14:28:59.782881 26451 tablet_bootstrap.cc:654] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.782701 26452 raft_consensus.cc:348] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:28:59.785656 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:28:59.787711 26452 raft_consensus.cc:374] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.788067 26452 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.786185 26453 tablet_bootstrap.cc:654] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.790964 26456 ts_tablet_manager.cc:1428] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.102s	user 0.006s	sys 0.002s
I20260924 14:28:59.788627 26452 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.786953 26078 catalog_manager.cc:5495] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:59.793835 26452 ts_tablet_manager.cc:1428] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260924 14:28:59.795109 26452 tablet_bootstrap.cc:492] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:59.800151 26479 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.800714 26452 tablet_bootstrap.cc:654] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.800792 26479 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.806023 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:28:59.806555 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:59.806957 26331 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
W20260924 14:28:59.810529 26149 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 -> Peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Couldn't send request to peer 45ab97b763804767846e9d27a01b561f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:28:59.811286 26544 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.811934 26544 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.815181 26453 tablet_bootstrap.cc:492] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:28:59.815706 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:28:59.816054 26369 leader_election.cc:341] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.821080 26256 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:28:59.803191 26479 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:59.821784 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:28:59.815424 26544 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:28:59.822752 26544 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.823170 26544 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.823567 26544 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.823704 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:28:59.829035 26452 tablet_bootstrap.cc:492] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:28:59.829602 26452 ts_tablet_manager.cc:1397] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.013s
I20260924 14:28:59.831619 26490 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:59.832392 26452 raft_consensus.cc:348] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.833151 26452 raft_consensus.cc:374] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.833423 26452 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.833528 26456 tablet_bootstrap.cc:492] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:28:59.834497 26452 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.836390 26475 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:59.838827 26451 tablet_bootstrap.cc:492] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:28:59.839329 26451 ts_tablet_manager.cc:1397] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.010s
I20260924 14:28:59.841431 26479 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.841959 26479 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.842387 26479 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.842537 26544 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.845393 26452 ts_tablet_manager.cc:1428] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260924 14:28:59.845595 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:28:59.846261 26331 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.846611 26451 raft_consensus.cc:348] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.847316 26451 raft_consensus.cc:374] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.847601 26451 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.848234 26451 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.852551 26456 tablet_bootstrap.cc:654] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.817091 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:28:59.856806 26181 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:28:59.816128 26453 ts_tablet_manager.cc:1397] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.068s	user 0.007s	sys 0.009s
I20260924 14:28:59.858935 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:28:59.866734 26453 raft_consensus.cc:348] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.871197 26453 raft_consensus.cc:374] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:28:59.863126 26525 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.871654 26453 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:28:59.872237 26525 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.873695 26331 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:28:59.859426 26544 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:59.876225 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:28:59.876962 26332 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:28:59.874569 26453 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:28:59.869647 26451 ts_tablet_manager.cc:1428] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260924 14:28:59.870781 26452 tablet_bootstrap.cc:492] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:28:59.878587 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:28:59.879618 26531 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:28:59.879978 26531 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:28:59.880311 26531 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:28:59.867208 26369 leader_election.cc:341] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.885259 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:28:59.885824 26452 tablet_bootstrap.cc:654] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.886850 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:28:59.887621 26181 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.888474 26451 tablet_bootstrap.cc:492] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:28:59.888609 26544 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:28:59.889323 26544 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:28:59.890209 26544 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.874485 26525 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:28:59.894560 26224 leader_election.cc:341] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.874931 26479 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:28:59.899891 26475 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:28:59.901227 26451 tablet_bootstrap.cc:654] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.902971 26531 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:28:59.924957 26453 ts_tablet_manager.cc:1428] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.061s	user 0.010s	sys 0.022s
I20260924 14:28:59.926000 26453 tablet_bootstrap.cc:492] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:28:59.927733 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:28:59.937055 26475 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:28:59.928640 26181 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.943274 26479 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:28:59.952209 26525 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260924 14:28:59.974957 26453 tablet_bootstrap.cc:654] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:28:59.952231 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:28:59.976130 26258 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:28:59.947135 26456 tablet_bootstrap.cc:492] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:28:59.983193 26456 ts_tablet_manager.cc:1397] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.150s	user 0.013s	sys 0.014s
I20260924 14:28:59.964344 26081 catalog_manager.cc:5495] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:28:59.958847 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:28:59.957722 26531 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:28:59.973559 26181 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:28:59.973273 26562 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.963567 26479 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:28:59.999750 26479 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:28:59.999851 26495 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:00.001696 26479 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:28:59.959192 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:00.002400 26332 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.001719 26258 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:00.004483 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:00.005391 26479 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:00.006049 26479 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:00.006896 26479 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:00.007773 26224 leader_election.cc:341] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:28:59.999015 26562 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.009958 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:00.014310 26562 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:00.017763 26456 raft_consensus.cc:348] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.018532 26456 raft_consensus.cc:374] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.018782 26456 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.019358 26456 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.015249 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:00.027087 26456 ts_tablet_manager.cc:1428] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260924 14:29:00.036374 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:00.036420 26475 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260924 14:29:00.041882 26456 tablet_bootstrap.cc:492] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
W20260924 14:29:00.025012 26369 leader_election.cc:341] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:00.044525 26369 leader_election.cc:341] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.045217 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:00.047679 26507 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:00.049682 26451 tablet_bootstrap.cc:492] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:00.050371 26451 ts_tablet_manager.cc:1397] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.162s	user 0.008s	sys 0.011s
I20260924 14:29:00.028738 26257 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:00.037657 26181 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.055364 26525 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:00.055258 26148 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:00.060587 26562 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:00.061025 26562 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:00.061380 26562 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.068195 26451 raft_consensus.cc:348] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.073782 26451 raft_consensus.cc:374] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.074132 26451 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.074990 26451 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.097666 26451 ts_tablet_manager.cc:1428] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260924 14:29:00.101346 26456 tablet_bootstrap.cc:654] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.107121 26452 tablet_bootstrap.cc:492] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:00.107959 26452 ts_tablet_manager.cc:1397] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.237s	user 0.027s	sys 0.008s
I20260924 14:29:00.110944 26451 tablet_bootstrap.cc:492] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:00.133558 26544 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:00.141618 26406 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.167915 26451 tablet_bootstrap.cc:654] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.169930 26544 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.194725 26332 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:00.194938 26452 raft_consensus.cc:348] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.196118 26452 raft_consensus.cc:374] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.196380 26452 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.197067 26452 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.197847 26479 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.198320 26479 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:00.203059 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:00.204419 26475 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:00.191841 26562 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.210604 26562 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:00.187839 26544 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.213416 26544 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:00.214529 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:00.214769 26479 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:00.216601 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:00.216862 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:00.218528 26369 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:00.227133 26482 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.224423 26479 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:00.227754 26482 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.229816 26482 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:00.231282 26452 ts_tablet_manager.cc:1428] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.123s	user 0.006s	sys 0.001s
I20260924 14:29:00.231707 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:00.232206 26536 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:00.232969 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:00.236245 26453 tablet_bootstrap.cc:492] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:00.236871 26453 ts_tablet_manager.cc:1397] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.311s	user 0.038s	sys 0.017s
I20260924 14:29:00.235891 26256 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.240475 26536 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:00.250641 26452 tablet_bootstrap.cc:492] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:00.233383 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:00.262508 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:00.263267 26405 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:00.264168 26536 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:00.273149 26482 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.273736 26482 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.275036 26456 tablet_bootstrap.cc:492] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:00.275556 26456 ts_tablet_manager.cc:1397] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.234s	user 0.015s	sys 0.004s
I20260924 14:29:00.281128 26562 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.281905 26562 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.282574 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:00.284489 26562 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:00.286935 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:00.298723 26081 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:00.308892 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:00.309124 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:00.292833 26531 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:00.294368 26482 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:00.354812 26541 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:00.358566 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:00.358566 26256 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:00.361325 26148 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:00.359838 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:00.361725 26456 raft_consensus.cc:348] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.362447 26456 raft_consensus.cc:374] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.362278 26562 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:00.363026 26562 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:00.363345 26146 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:00.363978 26562 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.323048 26531 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:00.367729 26564 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:00.368049 26531 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.367815 26456 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.371836 26482 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:00.371426 26456 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.374962 26456 ts_tablet_manager.cc:1428] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.099s	user 0.007s	sys 0.001s
I20260924 14:29:00.376451 26456 tablet_bootstrap.cc:492] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:00.383574 26585 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:00.387801 26451 tablet_bootstrap.cc:492] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:00.388449 26451 ts_tablet_manager.cc:1397] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.278s	user 0.036s	sys 0.027s
I20260924 14:29:00.389087 26456 tablet_bootstrap.cc:654] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.393959 26453 raft_consensus.cc:348] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.394757 26453 raft_consensus.cc:374] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.395043 26453 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.391062 26451 raft_consensus.cc:348] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:00.395710 26332 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:00.402428 26452 tablet_bootstrap.cc:654] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.398861 26453 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.400071 26451 raft_consensus.cc:374] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.405635 26451 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.406787 26451 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:00.417021 26453 ts_tablet_manager.cc:1428] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.180s	user 0.007s	sys 0.000s
I20260924 14:29:00.417518 26451 ts_tablet_manager.cc:1428] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260924 14:29:00.418931 26451 tablet_bootstrap.cc:492] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:00.420271 26453 tablet_bootstrap.cc:492] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:00.416303 26531 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.404944 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:00.404999 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:00.407294 26221 leader_election.cc:341] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.425026 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:00.425118 26221 leader_election.cc:341] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.422552 26531 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:00.413822 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 130107us
I20260924 14:29:00.433368 26456 tablet_bootstrap.cc:492] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:00.433795 26456 ts_tablet_manager.cc:1397] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.000s
I20260924 14:29:00.425942 26544 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:00.427696 26405 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.436269 26456 raft_consensus.cc:348] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.436964 26456 raft_consensus.cc:374] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.437222 26456 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.437865 26456 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.442679 26453 tablet_bootstrap.cc:654] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.442633 26079 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:00.444262 26451 tablet_bootstrap.cc:654] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.427026 26331 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.449985 26456 ts_tablet_manager.cc:1428] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260924 14:29:00.455703 26456 tablet_bootstrap.cc:492] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:00.464048 26544 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:00.464428 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:00.475816 26544 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:00.477174 26452 tablet_bootstrap.cc:492] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:00.477830 26452 ts_tablet_manager.cc:1397] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.227s	user 0.009s	sys 0.004s
I20260924 14:29:00.481959 26586 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.482788 26586 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.491331 26586 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:00.494971 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:00.496049 26221 leader_election.cc:341] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:00.496762 26295 leader_election.cc:341] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.489722 26081 catalog_manager.cc:5495] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:00.506732 26452 raft_consensus.cc:348] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:00.507908 26452 raft_consensus.cc:374] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.508249 26452 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.515385 26453 tablet_bootstrap.cc:492] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:00.514760 26452 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:00.515944 26453 ts_tablet_manager.cc:1397] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.096s	user 0.012s	sys 0.001s
I20260924 14:29:00.521114 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:00.521719 26452 ts_tablet_manager.cc:1428] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260924 14:29:00.522984 26452 tablet_bootstrap.cc:492] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:00.529618 26531 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.530157 26531 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:00.532267 26531 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:00.532366 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:00.534116 26221 leader_election.cc:341] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.534584 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:00.534498 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:00.536320 26224 leader_election.cc:341] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:00.537803 26298 leader_election.cc:341] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.538558 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:00.540252 26451 tablet_bootstrap.cc:492] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:00.546274 26451 ts_tablet_manager.cc:1397] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.128s	user 0.019s	sys 0.003s
I20260924 14:29:00.536499 26453 raft_consensus.cc:348] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.539073 26535 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.547188 26453 raft_consensus.cc:374] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.549870 26453 raft_consensus.cc:729] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.550820 26453 consensus_queue.cc:260] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.557816 26451 raft_consensus.cc:348] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.558874 26456 tablet_bootstrap.cc:654] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.558923 26451 raft_consensus.cc:374] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.560487 26451 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.545209 26531 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:00.561388 26451 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.562070 26453 ts_tablet_manager.cc:1428] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.001s
I20260924 14:29:00.566515 26406 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.577754 26451 ts_tablet_manager.cc:1428] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.004s
I20260924 14:29:00.579164 26451 tablet_bootstrap.cc:492] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:00.562777 26535 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.585196 26535 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:00.546370 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:00.586658 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:00.586907 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 49146.3us
W20260924 14:29:00.588651 26221 leader_election.cc:341] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.589078 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:00.589408 26586 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:00.590040 26536 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:00.592226 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:00.593554 26295 leader_election.cc:341] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.602355 26453 tablet_bootstrap.cc:492] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:00.606738 26452 tablet_bootstrap.cc:654] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.601033 26479 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260924 14:29:00.610844 26453 tablet_bootstrap.cc:654] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.614645 26456 tablet_bootstrap.cc:492] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:00.615201 26456 ts_tablet_manager.cc:1397] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.163s	user 0.013s	sys 0.015s
I20260924 14:29:00.618888 26456 raft_consensus.cc:348] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.619685 26456 raft_consensus.cc:374] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.619935 26456 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.620529 26456 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.622517 26456 ts_tablet_manager.cc:1428] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260924 14:29:00.623459 26456 tablet_bootstrap.cc:492] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:00.640086 26451 tablet_bootstrap.cc:654] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.640450 26456 tablet_bootstrap.cc:654] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
W20260924 14:29:00.631430 26298 leader_election.cc:341] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.653225 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:00.660499 26494 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:00.660673 26405 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:00.681330 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:00.683709 26531 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:00.684270 26531 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:00.692181 26531 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.693428 26453 tablet_bootstrap.cc:492] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:00.693964 26453 ts_tablet_manager.cc:1397] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.008s
I20260924 14:29:00.700176 26453 raft_consensus.cc:348] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.702111 26453 raft_consensus.cc:374] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.702834 26453 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.704039 26453 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.680446 26331 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:00.708496 26451 tablet_bootstrap.cc:492] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:00.709523 26451 ts_tablet_manager.cc:1397] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.131s	user 0.016s	sys 0.000s
I20260924 14:29:00.713052 26451 raft_consensus.cc:348] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.713721 26451 raft_consensus.cc:374] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.713994 26451 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.714598 26451 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.716646 26451 ts_tablet_manager.cc:1428] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:29:00.717536 26451 tablet_bootstrap.cc:492] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:00.720301 26453 ts_tablet_manager.cc:1428] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260924 14:29:00.722448 26586 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.729032 26586 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.729605 26081 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:00.731070 26586 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:00.723873 26451 tablet_bootstrap.cc:654] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.733053 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:00.734553 26295 leader_election.cc:341] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.724697 26452 tablet_bootstrap.cc:492] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:00.735424 26452 ts_tablet_manager.cc:1397] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.213s	user 0.014s	sys 0.016s
I20260924 14:29:00.737982 26452 raft_consensus.cc:348] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.738637 26452 raft_consensus.cc:374] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.738915 26452 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.739741 26452 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:00.741669 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:00.742830 26257 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:00.743901 26541 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.744091 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:00.744431 26541 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:00.745891 26586 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:00.747016 26541 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:00.747792 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:00.750530 26371 leader_election.cc:341] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.751340 26586 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:00.752019 26586 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.757210 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:00.778015 26224 consensus_peers.cc:487] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e -> Peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Couldn't send request to peer 45ab97b763804767846e9d27a01b561f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:00.786135 26369 leader_election.cc:341] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.786607 26369 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:00.787503 26541 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:00.789017 26541 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.789476 26541 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:00.791708 26541 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:00.794138 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:00.795738 26369 leader_election.cc:341] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.795637 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:00.798237 26371 leader_election.cc:341] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:00.798966 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:00.800923 26541 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:00.780328 26452 ts_tablet_manager.cc:1428] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.045s	user 0.002s	sys 0.004s
W20260924 14:29:00.806874 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:29:00.802987 26452 tablet_bootstrap.cc:492] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:00.811494 26453 tablet_bootstrap.cc:492] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:00.830031 26456 tablet_bootstrap.cc:492] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:00.830588 26456 ts_tablet_manager.cc:1397] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.207s	user 0.044s	sys 0.050s
I20260924 14:29:00.844553 26456 raft_consensus.cc:348] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:00.845387 26456 raft_consensus.cc:374] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:00.845683 26456 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:00.846410 26456 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:00.848068 26453 tablet_bootstrap.cc:654] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.851503 26562 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.852002 26562 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.855115 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:00.856513 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:00.858132 26149 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:00.858975 26564 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:00.860095 26562 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:00.862105 26456 ts_tablet_manager.cc:1428] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.005s
I20260924 14:29:00.866779 26562 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:00.870324 26456 tablet_bootstrap.cc:492] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:00.875658 26181 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:00.880340 26456 tablet_bootstrap.cc:654] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.890138 26452 tablet_bootstrap.cc:654] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:00.908762 26562 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:00.909384 26562 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:00.913094 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:00.915905 26531 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
I20260924 14:29:00.913816 26562 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:00.909404 26564 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.068s
I20260924 14:29:00.914072 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:00.912014 26586 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.002316 26586 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:01.061689 26451 tablet_bootstrap.cc:492] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:01.063298 26451 ts_tablet_manager.cc:1397] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.346s	user 0.049s	sys 0.084s
I20260924 14:29:01.068625 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:01.070232 26456 tablet_bootstrap.cc:492] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:01.070452 26149 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:01.071484 26456 ts_tablet_manager.cc:1397] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.201s	user 0.009s	sys 0.012s
I20260924 14:29:01.089773 26564 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.102819 26494 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.070319 26256 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.103888 26564 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.110208 26182 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:01.110981 26611 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260924 14:29:01.122298 26451 raft_consensus.cc:348] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:01.147784 26451 raft_consensus.cc:374] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.148247 26451 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.149264 26451 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:01.153928 26564 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:01.155634 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:01.157418 26451 ts_tablet_manager.cc:1428] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.093s	user 0.009s	sys 0.002s
W20260924 14:29:01.157692 26148 leader_election.cc:341] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.159195 26451 tablet_bootstrap.cc:492] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:01.123001 26456 raft_consensus.cc:348] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.125958 26453 tablet_bootstrap.cc:492] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:01.164343 26453 ts_tablet_manager.cc:1397] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.356s	user 0.010s	sys 0.004s
I20260924 14:29:01.125733 26531 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:01.104017 26494 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:01.165036 26456 raft_consensus.cc:374] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.165941 26456 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.172552 26453 raft_consensus.cc:348] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.175462 26453 raft_consensus.cc:374] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.175770 26453 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.175843 26494 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:01.178692 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:01.180877 26298 leader_election.cc:341] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.183054 26456 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.184039 26453 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.187546 26479 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.188416 26453 ts_tablet_manager.cc:1428] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.005s
I20260924 14:29:01.188278 26479 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.190212 26453 tablet_bootstrap.cc:492] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:01.191054 26479 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:01.192590 26562 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:01.192443 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:01.196152 26452 tablet_bootstrap.cc:492] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:01.196040 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:01.197139 26331 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:29:01.197944 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:01.198839 26254 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
W20260924 14:29:01.202127 26369 leader_election.cc:341] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.204691 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:01.205636 26452 ts_tablet_manager.cc:1397] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.403s	user 0.013s	sys 0.002s
I20260924 14:29:01.209568 26479 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:01.224673 26479 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:01.230659 26479 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.234393 26256 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:01.248914 26456 ts_tablet_manager.cc:1428] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.177s	user 0.010s	sys 0.000s
I20260924 14:29:01.253073 26451 tablet_bootstrap.cc:654] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.260082 26405 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.261189 26456 tablet_bootstrap.cc:492] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:01.258836 26452 raft_consensus.cc:348] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.270654 26452 raft_consensus.cc:374] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.271908 26452 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.277729 26453 tablet_bootstrap.cc:654] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
W20260924 14:29:01.286039 26221 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:01.288301 26562 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.289007 26562 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.292961 26452 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.295965 26479 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.297034 26562 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:01.299710 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:01.300046 26479 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:01.301004 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:01.301573 26146 leader_election.cc:341] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:01.302060 26149 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 250955us
W20260924 14:29:01.302395 26369 leader_election.cc:341] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.304949 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:01.305894 26403 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:01.306819 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:01.307260 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:01.307617 26536 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.308204 26404 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:01.308241 26536 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.320622 26536 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:01.321797 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:01.321887 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:01.325423 26221 leader_election.cc:341] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:01.326916 26221 leader_election.cc:341] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.327540 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:01.329172 26536 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:01.328997 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:01.330862 26535 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:01.331310 26535 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:01.331837 26535 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.358094 26451 tablet_bootstrap.cc:492] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
W20260924 14:29:01.345148 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 108116us
I20260924 14:29:01.364290 26451 ts_tablet_manager.cc:1397] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.206s	user 0.027s	sys 0.013s
I20260924 14:29:01.366032 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:01.367208 26494 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:01.367719 26494 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:01.369119 26451 raft_consensus.cc:348] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.370088 26451 raft_consensus.cc:374] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.370465 26451 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.369058 26494 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.366086 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:01.371814 26331 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.349892 26535 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:01.373879 26535 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:01.375078 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:01.376878 26298 leader_election.cc:341] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.377275 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:01.377971 26257 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.361893 26453 tablet_bootstrap.cc:492] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:01.380183 26453 ts_tablet_manager.cc:1397] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.190s	user 0.012s	sys 0.006s
I20260924 14:29:01.382148 26331 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:01.383402 26453 raft_consensus.cc:348] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.384343 26453 raft_consensus.cc:374] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.384652 26453 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.384686 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:01.385689 26453 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:01.343331 26622 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260924 14:29:01.348461 26452 ts_tablet_manager.cc:1428] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.142s	user 0.008s	sys 0.001s
I20260924 14:29:01.347877 26149 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:01.391141 26621 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:01.377694 26451 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.391573 26621 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:01.391625 26544 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:01.392066 26621 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.392215 26544 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:01.393466 26453 ts_tablet_manager.cc:1428] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260924 14:29:01.393374 26544 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.394724 26453 tablet_bootstrap.cc:492] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:01.393530 26451 ts_tablet_manager.cc:1428] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.003s
W20260924 14:29:01.399998 26149 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 52269.2us
I20260924 14:29:01.400771 26451 tablet_bootstrap.cc:492] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:01.401147 26149 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:01.402285 26562 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:01.402791 26562 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:01.403159 26562 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:01.405943 26295 leader_election.cc:341] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.411072 26453 tablet_bootstrap.cc:654] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.412561 26452 tablet_bootstrap.cc:492] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:01.335417 26456 tablet_bootstrap.cc:654] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.415740 26451 tablet_bootstrap.cc:654] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.426672 26452 tablet_bootstrap.cc:654] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.431856 26453 tablet_bootstrap.cc:492] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:01.436906 26453 ts_tablet_manager.cc:1397] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.003s
I20260924 14:29:01.439690 26453 raft_consensus.cc:348] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.440440 26453 raft_consensus.cc:374] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.440738 26453 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.440181 26079 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:01.418605 26621 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.441545 26562 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.442942 26453 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.444494 26621 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:01.444411 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:01.446110 26146 leader_election.cc:341] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.447201 26257 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:01.447914 26453 ts_tablet_manager.cc:1428] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260924 14:29:01.449337 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:01.450155 26453 tablet_bootstrap.cc:492] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:01.450699 26494 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:01.451329 26494 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:01.452404 26494 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:01.453995 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:01.455325 26148 leader_election.cc:341] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.457306 26562 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:01.458024 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:01.465251 26405 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.474180 26456 tablet_bootstrap.cc:492] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:01.474867 26456 ts_tablet_manager.cc:1397] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.214s	user 0.011s	sys 0.002s
I20260924 14:29:01.475293 26405 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:01.479672 26149 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:01.481683 26621 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:01.482347 26621 raft_consensus.cc:686] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:01.483397 26621 consensus_queue.cc:237] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.461334 26453 tablet_bootstrap.cc:654] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.463433 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:01.486361 26406 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.487278 26451 tablet_bootstrap.cc:492] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:01.487973 26451 ts_tablet_manager.cc:1397] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.003s
I20260924 14:29:01.491277 26451 raft_consensus.cc:348] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:01.492159 26451 raft_consensus.cc:374] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.492447 26451 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.494065 26451 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:01.505427 26451 ts_tablet_manager.cc:1428] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260924 14:29:01.514640 26451 tablet_bootstrap.cc:492] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:01.515889 26406 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:01.517360 26149 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:01.518949 26452 tablet_bootstrap.cc:492] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:01.518791 26621 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:01.519677 26452 ts_tablet_manager.cc:1397] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.107s	user 0.021s	sys 0.006s
I20260924 14:29:01.520556 26621 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:01.521917 26621 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.523124 26452 raft_consensus.cc:348] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.524231 26452 raft_consensus.cc:374] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.524621 26452 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.525651 26451 tablet_bootstrap.cc:654] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.525511 26452 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.529526 26456 raft_consensus.cc:348] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.546226 26456 raft_consensus.cc:374] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.546644 26456 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.548722 26456 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.564550 26544 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.565205 26544 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:01.557226 26080 catalog_manager.cc:5495] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:01.569304 26544 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:01.571332 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:01.578159 26256 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 1: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:01.577950 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:01.579666 26452 ts_tablet_manager.cc:1428] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.002s
I20260924 14:29:01.580237 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:01.583101 26453 tablet_bootstrap.cc:492] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:01.583407 26544 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.583712 26453 ts_tablet_manager.cc:1397] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.134s	user 0.028s	sys 0.034s
I20260924 14:29:01.584127 26452 tablet_bootstrap.cc:492] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:01.586709 26453 raft_consensus.cc:348] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.591770 26453 raft_consensus.cc:374] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.592100 26453 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.592895 26453 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.595120 26453 ts_tablet_manager.cc:1428] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260924 14:29:01.590780 26544 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:01.596875 26453 tablet_bootstrap.cc:492] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:01.598176 26621 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.598837 26621 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.580559 26456 ts_tablet_manager.cc:1428] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.105s	user 0.012s	sys 0.000s
I20260924 14:29:01.603205 26452 tablet_bootstrap.cc:654] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.603858 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:01.604264 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:01.605105 26621 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:01.605258 26331 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:01.605710 26456 tablet_bootstrap.cc:492] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
W20260924 14:29:01.605566 26148 leader_election.cc:341] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.610672 26146 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:01.607467 26453 tablet_bootstrap.cc:654] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.607326 26078 catalog_manager.cc:5495] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:01.618762 26451 tablet_bootstrap.cc:492] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:01.619367 26451 ts_tablet_manager.cc:1397] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.004s
I20260924 14:29:01.609161 26079 catalog_manager.cc:5495] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:01.621670 26621 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:01.631503 26451 raft_consensus.cc:348] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.632352 26451 raft_consensus.cc:374] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.632797 26451 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.637436 26451 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.628194 26621 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:01.640856 26452 tablet_bootstrap.cc:492] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:01.641650 26494 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.642223 26494 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.643327 26621 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.644548 26456 tablet_bootstrap.cc:654] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.647425 26451 ts_tablet_manager.cc:1428] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260924 14:29:01.641690 26452 ts_tablet_manager.cc:1397] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.003s
I20260924 14:29:01.645159 26494 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:01.650508 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:01.651248 26182 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:01.655530 26078 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:01.647831 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:01.658968 26406 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:01.658162 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:01.660665 26453 tablet_bootstrap.cc:492] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:01.661338 26453 ts_tablet_manager.cc:1397] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.002s
I20260924 14:29:01.661741 26494 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:01.662201 26494 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:01.662559 26494 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.671626 26453 raft_consensus.cc:348] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.672506 26453 raft_consensus.cc:374] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.672785 26453 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.673627 26453 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.680732 26456 tablet_bootstrap.cc:492] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:01.681262 26456 ts_tablet_manager.cc:1397] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.077s	user 0.020s	sys 0.016s
I20260924 14:29:01.682126 26453 ts_tablet_manager.cc:1428] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260924 14:29:01.683290 26453 tablet_bootstrap.cc:492] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:01.683851 26456 raft_consensus.cc:348] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.684620 26456 raft_consensus.cc:374] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.684891 26456 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.685539 26456 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.686909 26621 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.695739 26451 tablet_bootstrap.cc:492] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:01.696398 26456 ts_tablet_manager.cc:1428] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260924 14:29:01.697703 26456 tablet_bootstrap.cc:492] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:01.703730 26456 tablet_bootstrap.cc:654] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.704434 26621 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:01.707123 26453 tablet_bootstrap.cc:654] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.710556 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:01.714931 26148 leader_election.cc:341] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.715253 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:01.720197 26331 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.691893 26452 raft_consensus.cc:348] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.723572 26452 raft_consensus.cc:374] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.724170 26452 raft_consensus.cc:729] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.734248 26451 tablet_bootstrap.cc:654] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.762696 26453 tablet_bootstrap.cc:492] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:01.763317 26453 ts_tablet_manager.cc:1397] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.080s	user 0.015s	sys 0.043s
I20260924 14:29:01.748111 26494 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.768953 26453 raft_consensus.cc:348] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.772899 26453 raft_consensus.cc:374] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.773231 26453 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.725353 26452 consensus_queue.cc:260] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.775775 26452 ts_tablet_manager.cc:1428] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.127s	user 0.008s	sys 0.000s
I20260924 14:29:01.776156 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:01.776329 26453 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.776898 26406 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.780712 26451 tablet_bootstrap.cc:492] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:01.781397 26451 ts_tablet_manager.cc:1397] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.092s	user 0.009s	sys 0.005s
I20260924 14:29:01.784566 26452 tablet_bootstrap.cc:492] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:01.789634 26453 ts_tablet_manager.cc:1428] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260924 14:29:01.795763 26453 tablet_bootstrap.cc:492] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:01.785029 26451 raft_consensus.cc:348] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.800182 26451 raft_consensus.cc:374] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.800588 26451 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.801429 26451 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.808816 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:01.809605 26182 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.811017 26451 ts_tablet_manager.cc:1428] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260924 14:29:01.811941 26620 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.812495 26451 tablet_bootstrap.cc:492] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:01.813308 26453 tablet_bootstrap.cc:654] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.812493 26620 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.818804 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:01.820227 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:01.818861 26620 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:01.821939 26298 leader_election.cc:341] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:01.823474 26295 leader_election.cc:341] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.823918 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:01.824445 26452 tablet_bootstrap.cc:654] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.832381 26620 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:01.853224 26451 tablet_bootstrap.cc:654] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:01.854213 26621 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.854946 26621 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.858270 26621 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:01.863183 26453 tablet_bootstrap.cc:492] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:01.863691 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:01.863801 26453 ts_tablet_manager.cc:1397] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.068s	user 0.021s	sys 0.000s
I20260924 14:29:01.865599 26479 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.866279 26479 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.867563 26494 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:01.867424 26453 raft_consensus.cc:348] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.868283 26453 raft_consensus.cc:374] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.868589 26453 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.868899 26479 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:01.871754 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:01.872716 26405 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:01.873029 26406 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:01.875384 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:01.876212 26331 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:01.876863 26149 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:01.878444 26451 tablet_bootstrap.cc:492] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
W20260924 14:29:01.878544 26148 leader_election.cc:341] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.878841 26621 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:01.877645 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:01.879397 26621 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:01.879841 26621 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.878809 26453 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:01.881333 26371 leader_election.cc:341] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.878983 26451 ts_tablet_manager.cc:1397] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.006s
I20260924 14:29:01.889497 26452 tablet_bootstrap.cc:492] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:01.889992 26452 ts_tablet_manager.cc:1397] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.106s	user 0.019s	sys 0.008s
I20260924 14:29:01.890308 26451 raft_consensus.cc:348] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.891144 26451 raft_consensus.cc:374] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.891482 26451 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.892531 26452 raft_consensus.cc:348] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:01.892781 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:01.894438 26146 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:01.895522 26564 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:01.896108 26564 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:01.897045 26564 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:01.893199 26452 raft_consensus.cc:374] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:01.906049 26452 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:01.908761 26452 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:01.909578 26494 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:01.910373 26494 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:01.911247 26494 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:01.929286 26451 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.892325 26621 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.935814 26453 ts_tablet_manager.cc:1428] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.005s
I20260924 14:29:01.937183 26451 ts_tablet_manager.cc:1428] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.001s
I20260924 14:29:01.940728 26452 ts_tablet_manager.cc:1428] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.001s
I20260924 14:29:01.942816 26452 tablet_bootstrap.cc:492] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
W20260924 14:29:01.943187 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:01.949848 26183 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 1: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:01.950657 26621 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:01.951709 26535 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:01.952592 26182 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:01.959965 26451 tablet_bootstrap.cc:492] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:01.962313 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:01.964594 26148 leader_election.cc:341] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:01.965051 26583 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.965664 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:01.965610 26583 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:01.966951 26544 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.978173 26583 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:01.981797 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:01.982052 26562 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:01.982689 26180 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:01.983815 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:01.983909 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:01.984961 26536 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:01.985349 26536 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:01.985659 26536 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:01.988761 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:01.989471 26403 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:01.991642 26453 tablet_bootstrap.cc:492] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:01.992376 26562 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 14:29:01.978050 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:29:01.998621 26562 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:01.999306 26562 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:02.003088 26562 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:01.970309 26621 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:02.003348 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:02.003746 26621 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:02.003391 26536 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:02.004792 26146 leader_election.cc:341] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:02.013609 26452 tablet_bootstrap.cc:654] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.027750 26453 tablet_bootstrap.cc:654] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.019550 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:02.018435 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:02.009820 26544 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:02.006583 26536 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:02.019986 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:02.005946 26621 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:02.007961 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:02.033193 26369 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:02.033414 26256 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.028152 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:02.005396 26494 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260924 14:29:02.030181 26402 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.043359 26456 tablet_bootstrap.cc:492] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
W20260924 14:29:02.043560 26146 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:02.028220 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 84605.6us
I20260924 14:29:02.050357 26456 ts_tablet_manager.cc:1397] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.353s	user 0.084s	sys 0.154s
I20260924 14:29:02.042089 26180 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.054955 26149 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:02.056880 26621 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:02.060282 26456 raft_consensus.cc:348] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:02.061138 26456 raft_consensus.cc:374] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.061434 26456 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:02.062130 26456 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:02.048326 26254 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:02.064945 26148 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:02.069321 26562 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:02.073019 26451 tablet_bootstrap.cc:654] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.072256 26077 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:02.069731 26562 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:02.080853 26562 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.086875 26406 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:02.088265 26617 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:02.089037 26149 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:02.091642 26617 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:02.092221 26617 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:02.093057 26617 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:02.087110 26079 catalog_manager.cc:5495] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:02.105386 26456 ts_tablet_manager.cc:1428] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260924 14:29:02.113446 26562 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:02.116160 26456 tablet_bootstrap.cc:492] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:02.128688 26453 tablet_bootstrap.cc:492] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:02.129266 26453 ts_tablet_manager.cc:1397] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.160s	user 0.017s	sys 0.001s
I20260924 14:29:02.131361 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:02.132035 26258 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.131919 26453 raft_consensus.cc:348] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.132737 26453 raft_consensus.cc:374] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.133065 26453 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:02.137319 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:02.139560 26146 leader_election.cc:341] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:02.141587 26562 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:02.144378 26453 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.153532 26479 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:02.154206 26507 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:02.154192 26479 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.164669 26453 ts_tablet_manager.cc:1428] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.001s
I20260924 14:29:02.156605 26258 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:02.172559 26479 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:02.179086 26494 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260924 14:29:02.196755 26148 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:02.206472 26402 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:02.207597 26453 tablet_bootstrap.cc:492] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:02.212306 26180 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:02.228052 26456 tablet_bootstrap.cc:654] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.244032 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:02.249934 26583 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
W20260924 14:29:02.250401 26148 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 67336.7us
I20260924 14:29:02.251456 26652 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:02.257073 26652 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.261579 26453 tablet_bootstrap.cc:654] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.229346 26494 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:02.265465 26621 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:02.284843 26564 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:02.285454 26564 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.283236 26494 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:02.287858 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:02.275585 26621 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:02.305656 26451 tablet_bootstrap.cc:492] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:02.306833 26451 ts_tablet_manager.cc:1397] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.347s	user 0.028s	sys 0.007s
I20260924 14:29:02.287823 26564 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:02.309549 26452 tablet_bootstrap.cc:492] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:02.310062 26452 ts_tablet_manager.cc:1397] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.368s	user 0.018s	sys 0.023s
I20260924 14:29:02.288606 26402 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:02.305006 26652 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:02.316033 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:02.325146 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:02.326326 26331 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 1: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:02.327809 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:02.323216 26673 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:02.329872 26494 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:02.347616 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:02.332733 26583 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:02.391984 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:02.397919 26583 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:02.344338 26295 leader_election.cc:341] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:02.338713 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:02.343061 26146 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:02.407428 26331 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
W20260924 14:29:02.415299 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 80115.2us
I20260924 14:29:02.340384 26673 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:02.341521 26621 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:02.407873 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 64464.3us
I20260924 14:29:02.415349 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:02.426358 26451 raft_consensus.cc:348] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.419744 26494 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:02.427722 26451 raft_consensus.cc:374] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.428997 26451 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:02.427858 26617 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.432590 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:02.430392 26451 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.433300 26254 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:02.435431 26451 ts_tablet_manager.cc:1428] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.128s	user 0.007s	sys 0.002s
I20260924 14:29:02.436909 26451 tablet_bootstrap.cc:492] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:02.437585 26452 raft_consensus.cc:348] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:02.438357 26452 raft_consensus.cc:374] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.438625 26452 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:02.441318 26453 tablet_bootstrap.cc:492] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:02.439252 26452 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:02.444317 26453 ts_tablet_manager.cc:1397] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.250s	user 0.031s	sys 0.044s
I20260924 14:29:02.463192 26451 tablet_bootstrap.cc:654] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.466472 26079 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:02.469575 26452 ts_tablet_manager.cc:1428] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.159s	user 0.007s	sys 0.002s
I20260924 14:29:02.471153 26452 tablet_bootstrap.cc:492] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:02.473809 26456 tablet_bootstrap.cc:492] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:02.474491 26456 ts_tablet_manager.cc:1397] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.359s	user 0.031s	sys 0.019s
I20260924 14:29:02.489842 26494 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:02.490384 26494 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.499411 26456 raft_consensus.cc:348] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.500571 26456 raft_consensus.cc:374] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.500931 26456 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:02.501821 26456 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.505183 26452 tablet_bootstrap.cc:654] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.509999 26456 ts_tablet_manager.cc:1428] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260924 14:29:02.513087 26456 tablet_bootstrap.cc:492] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:02.530633 26451 tablet_bootstrap.cc:492] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:02.531334 26451 ts_tablet_manager.cc:1397] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.095s	user 0.034s	sys 0.015s
I20260924 14:29:02.511554 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:02.524259 26148 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:02.533427 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 51895us
I20260924 14:29:02.543107 26453 raft_consensus.cc:348] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:02.543931 26453 raft_consensus.cc:374] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.544196 26453 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
W20260924 14:29:02.544482 26369 leader_election.cc:341] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:02.544871 26453 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:02.533682 26479 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:02.546036 26479 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:02.546357 26479 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.547176 26453 ts_tablet_manager.cc:1428] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.103s	user 0.004s	sys 0.000s
I20260924 14:29:02.531163 26673 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:02.526494 26146 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 188029us
I20260924 14:29:02.548023 26535 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:02.545671 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:02.552111 26180 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.536931 26451 raft_consensus.cc:348] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:02.555866 26451 raft_consensus.cc:374] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.556236 26451 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:02.556922 26451 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:02.558416 26221 leader_election.cc:341] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:02.559149 26451 ts_tablet_manager.cc:1428] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.000s
I20260924 14:29:02.560153 26451 tablet_bootstrap.cc:492] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:02.544775 26617 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:02.563189 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:02.563942 26404 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:02.567229 26456 tablet_bootstrap.cc:654] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.568025 26621 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260924 14:29:02.531689 26295 leader_election.cc:341] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:02.535094 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:02.582233 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47220.7us
I20260924 14:29:02.585683 26535 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:02.589221 26452 tablet_bootstrap.cc:492] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:02.589984 26452 ts_tablet_manager.cc:1397] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.119s	user 0.023s	sys 0.000s
I20260924 14:29:02.586216 26535 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:02.591270 26535 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.595582 26453 tablet_bootstrap.cc:492] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:02.622982 26689 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:02.623601 26689 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.625993 26689 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:02.625998 26254 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:02.599025 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 67375.5us
I20260924 14:29:02.628611 26079 catalog_manager.cc:5495] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
W20260924 14:29:02.608084 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 55435.2us
I20260924 14:29:02.670650 26451 tablet_bootstrap.cc:654] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.673838 26179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:02.683332 26329 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.682783 26452 raft_consensus.cc:348] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:02.686280 26452 raft_consensus.cc:374] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.686690 26452 raft_consensus.cc:729] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:02.689692 26453 tablet_bootstrap.cc:654] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.688180 26494 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.694902 26452 consensus_queue.cc:260] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:02.716549 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:02.739519 26583 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:02.740051 26583 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:02.742015 26583 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:02.742094 26620 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:02.745704 26452 ts_tablet_manager.cc:1428] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.155s	user 0.010s	sys 0.000s
I20260924 14:29:02.751463 26479 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.784096 26494 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:02.785872 26479 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:02.789654 26452 tablet_bootstrap.cc:492] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:02.798214 26456 tablet_bootstrap.cc:492] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:02.798947 26456 ts_tablet_manager.cc:1397] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.286s	user 0.017s	sys 0.013s
I20260924 14:29:02.808455 26456 raft_consensus.cc:348] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:02.809511 26456 raft_consensus.cc:374] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.809842 26456 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:02.777074 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:02.815840 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 67927.1us
I20260924 14:29:02.819000 26479 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:02.821393 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:02.826547 26479 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:02.821859 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:02.828533 26404 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 1: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:02.829612 26479 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:02.822019 26621 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:02.824060 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:02.780606 26535 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:02.810745 26456 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:02.836859 26295 leader_election.cc:341] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:02.837631 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:02.839252 26371 leader_election.cc:341] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:02.782464 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 194321us
I20260924 14:29:02.839064 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:02.865097 26453 tablet_bootstrap.cc:492] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:02.867118 26617 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:02.868844 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:02.868988 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:02.870306 26453 ts_tablet_manager.cc:1397] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.275s	user 0.012s	sys 0.008s
I20260924 14:29:02.872190 26183 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:02.874189 26221 leader_election.cc:341] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:02.874740 26689 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 14:29:02.878159 26456 ts_tablet_manager.cc:1428] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.003s
I20260924 14:29:02.872762 26256 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.880167 26691 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:02.869656 26535 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:02.882835 26456 tablet_bootstrap.cc:492] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:02.883625 26621 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:02.886327 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:02.890637 26224 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:02.896184 26452 tablet_bootstrap.cc:654] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:02.897477 26583 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:02.900048 26583 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:02.901755 26583 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.920169 26453 raft_consensus.cc:348] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.921278 26453 raft_consensus.cc:374] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.927335 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:02.928282 26330 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 1: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:02.929596 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:02.930027 26696 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:02.930486 26224 leader_election.cc:341] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:02.921622 26453 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:02.900087 26183 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:02.933549 26224 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:02.953629 26453 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.957930 26451 tablet_bootstrap.cc:492] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:02.958668 26451 ts_tablet_manager.cc:1397] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.399s	user 0.033s	sys 0.013s
I20260924 14:29:02.944150 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:02.945099 26673 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260924 14:29:02.960861 26178 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.944319 26330 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:02.930599 26696 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:02.970570 26674 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:02.972177 26295 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:02.974182 26691 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:02.974828 26691 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:02.975961 26691 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:02.984001 26696 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:02.985939 26256 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:02.992404 26696 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:02.990645 26451 raft_consensus.cc:348] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:02.994704 26451 raft_consensus.cc:374] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:02.995048 26451 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:02.993240 26696 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:03.000818 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 70205.9us
W20260924 14:29:03.001039 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 70588.6us
I20260924 14:29:03.004721 26453 ts_tablet_manager.cc:1428] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.134s	user 0.008s	sys 0.002s
I20260924 14:29:03.019636 26696 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:02.997480 26583 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:03.034206 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:29:03.034430 26583 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:03.039204 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:03.041965 26451 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.042657 26178 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:03.042520 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:03.045163 26371 leader_election.cc:341] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.046989 26456 tablet_bootstrap.cc:654] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.045499 26451 ts_tablet_manager.cc:1428] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.086s	user 0.006s	sys 0.003s
I20260924 14:29:03.049019 26451 tablet_bootstrap.cc:492] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:03.050693 26452 tablet_bootstrap.cc:492] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:03.051178 26452 ts_tablet_manager.cc:1397] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.268s	user 0.023s	sys 0.001s
I20260924 14:29:03.054287 26453 tablet_bootstrap.cc:492] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
W20260924 14:29:03.057714 26369 leader_election.cc:341] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.058213 26369 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:03.059106 26696 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:03.062417 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:03.062522 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:03.063300 26696 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:03.063804 26696 raft_consensus.cc:686] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:03.064601 26696 consensus_queue.cc:237] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.065738 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:03.066438 26331 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:29:03.066495 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:03.056471 26221 leader_election.cc:341] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.070137 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:03.070783 26404 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.072084 26078 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:03.074285 26681 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.074821 26681 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.079684 26681 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:03.082062 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:03.085060 26404 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:03.086849 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:03.086724 26621 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:03.087083 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:03.097146 26453 tablet_bootstrap.cc:654] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.100502 26479 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:03.056876 26452 raft_consensus.cc:348] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.106000 26452 raft_consensus.cc:374] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.106367 26452 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.107178 26452 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:03.101115 26369 leader_election.cc:341] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.105674 26479 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:03.111776 26451 tablet_bootstrap.cc:654] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.110056 26479 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.112054 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:03.111379 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:03.132267 26452 ts_tablet_manager.cc:1428] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.081s	user 0.009s	sys 0.000s
W20260924 14:29:03.136651 26369 leader_election.cc:341] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.140192 26686 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:03.150708 26078 catalog_manager.cc:5495] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:03.149362 26224 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:03.154917 26452 tablet_bootstrap.cc:492] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:03.156715 26689 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:03.159014 26583 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:03.159566 26583 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:03.160429 26583 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.179683 26453 tablet_bootstrap.cc:492] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:03.180474 26453 ts_tablet_manager.cc:1397] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.134s	user 0.010s	sys 0.008s
I20260924 14:29:03.182050 26709 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:03.184723 26453 raft_consensus.cc:348] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.185518 26453 raft_consensus.cc:374] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.185734 26453 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.169606 26479 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.189253 26451 tablet_bootstrap.cc:492] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:03.193313 26451 ts_tablet_manager.cc:1397] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.144s	user 0.009s	sys 0.019s
I20260924 14:29:03.196334 26451 raft_consensus.cc:348] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.191223 26452 tablet_bootstrap.cc:654] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.198710 26479 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:03.200999 26451 raft_consensus.cc:374] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.201485 26451 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.196077 26077 catalog_manager.cc:5495] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:03.188135 26453 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.206326 26453 ts_tablet_manager.cc:1428] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.004s
I20260924 14:29:03.203190 26451 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.208485 26453 tablet_bootstrap.cc:492] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:03.224000 26456 tablet_bootstrap.cc:492] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:03.224550 26456 ts_tablet_manager.cc:1397] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.342s	user 0.014s	sys 0.013s
I20260924 14:29:03.225569 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:03.225920 26452 tablet_bootstrap.cc:492] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:03.226549 26452 ts_tablet_manager.cc:1397] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.073s	user 0.012s	sys 0.001s
I20260924 14:29:03.227151 26456 raft_consensus.cc:348] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.227630 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:03.228027 26456 raft_consensus.cc:374] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.228251 26331 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.228340 26456 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.229164 26456 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.229323 26452 raft_consensus.cc:348] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.229952 26452 raft_consensus.cc:374] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.230221 26452 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.231160 26451 ts_tablet_manager.cc:1428] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.001s
I20260924 14:29:03.231633 26456 ts_tablet_manager.cc:1428] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:29:03.232220 26451 tablet_bootstrap.cc:492] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:03.232692 26456 tablet_bootstrap.cc:492] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:03.234658 26453 tablet_bootstrap.cc:654] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.238466 26456 tablet_bootstrap.cc:654] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.245002 26452 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.246218 26079 catalog_manager.cc:5495] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
W20260924 14:29:03.250304 26369 leader_election.cc:341] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.253858 26452 ts_tablet_manager.cc:1428] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
I20260924 14:29:03.254688 26331 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:03.260764 26451 tablet_bootstrap.cc:654] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.263731 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:03.265544 26686 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:03.263870 26452 tablet_bootstrap.cc:492] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:03.266225 26686 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:03.267429 26686 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.293372 26456 tablet_bootstrap.cc:492] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:03.294400 26456 ts_tablet_manager.cc:1397] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.012s
I20260924 14:29:03.303076 26453 tablet_bootstrap.cc:492] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:03.303821 26453 ts_tablet_manager.cc:1397] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.096s	user 0.010s	sys 0.016s
I20260924 14:29:03.310036 26452 tablet_bootstrap.cc:654] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.314604 26453 raft_consensus.cc:348] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.315436 26453 raft_consensus.cc:374] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.315747 26453 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.316485 26453 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.311146 26456 raft_consensus.cc:348] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.318310 26456 raft_consensus.cc:374] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.318651 26456 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.321698 26453 ts_tablet_manager.cc:1428] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260924 14:29:03.322773 26453 tablet_bootstrap.cc:492] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:03.324234 26452 tablet_bootstrap.cc:492] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:03.325001 26452 ts_tablet_manager.cc:1397] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.005s
I20260924 14:29:03.328814 26452 raft_consensus.cc:348] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.329535 26452 raft_consensus.cc:374] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.329631 26453 tablet_bootstrap.cc:654] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.329854 26452 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.330538 26452 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.333586 26452 ts_tablet_manager.cc:1428] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260924 14:29:03.336232 26452 tablet_bootstrap.cc:492] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:03.336601 26453 tablet_bootstrap.cc:492] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:03.337251 26453 ts_tablet_manager.cc:1397] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260924 14:29:03.339859 26453 raft_consensus.cc:348] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.329638 26079 catalog_manager.cc:5495] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:03.340634 26453 raft_consensus.cc:374] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.341019 26453 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.341756 26453 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.345918 26453 ts_tablet_manager.cc:1428] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260924 14:29:03.349663 26453 tablet_bootstrap.cc:492] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:03.319590 26456 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.354101 26456 ts_tablet_manager.cc:1428] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260924 14:29:03.355134 26456 tablet_bootstrap.cc:492] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:03.365661 26451 tablet_bootstrap.cc:492] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:03.366191 26451 ts_tablet_manager.cc:1397] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.134s	user 0.015s	sys 0.024s
I20260924 14:29:03.366393 26452 tablet_bootstrap.cc:654] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.369114 26456 tablet_bootstrap.cc:654] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.370332 26689 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.370895 26689 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.374778 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:03.375645 26453 tablet_bootstrap.cc:654] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
W20260924 14:29:03.378126 26297 leader_election.cc:341] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.369429 26451 raft_consensus.cc:348] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.378950 26451 raft_consensus.cc:374] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.379292 26451 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.384130 26453 tablet_bootstrap.cc:492] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:03.384696 26453 ts_tablet_manager.cc:1397] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.035s	user 0.005s	sys 0.005s
I20260924 14:29:03.387182 26451 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.392668 26452 tablet_bootstrap.cc:492] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:03.393250 26452 ts_tablet_manager.cc:1397] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.003s
I20260924 14:29:03.394629 26456 tablet_bootstrap.cc:492] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:03.395112 26456 ts_tablet_manager.cc:1397] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.001s
I20260924 14:29:03.392933 26451 ts_tablet_manager.cc:1428] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260924 14:29:03.392681 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:03.392737 26689 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:03.396435 26452 raft_consensus.cc:348] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.397496 26404 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:03.397710 26452 raft_consensus.cc:374] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.398177 26452 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.397814 26456 raft_consensus.cc:348] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.398808 26456 raft_consensus.cc:374] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.399065 26452 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.399423 26298 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:03.399085 26456 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.400857 26456 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.401867 26691 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:03.402227 26691 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:03.399376 26453 raft_consensus.cc:348] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.402549 26691 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.405781 26452 ts_tablet_manager.cc:1428] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260924 14:29:03.406817 26452 tablet_bootstrap.cc:492] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:03.403885 26456 ts_tablet_manager.cc:1428] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260924 14:29:03.409228 26456 tablet_bootstrap.cc:492] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:03.404024 26453 raft_consensus.cc:374] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.413004 26453 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.413748 26453 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.402460 26451 tablet_bootstrap.cc:492] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:03.414115 26691 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.416399 26453 ts_tablet_manager.cc:1428] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260924 14:29:03.418207 26452 tablet_bootstrap.cc:654] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.433179 26453 tablet_bootstrap.cc:492] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:03.434991 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:03.436743 26297 leader_election.cc:341] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.439975 26583 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.440328 26453 tablet_bootstrap.cc:654] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.440616 26583 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.419548 26456 tablet_bootstrap.cc:654] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.421062 26686 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.443122 26583 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:03.443503 26686 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.450011 26686 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:03.451408 26621 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.452059 26621 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.454280 26621 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:03.446269 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:03.454861 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:03.455642 26330 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:29:03.447410 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:03.457752 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:03.458985 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:03.459697 26402 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:03.460470 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:03.461007 26456 tablet_bootstrap.cc:492] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:03.461495 26456 ts_tablet_manager.cc:1397] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.000s
I20260924 14:29:03.462747 26149 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:03.464368 26456 raft_consensus.cc:348] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.465205 26456 raft_consensus.cc:374] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.464450 26679 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:03.465794 26679 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:03.466737 26679 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:03.469841 26221 leader_election.cc:341] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.471167 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:03.471410 26452 tablet_bootstrap.cc:492] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:03.472059 26452 ts_tablet_manager.cc:1397] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.010s
W20260924 14:29:03.473915 26146 leader_election.cc:341] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.474472 26679 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.476013 26452 raft_consensus.cc:348] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.477248 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:03.479807 26452 raft_consensus.cc:374] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260924 14:29:03.480158 26146 leader_election.cc:341] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.480224 26452 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.481158 26544 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:03.481618 26544 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:03.481969 26544 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.482168 26452 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.487025 26452 ts_tablet_manager.cc:1428] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260924 14:29:03.465466 26456 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.478137 26679 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:03.490171 26453 tablet_bootstrap.cc:492] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
W20260924 14:29:03.496447 26371 leader_election.cc:341] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.492178 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:03.497121 26453 ts_tablet_manager.cc:1397] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.002s
I20260924 14:29:03.497452 26402 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:03.498319 26221 leader_election.cc:341] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.498814 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:03.499604 26674 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:03.500372 26453 raft_consensus.cc:348] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.501312 26453 raft_consensus.cc:374] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.501587 26453 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.502892 26452 tablet_bootstrap.cc:492] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:03.504804 26689 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:03.508558 26453 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.511106 26452 tablet_bootstrap.cc:654] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.512171 26621 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.512944 26621 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.515126 26621 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:03.516373 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:03.519150 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:03.522360 26148 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:03.496521 26456 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.526191 26621 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:03.526417 26689 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.526846 26456 ts_tablet_manager.cc:1428] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.001s
I20260924 14:29:03.526970 26689 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.528000 26456 tablet_bootstrap.cc:492] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:03.530624 26689 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:03.514633 26402 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:03.533753 26149 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:03.534257 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:03.535226 26621 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
W20260924 14:29:03.535652 26295 leader_election.cc:341] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.535974 26621 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:03.536960 26621 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.539816 26686 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.533650 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:03.540405 26686 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.542658 26686 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:03.513360 26453 ts_tablet_manager.cc:1428] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260924 14:29:03.544967 26453 tablet_bootstrap.cc:492] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:03.545032 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:03.546797 26691 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:03.547612 26369 leader_election.cc:341] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.524152 26544 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:03.555920 26298 leader_election.cc:341] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.556463 26298 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:03.558424 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:03.559785 26691 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:29:03.561972 26371 leader_election.cc:341] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.562465 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:03.564769 26583 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.564918 26674 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.565274 26583 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.566004 26674 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.567878 26689 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:03.568547 26674 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:03.571729 26453 tablet_bootstrap.cc:654] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.574536 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:03.576114 26221 leader_election.cc:341] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.570842 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:03.578315 26456 tablet_bootstrap.cc:654] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.577967 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:03.578938 26221 leader_election.cc:341] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.578989 26178 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:03.563537 26686 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:03.581552 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:03.583009 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:03.588430 26544 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:03.592048 26674 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:03.592547 26674 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:03.592937 26674 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.595957 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:03.572755 26583 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:03.600646 26371 leader_election.cc:341] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.591804 26404 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.616902 26451 tablet_bootstrap.cc:654] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.625422 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:03.626294 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:03.626953 26331 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.618839 26686 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
W20260924 14:29:03.635245 26224 leader_election.cc:341] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.635852 26224 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:03.636618 26583 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:03.642473 26331 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:03.645138 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:03.645920 26689 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.646476 26689 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.633163 26080 catalog_manager.cc:5495] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:03.655432 26452 tablet_bootstrap.cc:492] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:03.656136 26452 ts_tablet_manager.cc:1397] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.153s	user 0.034s	sys 0.047s
I20260924 14:29:03.652573 26544 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:03.662919 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:03.665694 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:03.666531 26402 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 1: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:03.671766 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:03.662462 26544 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:03.663483 26689 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:03.693212 26453 tablet_bootstrap.cc:492] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:03.694979 26453 ts_tablet_manager.cc:1397] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.150s	user 0.022s	sys 0.003s
I20260924 14:29:03.698238 26456 tablet_bootstrap.cc:492] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:03.698737 26456 ts_tablet_manager.cc:1397] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.171s	user 0.007s	sys 0.005s
I20260924 14:29:03.699218 26182 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.700050 26583 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:03.705322 26456 raft_consensus.cc:348] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.706372 26456 raft_consensus.cc:374] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.706718 26456 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.708395 26456 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.716507 26452 raft_consensus.cc:348] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.717598 26452 raft_consensus.cc:374] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.708166 26689 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.717957 26452 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.721418 26453 raft_consensus.cc:348] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.722383 26452 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.722693 26453 raft_consensus.cc:374] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.724216 26453 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.732275 26456 ts_tablet_manager.cc:1428] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260924 14:29:03.734261 26452 ts_tablet_manager.cc:1428] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.001s
I20260924 14:29:03.729552 26453 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.745183 26456 tablet_bootstrap.cc:492] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:03.747529 26674 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.749648 26674 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:03.736931 26544 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.753870 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:03.754572 26180 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.747591 26617 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:03.737643 26736 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260924 14:29:03.729656 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57977.7us
I20260924 14:29:03.757328 26404 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:03.768886 26453 ts_tablet_manager.cc:1428] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.074s	user 0.009s	sys 0.000s
I20260924 14:29:03.770624 26178 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:03.773519 26479 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:03.755244 26617 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.780496 26621 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.781183 26621 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:03.782070 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:03.782335 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:03.777439 26298 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:03.782793 26331 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:03.783968 26148 leader_election.cc:341] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.784477 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:03.784631 26741 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:03.781512 26617 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:03.794390 26453 tablet_bootstrap.cc:492] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:03.785998 26479 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260924 14:29:03.803610 26617 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.806885 26617 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.808995 26617 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:03.795292 26741 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:03.811165 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:03.815243 26256 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:03.816540 26741 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.819108 26148 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:03.823613 26617 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:03.824038 26617 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:03.824122 26479 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:03.824410 26617 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:03.826579 26146 leader_election.cc:341] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.836486 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:03.837201 26401 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:03.842568 26331 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:03.836424 26452 tablet_bootstrap.cc:492] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:03.827221 26180 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:03.857571 26456 tablet_bootstrap.cc:654] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.862618 26748 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.827097 26146 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:03.879122 26453 tablet_bootstrap.cc:654] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.880203 26621 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:03.901468 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:03.895179 26674 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:03.902940 26674 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.903765 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:03.904965 26674 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:03.905500 26674 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:03.895571 26679 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:03.872924 26748 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.906625 26674 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:29:03.899099 26146 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 72567.7us
I20260924 14:29:03.917577 26451 tablet_bootstrap.cc:492] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:03.918077 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:03.919836 26224 leader_election.cc:341] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:03.918107 26451 ts_tablet_manager.cc:1397] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.516s	user 0.018s	sys 0.006s
I20260924 14:29:03.923786 26451 raft_consensus.cc:348] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.924515 26451 raft_consensus.cc:374] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:03.924787 26451 raft_consensus.cc:729] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:03.925411 26451 consensus_queue.cc:260] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:03.930820 26451 ts_tablet_manager.cc:1428] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260924 14:29:03.928855 26748 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:03.912369 26674 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:03.934495 26451 tablet_bootstrap.cc:492] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:03.946496 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:03.947733 26401 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 1: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:03.949870 26452 tablet_bootstrap.cc:654] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:03.962184 26689 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:03.954523 26081 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:03.963587 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:03.963910 26617 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:03.970582 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:03.973707 26617 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:03.976857 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:03.977676 26401 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.013213 26451 tablet_bootstrap.cc:654] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.014004 26456 tablet_bootstrap.cc:492] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:04.014515 26456 ts_tablet_manager.cc:1397] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.270s	user 0.022s	sys 0.002s
I20260924 14:29:04.015578 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:04.016203 26255 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.028386 26081 catalog_manager.cc:5495] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:04.029738 26453 tablet_bootstrap.cc:492] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:04.030268 26453 ts_tablet_manager.cc:1397] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.241s	user 0.004s	sys 0.020s
I20260924 14:29:04.032814 26453 raft_consensus.cc:348] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.033509 26453 raft_consensus.cc:374] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.033804 26453 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.033651 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:04.034520 26453 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.036425 26453 ts_tablet_manager.cc:1428] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260924 14:29:04.037372 26453 tablet_bootstrap.cc:492] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:04.009048 26146 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:04.042851 26453 tablet_bootstrap.cc:654] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.051633 26456 raft_consensus.cc:348] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.056212 26456 raft_consensus.cc:374] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.056504 26456 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.058288 26456 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.043037 26621 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:04.071828 26621 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 14:29:04.047537 26221 leader_election.cc:341] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:04.043201 26146 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 66090.2us
I20260924 14:29:04.043277 26255 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:04.083168 26456 ts_tablet_manager.cc:1428] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.001s
I20260924 14:29:04.078568 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:04.091339 26747 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:04.098232 26456 tablet_bootstrap.cc:492] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:04.099653 26674 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:04.101087 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:04.101950 26674 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.101367 26452 tablet_bootstrap.cc:492] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:04.102989 26452 ts_tablet_manager.cc:1397] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.267s	user 0.021s	sys 0.010s
W20260924 14:29:04.088544 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50679.9us
I20260924 14:29:04.104893 26148 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:04.124055 26679 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:04.124564 26679 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:04.125396 26679 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.139539 26452 raft_consensus.cc:348] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.140802 26452 raft_consensus.cc:374] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.141132 26452 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.141821 26452 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.148528 26077 catalog_manager.cc:5495] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:04.170249 26452 ts_tablet_manager.cc:1428] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.001s
I20260924 14:29:04.161981 26674 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:04.188314 26617 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.188958 26617 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.191092 26617 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:04.168676 26762 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260924 14:29:04.192941 26456 tablet_bootstrap.cc:654] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.194334 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:04.196610 26149 leader_election.cc:341] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:04.189270 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:29:04.198669 26452 tablet_bootstrap.cc:492] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:04.201773 26453 tablet_bootstrap.cc:492] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:04.202319 26453 ts_tablet_manager.cc:1397] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.165s	user 0.018s	sys 0.016s
I20260924 14:29:04.182389 26401 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
W20260924 14:29:04.217558 26146 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:04.224155 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:04.224923 26332 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:04.225662 26451 tablet_bootstrap.cc:492] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:04.226189 26451 ts_tablet_manager.cc:1397] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.292s	user 0.017s	sys 0.038s
I20260924 14:29:04.229328 26451 raft_consensus.cc:348] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.230098 26451 raft_consensus.cc:374] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.230357 26451 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.230999 26451 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.233475 26451 ts_tablet_manager.cc:1428] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:29:04.234505 26451 tablet_bootstrap.cc:492] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:04.238588 26453 raft_consensus.cc:348] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.245783 26146 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:04.239255 26453 raft_consensus.cc:374] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.246587 26453 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.247684 26617 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:04.247275 26453 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.248216 26617 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:04.248708 26617 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.251823 26452 tablet_bootstrap.cc:654] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.257114 26453 ts_tablet_manager.cc:1428] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.000s
I20260924 14:29:04.258113 26453 tablet_bootstrap.cc:492] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:04.259884 26451 tablet_bootstrap.cc:654] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.261785 26256 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.269731 26453 tablet_bootstrap.cc:654] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.271662 26452 tablet_bootstrap.cc:492] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:04.272150 26452 ts_tablet_manager.cc:1397] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.091s	user 0.009s	sys 0.003s
I20260924 14:29:04.274752 26452 raft_consensus.cc:348] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.275607 26452 raft_consensus.cc:374] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.275872 26452 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.274777 26617 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.285025 26077 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:04.287348 26544 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:04.291110 26456 tablet_bootstrap.cc:492] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:04.291754 26456 ts_tablet_manager.cc:1397] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.202s	user 0.020s	sys 0.009s
I20260924 14:29:04.292474 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:04.293156 26332 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.294559 26456 raft_consensus.cc:348] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.302632 26456 raft_consensus.cc:374] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.302623 26332 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:04.303808 26453 tablet_bootstrap.cc:492] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:04.304165 26146 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:04.304453 26453 ts_tablet_manager.cc:1397] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.047s	user 0.004s	sys 0.010s
I20260924 14:29:04.305222 26451 tablet_bootstrap.cc:492] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:04.305156 26679 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:04.305773 26451 ts_tablet_manager.cc:1397] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.071s	user 0.009s	sys 0.000s
I20260924 14:29:04.305909 26679 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:04.306838 26679 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.307125 26453 raft_consensus.cc:348] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.307966 26453 raft_consensus.cc:374] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.308234 26453 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.308288 26451 raft_consensus.cc:348] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.308977 26451 raft_consensus.cc:374] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.309314 26451 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.308914 26453 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.310053 26451 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.311100 26453 ts_tablet_manager.cc:1428] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:29:04.312152 26453 tablet_bootstrap.cc:492] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:04.312664 26451 ts_tablet_manager.cc:1428] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:29:04.313616 26451 tablet_bootstrap.cc:492] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:04.302958 26456 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.315740 26452 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.316843 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:04.317063 26456 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:04.318542 26149 leader_election.cc:341] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.320062 26617 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:04.318593 26452 ts_tablet_manager.cc:1428] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.046s	user 0.005s	sys 0.000s
I20260924 14:29:04.328405 26479 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.328807 26456 ts_tablet_manager.cc:1428] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.002s
I20260924 14:29:04.328985 26479 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.329818 26456 tablet_bootstrap.cc:492] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:04.330262 26453 tablet_bootstrap.cc:654] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.331072 26479 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:04.335271 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:04.336885 26369 leader_election.cc:341] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.339109 26456 tablet_bootstrap.cc:654] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.339825 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:04.340831 26256 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:29:04.360556 26456 tablet_bootstrap.cc:492] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:04.363359 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:04.365265 26479 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:04.365793 26479 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:04.366192 26479 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.367884 26456 ts_tablet_manager.cc:1397] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.001s
I20260924 14:29:04.376567 26689 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:04.381929 26452 tablet_bootstrap.cc:492] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:04.390570 26451 tablet_bootstrap.cc:654] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.403367 26535 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:04.403981 26456 raft_consensus.cc:348] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.405718 26452 tablet_bootstrap.cc:654] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.405637 26456 raft_consensus.cc:374] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.406288 26456 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.408128 26479 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.410128 26479 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:04.410352 26747 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.410893 26747 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.413100 26747 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:04.413810 26453 tablet_bootstrap.cc:492] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:04.414067 26451 tablet_bootstrap.cc:492] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:04.414367 26453 ts_tablet_manager.cc:1397] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.102s	user 0.021s	sys 0.008s
I20260924 14:29:04.414479 26451 ts_tablet_manager.cc:1397] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.101s	user 0.013s	sys 0.000s
I20260924 14:29:04.416281 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:04.417142 26181 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:04.416899 26453 raft_consensus.cc:348] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.418612 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:04.418942 26453 raft_consensus.cc:374] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.419258 26453 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.415277 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:04.419669 26689 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.420186 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:04.424839 26254 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.420042 26453 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.419556 26747 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:04.425891 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:04.426239 26747 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:04.426615 26747 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.427943 26453 ts_tablet_manager.cc:1428] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260924 14:29:04.429085 26453 tablet_bootstrap.cc:492] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:04.425885 26452 tablet_bootstrap.cc:492] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:04.432044 26452 ts_tablet_manager.cc:1397] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.000s
W20260924 14:29:04.432090 26369 leader_election.cc:341] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.420481 26689 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:04.422096 26224 leader_election.cc:341] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.422910 26674 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:04.436295 26451 raft_consensus.cc:348] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.437125 26451 raft_consensus.cc:374] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.437407 26451 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.438059 26451 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.440229 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:04.440946 26181 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:04.441210 26689 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:04.442112 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:04.443435 26535 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:04.443897 26535 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:04.443800 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:04.444437 26535 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.444670 26255 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:04.440395 26452 raft_consensus.cc:348] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.447062 26747 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.447371 26452 raft_consensus.cc:374] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.448339 26452 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.409193 26456 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.452211 26453 tablet_bootstrap.cc:654] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.453826 26451 ts_tablet_manager.cc:1428] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260924 14:29:04.454846 26451 tablet_bootstrap.cc:492] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:04.458007 26254 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:04.455428 26452 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.463565 26452 ts_tablet_manager.cc:1428] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260924 14:29:04.464715 26452 tablet_bootstrap.cc:492] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:04.464890 26535 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.468096 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:04.468772 26181 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.470698 26452 tablet_bootstrap.cc:654] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.455906 26456 ts_tablet_manager.cc:1428] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.087s	user 0.006s	sys 0.001s
I20260924 14:29:04.460403 26451 tablet_bootstrap.cc:654] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.475544 26456 tablet_bootstrap.cc:492] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:04.475414 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:04.477592 26479 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:04.478048 26479 raft_consensus.cc:686] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:04.478298 26689 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.478869 26689 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.478758 26479 consensus_queue.cc:237] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.480628 26181 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:04.482065 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:04.482883 26178 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:04.484526 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:04.485116 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:04.488154 26224 leader_election.cc:341] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.482714 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:04.489871 26295 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:04.490866 26691 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:04.492003 26741 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:04.492440 26741 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:04.492811 26741 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.482692 26747 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:04.495159 26748 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:04.501084 26456 tablet_bootstrap.cc:654] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.501691 26762 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.485298 26178 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.502283 26762 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.505127 26762 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:04.506878 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:04.508404 26369 leader_election.cc:341] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.516733 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:04.522893 26371 leader_election.cc:341] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.523499 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:04.524843 26762 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:04.533038 26691 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:04.534257 26691 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.536574 26689 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:04.538591 26741 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.547386 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:04.550719 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:04.551481 26181 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.551535 26741 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:04.550160 26178 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
W20260924 14:29:04.557091 26297 leader_election.cc:341] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.559264 26451 tablet_bootstrap.cc:492] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:04.560269 26451 ts_tablet_manager.cc:1397] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.106s	user 0.013s	sys 0.007s
I20260924 14:29:04.561615 26535 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:04.562135 26077 catalog_manager.cc:5495] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:04.559617 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:04.566174 26583 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:04.560238 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:04.566646 26583 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:04.545855 26081 catalog_manager.cc:5495] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:04.567519 26583 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:04.569341 26297 leader_election.cc:341] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.576009 26452 tablet_bootstrap.cc:492] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:04.577597 26453 tablet_bootstrap.cc:492] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:04.564407 26451 raft_consensus.cc:348] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.579984 26456 tablet_bootstrap.cc:492] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:04.580835 26453 ts_tablet_manager.cc:1397] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.152s	user 0.009s	sys 0.005s
I20260924 14:29:04.581269 26456 ts_tablet_manager.cc:1397] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.106s	user 0.026s	sys 0.007s
I20260924 14:29:04.580997 26451 raft_consensus.cc:374] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.581997 26451 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.582648 26451 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.583482 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:04.584034 26453 raft_consensus.cc:348] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.584149 26254 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.588012 26679 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:04.590219 26181 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:04.584836 26453 raft_consensus.cc:374] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.592180 26453 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.593376 26295 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:04.580842 26452 ts_tablet_manager.cc:1397] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.116s	user 0.009s	sys 0.007s
I20260924 14:29:04.594686 26691 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:04.595248 26691 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:04.596097 26691 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.597592 26254 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:04.593186 26453 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.599884 26453 ts_tablet_manager.cc:1428] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260924 14:29:04.600050 26079 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:04.610843 26453 tablet_bootstrap.cc:492] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:04.617182 26748 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.617769 26748 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.618515 26332 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.619778 26748 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:04.601179 26452 raft_consensus.cc:348] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.585007 26451 ts_tablet_manager.cc:1428] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260924 14:29:04.621970 26451 tablet_bootstrap.cc:492] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:04.587594 26762 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.622558 26762 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.586436 26456 raft_consensus.cc:348] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.623142 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:04.623605 26456 raft_consensus.cc:374] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.623894 26456 raft_consensus.cc:729] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.624003 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:04.624084 26402 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:04.624792 26181 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:04.625589 26224 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:04.626382 26762 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:04.624543 26456 consensus_queue.cc:260] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.627111 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:04.587085 26078 catalog_manager.cc:5495] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:04.627837 26256 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:29:04.629451 26456 ts_tablet_manager.cc:1428] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.000s
I20260924 14:29:04.629328 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:04.629518 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:04.630774 26479 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:04.631127 26479 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:04.631424 26479 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.632869 26451 tablet_bootstrap.cc:654] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.635386 26456 tablet_bootstrap.cc:492] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:04.635550 26452 raft_consensus.cc:374] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.635962 26452 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.636653 26748 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:04.637176 26748 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:04.637609 26748 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.642035 26456 tablet_bootstrap.cc:654] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
W20260924 14:29:04.631778 26369 leader_election.cc:341] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.646085 26453 tablet_bootstrap.cc:654] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.636649 26452 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.646330 26479 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.652477 26452 ts_tablet_manager.cc:1428] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.058s	user 0.003s	sys 0.003s
I20260924 14:29:04.654630 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:04.655332 26256 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.656006 26621 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.656635 26621 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.658450 26451 tablet_bootstrap.cc:492] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:04.658916 26451 ts_tablet_manager.cc:1397] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.037s	user 0.007s	sys 0.003s
I20260924 14:29:04.659559 26621 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:04.661449 26451 raft_consensus.cc:348] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.662132 26451 raft_consensus.cc:374] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.662456 26451 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.662647 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:04.663343 26255 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:04.662636 26256 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:04.665117 26148 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:04.665402 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:04.666349 26686 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:04.666888 26686 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:04.667266 26617 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:04.667537 26453 tablet_bootstrap.cc:492] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:04.667881 26617 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:04.668128 26453 ts_tablet_manager.cc:1397] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.003s
I20260924 14:29:04.668370 26617 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.667683 26686 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.670648 26453 raft_consensus.cc:348] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.671300 26453 raft_consensus.cc:374] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.671591 26453 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.672293 26453 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.674129 26748 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.679817 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:04.675855 26479 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:04.677647 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:04.680887 26402 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
W20260924 14:29:04.682133 26369 leader_election.cc:341] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.663115 26451 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.696730 26451 ts_tablet_manager.cc:1428] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.004s
I20260924 14:29:04.697844 26451 tablet_bootstrap.cc:492] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:04.678916 26452 tablet_bootstrap.cc:492] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:04.682236 26564 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:04.689921 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:04.701148 26181 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.704653 26452 tablet_bootstrap.cc:654] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.697681 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:04.709813 26402 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.706769 26564 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.715864 26564 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.718055 26564 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:04.719352 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:04.720037 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:04.721151 26149 leader_election.cc:341] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:04.721876 26146 leader_election.cc:341] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.722451 26146 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:04.723730 26564 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:04.685361 26453 ts_tablet_manager.cc:1428] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260924 14:29:04.690441 26748 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:04.724532 26451 tablet_bootstrap.cc:654] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.728196 26617 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.731277 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:04.732026 26404 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.737543 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:04.738276 26255 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.743687 26181 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:04.745132 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:04.745134 26453 tablet_bootstrap.cc:492] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:04.749949 26617 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:04.753916 26583 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:04.756126 26583 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:04.757032 26583 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.767928 26564 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:04.747301 26080 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:04.767071 26255 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:04.768251 26078 catalog_manager.cc:5495] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:04.778364 26148 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:04.779577 26402 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:04.780845 26621 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:04.781425 26621 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:04.782300 26621 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.770025 26404 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:04.815346 26535 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.816236 26453 tablet_bootstrap.cc:654] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.818061 26686 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.817174 26535 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.828167 26686 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.841950 26686 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:04.843685 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:04.844578 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:04.846957 26456 tablet_bootstrap.cc:492] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:04.847625 26456 ts_tablet_manager.cc:1397] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.212s	user 0.026s	sys 0.021s
W20260924 14:29:04.846799 26369 leader_election.cc:341] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.852144 26535 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:04.853813 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:04.856645 26371 leader_election.cc:341] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.857074 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:04.860929 26479 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:04.867791 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:04.868628 26456 raft_consensus.cc:348] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.869410 26456 raft_consensus.cc:374] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.869665 26456 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.870380 26456 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.872751 26456 ts_tablet_manager.cc:1428] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260924 14:29:04.873804 26456 tablet_bootstrap.cc:492] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
W20260924 14:29:04.873996 26149 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 -> Peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Couldn't send request to peer 45ab97b763804767846e9d27a01b561f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:04.881044 26452 tablet_bootstrap.cc:492] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:04.881567 26452 ts_tablet_manager.cc:1397] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.203s	user 0.016s	sys 0.017s
I20260924 14:29:04.884339 26451 tablet_bootstrap.cc:492] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:04.884163 26452 raft_consensus.cc:348] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.884896 26452 raft_consensus.cc:374] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.885213 26452 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.885046 26451 ts_tablet_manager.cc:1397] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.187s	user 0.022s	sys 0.021s
I20260924 14:29:04.885855 26452 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.868662 26254 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 1: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:04.888150 26451 raft_consensus.cc:348] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.888999 26451 raft_consensus.cc:374] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.879346 26456 tablet_bootstrap.cc:654] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.889319 26451 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.890196 26451 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.890648 26453 tablet_bootstrap.cc:492] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:04.891546 26453 ts_tablet_manager.cc:1397] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.165s	user 0.013s	sys 0.006s
I20260924 14:29:04.890957 26452 ts_tablet_manager.cc:1428] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260924 14:29:04.895084 26452 tablet_bootstrap.cc:492] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:04.898839 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:04.900712 26741 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.902148 26452 tablet_bootstrap.cc:654] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.910163 26535 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.910769 26535 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:04.894071 26453 raft_consensus.cc:348] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.912683 26451 ts_tablet_manager.cc:1428] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260924 14:29:04.916131 26451 tablet_bootstrap.cc:492] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:04.913965 26453 raft_consensus.cc:374] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.916024 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:04.921639 26453 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
W20260924 14:29:04.922802 26295 leader_election.cc:341] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.913728 26535 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:04.922919 26453 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.927734 26453 ts_tablet_manager.cc:1428] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260924 14:29:04.928871 26453 tablet_bootstrap.cc:492] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:04.928855 26079 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:04.931702 26456 tablet_bootstrap.cc:492] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:04.932197 26456 ts_tablet_manager.cc:1397] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.059s	user 0.019s	sys 0.009s
I20260924 14:29:04.934859 26456 raft_consensus.cc:348] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.929558 26452 tablet_bootstrap.cc:492] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:04.935662 26456 raft_consensus.cc:374] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.935930 26456 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.935997 26452 ts_tablet_manager.cc:1397] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.003s
I20260924 14:29:04.936750 26456 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.938789 26452 raft_consensus.cc:348] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.939559 26452 raft_consensus.cc:374] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:04.939838 26456 ts_tablet_manager.cc:1428] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:29:04.941059 26456 tablet_bootstrap.cc:492] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:04.942979 26451 tablet_bootstrap.cc:654] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.944088 26621 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.944690 26621 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.946936 26456 tablet_bootstrap.cc:654] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.946873 26621 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:04.939826 26452 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:04.948014 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:04.947958 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:04.948846 26254 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:04.949060 26452 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.949867 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:04.950515 26298 leader_election.cc:341] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.951033 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:04.951714 26149 leader_election.cc:341] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:04.951943 26535 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:04.952246 26583 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.952538 26148 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:04.952747 26583 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:04.953761 26621 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:04.954119 26621 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:04.954312 26452 ts_tablet_manager.cc:1428] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260924 14:29:04.954501 26621 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.955299 26453 tablet_bootstrap.cc:654] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:04.958935 26583 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:04.960994 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:04.961910 26181 raft_consensus.cc:2379] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 1: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:04.967809 26479 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.968267 26479 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:04.970038 26479 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:04.971179 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:04.971657 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:04.972636 26181 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 1: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:04.974104 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:04.974865 26479 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:04.990901 26786 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:04.991483 26452 tablet_bootstrap.cc:492] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:04.991581 26786 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:04.993934 26786 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:04.999516 26621 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:05.001863 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:05.005568 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:05.001947 26621 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:05.007588 26583 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:05.009351 26535 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:05.015112 26747 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:05.018673 26452 tablet_bootstrap.cc:654] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:05.009975 26451 tablet_bootstrap.cc:492] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:05.003129 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:05.021900 26451 ts_tablet_manager.cc:1397] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.106s	user 0.014s	sys 0.000s
I20260924 14:29:05.022150 26255 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:05.011655 26456 tablet_bootstrap.cc:492] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:05.014093 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:05.024227 26149 leader_election.cc:341] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.024423 26456 ts_tablet_manager.cc:1397] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.000s
I20260924 14:29:05.025701 26451 raft_consensus.cc:348] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.033260 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:05.034403 26254 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 1: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:05.034493 26686 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.035061 26686 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.037281 26686 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:05.026741 26451 raft_consensus.cc:374] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.038518 26451 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.038602 26741 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:05.039100 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:05.041031 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:05.042492 26255 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:05.043133 26583 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:29:05.045320 26148 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:29:05.046700 26148 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:05.047997 26621 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:05.048153 26451 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.049244 26621 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:05.050155 26621 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:05.050940 26451 ts_tablet_manager.cc:1428] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260924 14:29:05.052305 26451 tablet_bootstrap.cc:492] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:05.057284 26456 raft_consensus.cc:348] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.064268 26456 raft_consensus.cc:374] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.066567 26456 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.063561 26451 tablet_bootstrap.cc:654] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:05.072028 26456 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.065428 26479 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:05.077178 26456 ts_tablet_manager.cc:1428] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.002s
I20260924 14:29:05.078120 26456 tablet_bootstrap.cc:492] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:05.086511 26451 tablet_bootstrap.cc:492] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:05.086534 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:05.087299 26451 ts_tablet_manager.cc:1397] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.008s
I20260924 14:29:05.087896 26786 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:05.090644 26451 raft_consensus.cc:348] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.101296 26451 raft_consensus.cc:374] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.101756 26451 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.104686 26451 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.109840 26451 ts_tablet_manager.cc:1428] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.002s
I20260924 14:29:05.110921 26451 tablet_bootstrap.cc:492] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:05.091895 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:05.083844 26817 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260924 14:29:05.117609 26686 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:05.125092 26451 tablet_bootstrap.cc:654] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:05.127058 26741 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:05.131224 26786 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.131829 26786 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.133811 26786 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:05.136267 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:05.139025 26295 leader_election.cc:341] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.139942 26452 tablet_bootstrap.cc:492] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:05.142344 26741 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:29:05.142678 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50891us
I20260924 14:29:05.143086 26452 ts_tablet_manager.cc:1397] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.153s	user 0.040s	sys 0.035s
I20260924 14:29:05.141466 26451 tablet_bootstrap.cc:492] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:05.143673 26451 ts_tablet_manager.cc:1397] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.000s
I20260924 14:29:05.145586 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:05.146144 26451 raft_consensus.cc:348] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:05.146611 26686 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:05.146904 26451 raft_consensus.cc:374] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.147250 26451 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.147893 26451 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:05.149333 26452 raft_consensus.cc:348] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:05.150321 26452 raft_consensus.cc:374] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.150681 26452 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.153281 26452 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:05.156666 26452 ts_tablet_manager.cc:1428] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260924 14:29:05.149981 26451 ts_tablet_manager.cc:1428] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:29:05.156131 26077 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:05.157862 26452 tablet_bootstrap.cc:492] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:05.157862 26451 tablet_bootstrap.cc:492] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:05.136304 26078 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:05.162474 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:05.159524 26686 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:05.189673 26451 tablet_bootstrap.cc:654] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
W20260924 14:29:05.197199 26297 leader_election.cc:341] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.196870 26535 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:05.198105 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:05.196933 26741 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260924 14:29:05.199381 26786 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:05.202301 26479 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:05.205155 26479 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:05.205982 26762 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.206575 26762 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.209335 26762 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:05.214615 26456 tablet_bootstrap.cc:654] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:05.236512 26479 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.237196 26479 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.225543 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:05.243611 26479 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:05.250859 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:05.252298 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:05.256114 26369 leader_election.cc:341] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:05.259768 26369 leader_election.cc:341] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.278800 26451 tablet_bootstrap.cc:492] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:05.279680 26451 ts_tablet_manager.cc:1397] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.122s	user 0.016s	sys 0.006s
I20260924 14:29:05.270479 26181 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:29:05.271150 26748 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260924 14:29:05.297111 26451 raft_consensus.cc:348] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:05.297806 26451 raft_consensus.cc:374] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.298072 26451 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.270018 26830 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.299888 26830 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:05.270197 26747 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.301576 26747 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:05.302067 26830 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:05.300076 26451 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:05.303577 26747 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:05.303983 26179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:05.304767 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:05.305706 26253 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 1: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:05.306612 26453 tablet_bootstrap.cc:492] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:05.307112 26453 ts_tablet_manager.cc:1397] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.378s	user 0.058s	sys 0.074s
I20260924 14:29:05.312184 26451 ts_tablet_manager.cc:1428] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.002s
I20260924 14:29:05.313292 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:05.316562 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:05.324991 26453 raft_consensus.cc:348] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.326126 26453 raft_consensus.cc:374] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.329047 26479 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:05.329654 26479 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:05.330065 26479 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:05.326483 26453 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.334936 26453 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.336227 26451 tablet_bootstrap.cc:492] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:05.312873 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:05.337003 26829 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:05.359050 26456 tablet_bootstrap.cc:492] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:05.372660 26456 ts_tablet_manager.cc:1397] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.295s	user 0.008s	sys 0.045s
I20260924 14:29:05.359689 26479 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.365605 26452 tablet_bootstrap.cc:654] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:05.381759 26479 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:05.387881 26453 ts_tablet_manager.cc:1428] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.000s
I20260924 14:29:05.401119 26479 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.357503 26402 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:05.401731 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:05.403096 26479 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:05.409703 26479 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:05.416689 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:05.413400 26816 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
W20260924 14:29:05.421558 26369 leader_election.cc:341] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.402843 26617 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260924 14:29:05.422938 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:05.470534 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:05.471071 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 48259.1us
I20260924 14:29:05.475772 26583 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:05.476300 26583 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:05.476624 26583 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:05.487921 26453 tablet_bootstrap.cc:492] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:05.497954 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:05.428244 26179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:05.501644 26179 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:05.525632 26371 leader_election.cc:341] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.525270 26456 raft_consensus.cc:348] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:05.526165 26456 raft_consensus.cc:374] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.526440 26456 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.527074 26456 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:05.521170 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:05.548681 26456 ts_tablet_manager.cc:1428] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.176s	user 0.007s	sys 0.005s
I20260924 14:29:05.565384 26451 tablet_bootstrap.cc:654] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
W20260924 14:29:05.568462 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50572.7us
I20260924 14:29:05.568770 26331 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:05.577071 26456 tablet_bootstrap.cc:492] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:05.584187 26452 tablet_bootstrap.cc:492] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:05.584791 26452 ts_tablet_manager.cc:1397] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.427s	user 0.034s	sys 0.028s
W20260924 14:29:05.585986 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260924 14:29:05.588382 26369 leader_election.cc:341] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.592031 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:05.593464 26830 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:29:05.600409 26369 leader_election.cc:341] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.600857 26369 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:05.566116 26148 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 87976.5us
I20260924 14:29:05.609122 26747 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.609300 26830 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:05.618321 26452 raft_consensus.cc:348] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.622010 26452 raft_consensus.cc:374] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.622608 26452 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.624181 26452 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.642333 26452 ts_tablet_manager.cc:1428] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.002s
W20260924 14:29:05.631686 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 107879us
I20260924 14:29:05.628867 26747 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:05.651986 26402 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:05.653622 26453 tablet_bootstrap.cc:654] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:05.654989 26179 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:05.657507 26452 tablet_bootstrap.cc:492] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:05.660669 26829 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:05.676816 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:05.678961 26829 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:05.679428 26829 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:05.680250 26829 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.681643 26617 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.682134 26617 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.684087 26617 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:05.685338 26617 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260924 14:29:05.717610 26847 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.720721 26452 tablet_bootstrap.cc:654] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:05.670490 26583 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:05.750471 26453 tablet_bootstrap.cc:492] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:05.751176 26453 ts_tablet_manager.cc:1397] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.263s	user 0.034s	sys 0.011s
I20260924 14:29:05.750285 26080 catalog_manager.cc:5495] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:05.685748 26679 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
I20260924 14:29:05.766717 26456 tablet_bootstrap.cc:654] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:05.779953 26453 raft_consensus.cc:348] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.781054 26453 raft_consensus.cc:374] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.781385 26453 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.782289 26453 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.791522 26453 ts_tablet_manager.cc:1428] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.002s
I20260924 14:29:05.760903 26847 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.801618 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:05.808725 26401 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
W20260924 14:29:05.812958 26846 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260924 14:29:05.834476 26747 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:05.804481 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:05.837062 26583 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:05.855774 26847 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:05.858601 26453 tablet_bootstrap.cc:492] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:05.860810 26451 tablet_bootstrap.cc:492] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:05.861378 26451 ts_tablet_manager.cc:1397] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.525s	user 0.033s	sys 0.019s
I20260924 14:29:05.865032 26257 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:05.870222 26853 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:05.874922 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:05.875404 26452 tablet_bootstrap.cc:492] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:05.875696 26403 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:05.875939 26452 ts_tablet_manager.cc:1397] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.219s	user 0.026s	sys 0.021s
I20260924 14:29:05.875845 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:05.876544 26404 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:05.854049 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50330.7us
I20260924 14:29:05.884407 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:05.886497 26451 raft_consensus.cc:348] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.887226 26451 raft_consensus.cc:374] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.887504 26451 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.889849 26451 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.892148 26451 ts_tablet_manager.cc:1428] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260924 14:29:05.893244 26451 tablet_bootstrap.cc:492] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:05.894068 26149 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:05.894937 26853 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:05.895287 26853 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:05.895637 26853 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:05.904984 26146 leader_election.cc:341] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.907284 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:05.910679 26452 raft_consensus.cc:348] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.911567 26452 raft_consensus.cc:374] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.911839 26452 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.903316 26853 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.913069 26452 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.922890 26451 tablet_bootstrap.cc:654] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
W20260924 14:29:05.926230 26221 leader_election.cc:341] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.926323 26456 tablet_bootstrap.cc:492] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:05.927773 26456 ts_tablet_manager.cc:1397] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.351s	user 0.016s	sys 0.029s
I20260924 14:29:05.931393 26840 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:05.934156 26840 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:05.936371 26840 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:05.938611 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:05.942183 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:05.939368 26332 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:29:05.945237 26456 raft_consensus.cc:348] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:05.947125 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:05.948545 26762 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:05.948979 26762 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:05.949303 26762 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:05.945425 26452 ts_tablet_manager.cc:1428] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260924 14:29:05.960773 26456 raft_consensus.cc:374] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:05.961963 26456 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:05.951879 26453 tablet_bootstrap.cc:654] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
W20260924 14:29:05.952252 26369 leader_election.cc:341] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:05.964636 26456 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:05.968570 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 72399.3us
I20260924 14:29:05.971148 26456 ts_tablet_manager.cc:1428] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.003s
I20260924 14:29:05.975128 26452 tablet_bootstrap.cc:492] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:05.980962 26456 tablet_bootstrap.cc:492] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:05.982970 26224 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:05.984529 26847 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:05.984930 26847 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:05.985266 26847 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:05.999581 26762 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.019352 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:06.020334 26332 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.020745 26762 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:06.021548 26853 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:06.044544 26404 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:06.025848 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:06.055610 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:06.056277 26331 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.058074 26847 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.060071 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:06.064774 26369 leader_election.cc:341] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:06.079473 26331 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:06.080736 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:06.081815 26453 tablet_bootstrap.cc:492] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:06.065248 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:06.083428 26453 ts_tablet_manager.cc:1397] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.229s	user 0.015s	sys 0.012s
I20260924 14:29:06.089222 26456 tablet_bootstrap.cc:654] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.081836 26686 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:06.093652 26686 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:06.094558 26686 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.099668 26847 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:06.103901 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:06.105355 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:06.105914 26406 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.106459 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:06.118767 26848 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:06.131590 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:06.146261 26221 leader_election.cc:341] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:06.143008 26583 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
W20260924 14:29:06.150552 26862 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260924 14:29:06.141753 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 49472.5us
I20260924 14:29:06.160825 26583 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:06.166678 26583 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:06.170014 26146 leader_election.cc:341] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:06.172519 26869 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260924 14:29:06.174070 26456 tablet_bootstrap.cc:492] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:06.174753 26456 ts_tablet_manager.cc:1397] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.200s	user 0.020s	sys 0.022s
I20260924 14:29:06.177862 26456 raft_consensus.cc:348] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.178946 26456 raft_consensus.cc:374] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.179280 26456 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.180243 26456 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.184717 26456 ts_tablet_manager.cc:1428] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260924 14:29:06.189674 26452 tablet_bootstrap.cc:654] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.205140 26456 tablet_bootstrap.cc:492] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:06.207810 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:06.208551 26402 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.219153 26863 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.219959 26863 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.222604 26863 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:06.225937 26456 tablet_bootstrap.cc:654] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.226744 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:06.228950 26257 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:06.230500 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:06.234483 26369 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:06.235394 26863 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:06.236496 26863 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:06.237027 26863 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.243590 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:06.247599 26295 leader_election.cc:341] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:06.266341 26479 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:06.275080 26786 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.275734 26786 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.280261 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:06.282490 26786 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:06.290900 26402 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:06.282490 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:06.328529 26257 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 1: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:06.336026 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:06.337209 26786 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.350003 26786 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:06.261451 26453 raft_consensus.cc:348] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.323879 26451 tablet_bootstrap.cc:492] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:06.365221 26453 raft_consensus.cc:374] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.365501 26451 ts_tablet_manager.cc:1397] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.472s	user 0.013s	sys 0.017s
I20260924 14:29:06.364463 26079 catalog_manager.cc:5495] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:06.365743 26453 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.368156 26453 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.369505 26617 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.369982 26617 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.372236 26617 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:06.374433 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:06.375381 26327 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:06.377115 26146 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:06.380940 26844 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:06.381565 26844 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:06.382088 26844 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.386132 26451 raft_consensus.cc:348] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.388409 26453 ts_tablet_manager.cc:1428] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.298s	user 0.007s	sys 0.000s
I20260924 14:29:06.389636 26453 tablet_bootstrap.cc:492] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:06.387357 26451 raft_consensus.cc:374] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.391891 26451 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.325489 26332 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:06.307008 26406 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:06.360201 26224 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:06.361240 26456 tablet_bootstrap.cc:492] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:06.396466 26456 ts_tablet_manager.cc:1397] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.192s	user 0.012s	sys 0.009s
I20260924 14:29:06.396971 26149 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:06.398883 26847 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:06.399384 26847 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:06.400235 26847 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.402333 26617 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:06.401150 26456 raft_consensus.cc:348] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.403154 26456 raft_consensus.cc:374] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.403469 26456 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.404127 26617 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:06.362934 26452 tablet_bootstrap.cc:492] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:06.405012 26452 ts_tablet_manager.cc:1397] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.439s	user 0.013s	sys 0.000s
I20260924 14:29:06.405085 26617 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.407967 26456 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.398044 26451 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.410159 26456 ts_tablet_manager.cc:1428] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.000s
I20260924 14:29:06.411180 26456 tablet_bootstrap.cc:492] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:06.411767 26451 ts_tablet_manager.cc:1428] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.000s
W20260924 14:29:06.414492 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 54383us
I20260924 14:29:06.429584 26451 tablet_bootstrap.cc:492] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:06.424958 26452 raft_consensus.cc:348] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.431186 26452 raft_consensus.cc:374] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.439651 26452 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.452754 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:06.450554 26844 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.447002 26452 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.459371 26254 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:06.460312 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:06.460986 26332 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.434825 26456 tablet_bootstrap.cc:654] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.438503 26453 tablet_bootstrap.cc:654] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.463088 26844 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:06.465550 26451 tablet_bootstrap.cc:654] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.472021 26452 ts_tablet_manager.cc:1428] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.001s
I20260924 14:29:06.479192 26452 tablet_bootstrap.cc:492] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:06.335244 26863 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.485986 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:06.487578 26295 leader_election.cc:341] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:06.490020 26863 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:06.494081 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:06.494674 26254 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.497639 26453 tablet_bootstrap.cc:492] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:06.499979 26453 ts_tablet_manager.cc:1397] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.111s	user 0.019s	sys 0.000s
I20260924 14:29:06.501624 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:06.499482 26456 tablet_bootstrap.cc:492] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:06.502278 26256 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.502655 26456 ts_tablet_manager.cc:1397] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.092s	user 0.020s	sys 0.000s
I20260924 14:29:06.503613 26254 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:06.504781 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:06.505491 26456 raft_consensus.cc:348] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.505861 26863 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:06.506361 26456 raft_consensus.cc:374] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.506685 26863 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:06.507608 26863 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.512943 26453 raft_consensus.cc:348] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.513756 26453 raft_consensus.cc:374] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.513825 26256 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:06.514057 26453 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.514750 26453 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.518041 26453 ts_tablet_manager.cc:1428] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260924 14:29:06.506721 26456 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.519503 26456 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.468955 26332 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:06.521633 26456 ts_tablet_manager.cc:1428] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.000s
I20260924 14:29:06.522666 26456 tablet_bootstrap.cc:492] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:06.506697 26451 tablet_bootstrap.cc:492] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:06.526136 26451 ts_tablet_manager.cc:1397] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.097s	user 0.013s	sys 0.000s
I20260924 14:29:06.529271 26451 raft_consensus.cc:348] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.529033 26077 catalog_manager.cc:5495] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:06.529948 26451 raft_consensus.cc:374] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.530309 26451 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.530905 26451 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.532953 26451 ts_tablet_manager.cc:1428] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:29:06.533901 26451 tablet_bootstrap.cc:492] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:06.535964 26456 tablet_bootstrap.cc:654] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.537997 26148 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:06.539716 26451 tablet_bootstrap.cc:654] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.540185 26617 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:06.540706 26617 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:06.541545 26617 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.545919 26452 tablet_bootstrap.cc:654] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.548141 26453 tablet_bootstrap.cc:492] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:06.554105 26686 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.554890 26686 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.557596 26686 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:06.559921 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:06.561817 26371 leader_election.cc:341] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:06.564910 26178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:06.535756 26077 catalog_manager.cc:5495] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:06.566423 26453 tablet_bootstrap.cc:654] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.568624 26583 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 14:29:06.569849 26369 leader_election.cc:341] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:06.570326 26369 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:06.571913 26840 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:06.573472 26456 tablet_bootstrap.cc:492] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:06.573978 26456 ts_tablet_manager.cc:1397] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.052s	user 0.007s	sys 0.002s
I20260924 14:29:06.576467 26456 raft_consensus.cc:348] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.577175 26456 raft_consensus.cc:374] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.577442 26456 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.578245 26456 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.579864 26456 ts_tablet_manager.cc:1428] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260924 14:29:06.582587 26452 tablet_bootstrap.cc:492] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:06.583282 26452 ts_tablet_manager.cc:1397] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.104s	user 0.010s	sys 0.001s
I20260924 14:29:06.588246 26452 raft_consensus.cc:348] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.592525 26452 raft_consensus.cc:374] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.593210 26452 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.594329 26453 tablet_bootstrap.cc:492] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:06.596940 26453 ts_tablet_manager.cc:1397] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.049s	user 0.005s	sys 0.011s
I20260924 14:29:06.599499 26453 raft_consensus.cc:348] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.600206 26453 raft_consensus.cc:374] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.600449 26453 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.601116 26453 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.603387 26453 ts_tablet_manager.cc:1428] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:29:06.604434 26451 tablet_bootstrap.cc:492] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:06.604969 26451 ts_tablet_manager.cc:1397] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.020s
I20260924 14:29:06.605819 26453 tablet_bootstrap.cc:492] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:06.607504 26451 raft_consensus.cc:348] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.608264 26451 raft_consensus.cc:374] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.608538 26451 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.609175 26451 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.610719 26451 ts_tablet_manager.cc:1428] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260924 14:29:06.596553 26456 tablet_bootstrap.cc:492] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:06.611639 26451 tablet_bootstrap.cc:492] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:06.612116 26452 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.616003 26178 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.617002 26456 tablet_bootstrap.cc:654] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.619393 26451 tablet_bootstrap.cc:654] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.621505 26452 ts_tablet_manager.cc:1428] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260924 14:29:06.617238 26079 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:06.635164 26453 tablet_bootstrap.cc:654] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.625284 26452 tablet_bootstrap.cc:492] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:06.626652 26079 catalog_manager.cc:5495] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:06.642087 26829 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:06.646096 26452 tablet_bootstrap.cc:654] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.663184 26453 tablet_bootstrap.cc:492] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:06.663795 26453 ts_tablet_manager.cc:1397] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.019s
I20260924 14:29:06.666570 26453 raft_consensus.cc:348] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.667339 26453 raft_consensus.cc:374] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.671831 26453 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.672973 26452 tablet_bootstrap.cc:492] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:06.673507 26452 ts_tablet_manager.cc:1397] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.004s
I20260924 14:29:06.672518 26453 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.676199 26452 raft_consensus.cc:348] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.676887 26452 raft_consensus.cc:374] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.677119 26452 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.677706 26453 ts_tablet_manager.cc:1428] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260924 14:29:06.677785 26452 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.679039 26453 tablet_bootstrap.cc:492] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:06.680698 26452 ts_tablet_manager.cc:1428] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:29:06.685240 26453 tablet_bootstrap.cc:654] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.690690 26452 tablet_bootstrap.cc:492] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:06.695420 26863 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.696202 26863 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.698685 26863 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:06.699785 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:06.700110 26453 tablet_bootstrap.cc:492] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:06.700224 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:06.700943 26297 leader_election.cc:341] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:06.700574 26453 ts_tablet_manager.cc:1397] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260924 14:29:06.701128 26180 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:06.705550 26295 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:06.706543 26453 raft_consensus.cc:348] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.707280 26453 raft_consensus.cc:374] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.707584 26453 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.707640 26863 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:06.708057 26863 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:06.708500 26863 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.708380 26453 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.711038 26453 ts_tablet_manager.cc:1428] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260924 14:29:06.712208 26453 tablet_bootstrap.cc:492] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:06.717715 26453 tablet_bootstrap.cc:654] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.727514 26863 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.730862 26863 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:06.731189 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:06.732636 26297 leader_election.cc:341] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:06.732105 26452 tablet_bootstrap.cc:654] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:06.736415 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:06.737349 26180 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:06.742564 26080 catalog_manager.cc:5495] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:06.778589 26180 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:06.818208 26621 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.790071 26686 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260924 14:29:06.807209 26848 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.818773 26621 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.809628 26853 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.836457 26621 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:06.842023 26679 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.848368 26453 tablet_bootstrap.cc:492] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:06.848910 26453 ts_tablet_manager.cc:1397] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.137s	user 0.031s	sys 0.027s
I20260924 14:29:06.849210 26452 tablet_bootstrap.cc:492] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:06.849727 26452 ts_tablet_manager.cc:1397] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.168s	user 0.024s	sys 0.013s
I20260924 14:29:06.818851 26295 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:06.852341 26786 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:06.852195 26452 raft_consensus.cc:348] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.852882 26786 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:06.852923 26452 raft_consensus.cc:374] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.853813 26786 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.854368 26452 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.855166 26452 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.847987 26679 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.859961 26452 ts_tablet_manager.cc:1428] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260924 14:29:06.862073 26456 tablet_bootstrap.cc:492] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:06.862535 26456 ts_tablet_manager.cc:1397] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.266s	user 0.032s	sys 0.037s
I20260924 14:29:06.864049 26453 raft_consensus.cc:348] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.864888 26453 raft_consensus.cc:374] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.865149 26453 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.865778 26453 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.852913 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:06.875463 26451 tablet_bootstrap.cc:492] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:06.876058 26451 ts_tablet_manager.cc:1397] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.265s	user 0.033s	sys 0.053s
I20260924 14:29:06.837064 26853 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.831719 26848 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.880039 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:06.881096 26456 raft_consensus.cc:348] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.895076 26456 raft_consensus.cc:374] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.883785 26583 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.884138 26453 ts_tablet_manager.cc:1428] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260924 14:29:06.895514 26456 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
W20260924 14:29:06.887880 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:29:06.909224 26453 tablet_bootstrap.cc:492] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:06.910475 26679 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:06.912184 26149 leader_election.cc:341] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:06.900128 26456 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.892594 26686 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.924382 26451 raft_consensus.cc:348] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.925271 26451 raft_consensus.cc:374] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:06.925604 26451 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:06.926388 26451 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:06.914484 26859 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.943593 26848 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:06.943315 26859 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.944177 26848 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:06.944641 26848 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:06.946678 26848 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:06.937217 26686 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.950122 26853 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:06.951010 26453 tablet_bootstrap.cc:654] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
W20260924 14:29:06.954859 26146 leader_election.cc:341] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:06.955386 26146 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:06.957175 26451 ts_tablet_manager.cc:1428] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.001s
I20260924 14:29:06.957360 26617 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:06.908275 26583 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:06.960911 26583 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:06.965255 26451 tablet_bootstrap.cc:492] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:06.965435 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:06.951285 26686 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:06.968101 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:06.975735 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:06.977238 26405 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:06.973630 26748 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260924 14:29:06.978715 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:06.979748 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:06.923391 26907 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.053s
I20260924 14:29:06.979279 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:06.982475 26402 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:06.984967 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:06.989270 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:06.987705 26456 ts_tablet_manager.cc:1428] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.125s	user 0.015s	sys 0.002s
I20260924 14:29:06.990932 26181 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 1: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:06.991642 26456 tablet_bootstrap.cc:492] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:06.991434 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:06.992437 26182 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:06.988550 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:07.003935 26452 tablet_bootstrap.cc:492] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:07.008684 26224 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:07.020397 26906 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:07.026297 26840 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.026484 26906 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:07.026827 26840 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:06.987239 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:07.022414 26847 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:07.035205 26847 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:07.035625 26847 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.026856 26906 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:06.976315 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 64273.9us
I20260924 14:29:07.043036 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:07.043942 26146 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:07.025274 26148 leader_election.cc:341] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:07.047838 26149 leader_election.cc:341] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.048310 26148 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:07.049496 26617 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:29:07.058045 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 90058.8us
W20260924 14:29:07.066485 26221 leader_election.cc:341] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:07.068920 26371 leader_election.cc:341] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.058429 26844 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.059523 26564 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260924 14:29:07.082577 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:07.086862 26838 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.091224 26838 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:07.093359 26838 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:07.096251 26149 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 54272.4us
I20260924 14:29:07.099612 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:07.100313 26332 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:29:07.102370 26840 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:07.105337 26859 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:07.076689 26844 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:07.112311 26183 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.113435 26844 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:07.085497 26910 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.114379 26910 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.097927 26452 tablet_bootstrap.cc:654] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:07.119967 26369 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:07.122340 26451 tablet_bootstrap.cc:654] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:07.124347 26325 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.132398 26564 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:07.134271 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:07.138100 26479 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:07.138502 26479 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:07.138798 26479 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.136304 26829 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.110872 26847 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.115965 26848 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.147300 26829 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.150246 26829 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:07.154686 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:07.155774 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:07.156388 26181 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:29:07.156754 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:07.137562 26853 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:07.166321 26479 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:07.164855 26456 tablet_bootstrap.cc:654] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:07.169708 26479 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:07.164904 26369 leader_election.cc:341] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:07.172740 26369 leader_election.cc:341] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.173743 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:07.174504 26479 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:07.174985 26479 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:07.175436 26479 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.190642 26453 tablet_bootstrap.cc:492] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:07.191491 26453 ts_tablet_manager.cc:1397] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.282s	user 0.016s	sys 0.015s
I20260924 14:29:07.198393 26910 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:07.199776 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:07.200064 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:07.201030 26329 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.143769 26906 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:07.171381 26295 leader_election.cc:341] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.206404 26848 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:07.207834 26453 raft_consensus.cc:348] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:07.208401 26906 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:07.209574 26401 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 1: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:07.214134 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:07.215400 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:07.220844 26847 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:07.233387 26180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:07.230283 26453 raft_consensus.cc:374] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:07.235208 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:07.230877 26479 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.237725 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:07.238940 26254 raft_consensus.cc:2379] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 1: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:07.239018 26910 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:07.241334 26479 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:07.234694 26453 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:07.245368 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:07.244065 26453 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:07.247608 26179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:07.248219 26179 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.248517 26453 ts_tablet_manager.cc:1428] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.002s
I20260924 14:29:07.249554 26453 tablet_bootstrap.cc:492] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:07.254940 26329 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:07.258178 26148 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:07.259249 26844 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:07.260495 26621 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:29:07.261847 26369 leader_election.cc:341] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.262634 26369 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:07.263577 26762 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:07.264071 26762 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:07.264125 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:07.265192 26400 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.264999 26762 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:07.266661 26853 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:07.270151 26910 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:07.272948 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:07.273636 26401 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.279265 26453 tablet_bootstrap.cc:654] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:07.283154 26401 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:07.323371 26400 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:07.326202 26179 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:07.329429 26224 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:07.330644 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:29:07.332652 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 250150us
I20260924 14:29:07.332787 26847 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:07.337531 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:07.340335 26479 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:07.333002 26906 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:07.344278 26479 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:07.346696 26479 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.336503 26847 raft_consensus.cc:686] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:07.364063 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:07.359913 26906 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:07.385108 26452 tablet_bootstrap.cc:492] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:07.385949 26452 ts_tablet_manager.cc:1397] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.382s	user 0.019s	sys 0.008s
I20260924 14:29:07.362900 26748 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260924 14:29:07.395620 26847 consensus_queue.cc:237] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.396030 26906 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.398386 26452 raft_consensus.cc:348] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:07.399309 26452 raft_consensus.cc:374] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:07.399681 26452 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:07.399876 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:07.401351 26077 catalog_manager.cc:5495] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:07.406378 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:07.407119 26181 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.411600 26451 tablet_bootstrap.cc:492] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:07.412165 26451 ts_tablet_manager.cc:1397] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.447s	user 0.015s	sys 0.006s
W20260924 14:29:07.406394 26371 leader_election.cc:341] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.389809 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:07.412420 26452 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:07.436149 26221 leader_election.cc:341] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.435774 26181 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
W20260924 14:29:07.455536 26928 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260924 14:29:07.458176 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:07.436421 26748 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.462236 26748 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:07.464250 26748 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:07.473615 26453 tablet_bootstrap.cc:492] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:07.474200 26453 ts_tablet_manager.cc:1397] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.225s	user 0.017s	sys 0.000s
I20260924 14:29:07.475275 26252 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 0.
I20260924 14:29:07.476219 26452 ts_tablet_manager.cc:1428] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.000s
I20260924 14:29:07.478574 26452 tablet_bootstrap.cc:492] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:07.478699 26451 raft_consensus.cc:348] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.479791 26451 raft_consensus.cc:374] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:07.480162 26451 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:07.481020 26451 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.483924 26451 ts_tablet_manager.cc:1428] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.000s
I20260924 14:29:07.489339 26453 raft_consensus.cc:348] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.490428 26453 raft_consensus.cc:374] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:07.490774 26453 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:07.491816 26453 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.502146 26451 tablet_bootstrap.cc:492] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Bootstrap starting.
I20260924 14:29:07.503918 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:07.497511 26859 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260924 14:29:07.520102 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 76064.8us
I20260924 14:29:07.522786 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:07.532665 26456 tablet_bootstrap.cc:492] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:07.533243 26456 ts_tablet_manager.cc:1397] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.542s	user 0.046s	sys 0.010s
I20260924 14:29:07.534898 26453 ts_tablet_manager.cc:1428] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.002s
I20260924 14:29:07.540191 26456 raft_consensus.cc:348] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.541476 26456 raft_consensus.cc:374] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:07.541864 26456 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:07.554311 26452 tablet_bootstrap.cc:654] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:07.556712 26456 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:07.566123 26371 leader_election.cc:341] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.566653 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:07.570099 26456 ts_tablet_manager.cc:1428] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.037s	user 0.003s	sys 0.006s
I20260924 14:29:07.571772 26762 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:07.576309 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:07.577220 26332 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 1: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:07.578445 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:29:07.579632 26297 leader_election.cc:341] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.580027 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:07.581890 26691 raft_consensus.cc:2741] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260924 14:29:07.583297 26914 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:07.585806 26453 tablet_bootstrap.cc:492] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:07.590447 26863 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:07.590833 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:07.592407 26583 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.596272 26863 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:07.599682 26617 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.600225 26583 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:07.614651 26451 tablet_bootstrap.cc:654] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:07.617528 26456 tablet_bootstrap.cc:492] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:07.600250 26617 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:07.627110 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:07.627923 26332 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:07.628219 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:07.630777 26146 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:07.629570 26617 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:07.635823 26148 leader_election.cc:341] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.637718 26918 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:07.642809 26918 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:07.643206 26918 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.643187 26453 tablet_bootstrap.cc:654] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:07.648379 26451 tablet_bootstrap.cc:492] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: No bootstrap required, opened a new log
I20260924 14:29:07.649111 26451 ts_tablet_manager.cc:1397] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Time spent bootstrapping tablet: real 0.155s	user 0.012s	sys 0.003s
I20260924 14:29:07.659634 26456 tablet_bootstrap.cc:654] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:07.674186 26451 raft_consensus.cc:348] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.676934 26452 tablet_bootstrap.cc:492] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:07.683873 26452 ts_tablet_manager.cc:1397] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.206s	user 0.012s	sys 0.006s
I20260924 14:29:07.691764 26479 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:07.686573 26452 raft_consensus.cc:348] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:07.695588 26452 raft_consensus.cc:374] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:07.695894 26452 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:07.696990 26451 raft_consensus.cc:374] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:07.697270 26451 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Initialized, Role: FOLLOWER
I20260924 14:29:07.698647 26686 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.699146 26686 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.701758 26179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:07.702075 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:07.696573 26452 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:07.702420 26179 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 0.
I20260924 14:29:07.703613 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:29:07.704208 26371 leader_election.cc:341] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.704275 26452 ts_tablet_manager.cc:1428] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260924 14:29:07.704816 26479 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:07.705349 26479 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:07.705760 26479 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.702088 26686 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:07.711159 26451 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.713987 26451 ts_tablet_manager.cc:1428] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Time spent starting tablet: real 0.064s	user 0.005s	sys 0.001s
I20260924 14:29:07.720607 26077 catalog_manager.cc:5495] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:07.705332 26452 tablet_bootstrap.cc:492] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:07.722527 26077 catalog_manager.cc:5495] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:07.729017 26479 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.754531 26479 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:07.760272 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:07.766134 26179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:07.766902 26179 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:07.766916 26371 leader_election.cc:341] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.786401 26453 tablet_bootstrap.cc:492] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: No bootstrap required, opened a new log
I20260924 14:29:07.788466 26762 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:07.789767 26453 ts_tablet_manager.cc:1397] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.233s	user 0.023s	sys 0.037s
I20260924 14:29:07.789302 26179 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:07.792368 26453 raft_consensus.cc:348] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.795125 26453 raft_consensus.cc:374] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:07.795415 26453 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:07.794687 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:07.797717 26452 tablet_bootstrap.cc:654] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:07.797768 26762 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:07.798628 26762 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:07.798343 26453 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.799655 26918 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:07.800089 26762 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.801090 26453 ts_tablet_manager.cc:1428] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260924 14:29:07.802039 26078 catalog_manager.cc:5495] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:07.804399 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:07.805233 26254 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.805835 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:07.806422 26332 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.807605 26918 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:07.811831 26078 catalog_manager.cc:5495] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:07.823319 26254 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:07.825088 26148 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:07.826260 26918 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:07.827075 26918 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:07.828015 26918 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:07.830968 26332 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:07.828413 26456 tablet_bootstrap.cc:492] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:07.836181 26456 ts_tablet_manager.cc:1397] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.227s	user 0.036s	sys 0.029s
I20260924 14:29:07.838411 26456 raft_consensus.cc:348] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.846058 26844 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.846657 26844 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.848667 26844 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:07.850443 26456 raft_consensus.cc:374] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:07.850267 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:07.850791 26456 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:07.851038 26405 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 0.
I20260924 14:29:07.850870 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:07.852439 26146 leader_election.cc:341] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.853123 26149 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:07.853422 26456 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.858680 26621 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:29:07.859079 26621 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:07.859368 26621 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.864120 26906 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260924 14:29:07.884285 26621 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.885859 26456 ts_tablet_manager.cc:1428] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.002s
I20260924 14:29:07.888912 26456 tablet_bootstrap.cc:492] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:07.890353 26621 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:07.899938 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:07.900111 26183 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:07.901439 26146 leader_election.cc:341] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:07.912117 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:07.913071 26405 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.939694 26762 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:07.944782 26786 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.945574 26786 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.952154 26452 tablet_bootstrap.cc:492] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:07.952781 26452 ts_tablet_manager.cc:1397] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.248s	user 0.004s	sys 0.018s
I20260924 14:29:07.961300 26456 tablet_bootstrap.cc:654] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:07.959861 26564 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.965108 26564 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.955410 26452 raft_consensus.cc:348] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.968833 26452 raft_consensus.cc:374] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:07.969158 26452 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:07.971295 26452 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:07.970741 26583 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260924 14:29:07.968214 26859 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:07.982378 26786 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:07.983846 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:07.973497 26859 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:07.986048 26859 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:07.986784 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:07.988799 26332 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:07.989944 26564 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:07.991788 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:07.992520 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:07.985891 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:07.993337 26078 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:07.995254 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:07.971303 26080 catalog_manager.cc:5495] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 1 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:08.001987 26452 ts_tablet_manager.cc:1428] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.049s	user 0.000s	sys 0.008s
I20260924 14:29:08.002458 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:08.003091 26452 tablet_bootstrap.cc:492] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Bootstrap starting.
I20260924 14:29:08.003224 26402 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:08.005517 26149 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:08.008383 26907 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:08.010882 26452 tablet_bootstrap.cc:654] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:08.021257 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:08.031315 26859 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:08.031666 26863 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:29:08.050741 26762 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:08.086480 26452 tablet_bootstrap.cc:492] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: No bootstrap required, opened a new log
I20260924 14:29:08.087033 26452 ts_tablet_manager.cc:1397] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Time spent bootstrapping tablet: real 0.084s	user 0.020s	sys 0.051s
I20260924 14:29:08.096060 26456 tablet_bootstrap.cc:492] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:08.096661 26456 ts_tablet_manager.cc:1397] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.208s	user 0.007s	sys 0.006s
I20260924 14:29:08.081331 26405 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:08.085459 26844 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:08.106664 26149 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:08.108093 26621 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:08.108706 26621 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:08.111110 26920 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260924 14:29:08.116755 26844 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:08.117308 26844 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:08.121467 26456 raft_consensus.cc:348] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:08.122540 26456 raft_consensus.cc:374] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:08.109669 26621 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:08.128275 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:29:08.122874 26456 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:08.130324 26456 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:08.174701 26452 raft_consensus.cc:348] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:08.183202 26452 raft_consensus.cc:374] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:08.185010 26452 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Initialized, Role: FOLLOWER
I20260924 14:29:08.191066 26452 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:08.204751 26452 ts_tablet_manager.cc:1428] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Time spent starting tablet: real 0.117s	user 0.010s	sys 0.000s
I20260924 14:29:08.215925 26079 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:08.232337 26910 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:08.246423 26456 ts_tablet_manager.cc:1428] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.149s	user 0.006s	sys 0.017s
W20260924 14:29:08.248838 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 54183.9us
I20260924 14:29:08.258194 26456 tablet_bootstrap.cc:492] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Bootstrap starting.
I20260924 14:29:08.246237 26910 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:08.327950 26149 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 57185us
W20260924 14:29:08.329736 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 59946.2us
W20260924 14:29:08.331350 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 71677.8us
I20260924 14:29:08.340929 26583 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:08.341486 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 83029.3us
I20260924 14:29:08.418300 26332 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:08.447700 26906 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:08.400911 26583 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:08.471630 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 136735us
I20260924 14:29:08.464293 26910 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:08.506407 26456 tablet_bootstrap.cc:654] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Neither blocks nor log segments found. Creating new log.
I20260924 14:29:08.506958 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:08.510389 26964 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260924 14:29:08.516363 26564 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:08.555622 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:08.493625 26844 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:08.552270 26405 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 1: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:29:08.567768 26148 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 80007.2us
I20260924 14:29:08.472738 26906 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:08.575201 26966 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260924 14:29:08.587103 26181 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:08.615315 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:08.642611 26621 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:08.633461 26253 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:08.635154 26912 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:29:08.758225 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 183512us
W20260924 14:29:08.762205 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 128862us
I20260924 14:29:08.686172 26583 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:08.789628 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 152524us
I20260924 14:29:08.792481 26844 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:08.790877 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:08.817462 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:08.809473 26906 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:08.825714 26400 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 0.
I20260924 14:29:08.860423 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:08.878011 26456 tablet_bootstrap.cc:492] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: No bootstrap required, opened a new log
I20260924 14:29:08.878628 26456 ts_tablet_manager.cc:1397] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Time spent bootstrapping tablet: real 0.626s	user 0.022s	sys 0.004s
I20260924 14:29:08.909924 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:08.888468 26920 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260924 14:29:08.856194 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 53895.3us
I20260924 14:29:08.906056 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:08.917064 26920 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:08.943368 26920 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:08.952070 26621 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:08.939710 26182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:08.951088 26149 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:08.956738 26564 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:08.961604 26456 raft_consensus.cc:348] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:08.962381 26456 raft_consensus.cc:374] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:29:08.962664 26456 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Initialized, Role: FOLLOWER
I20260924 14:29:08.962780 26762 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:08.963325 26456 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:08.965935 26456 ts_tablet_manager.cc:1428] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Time spent starting tablet: real 0.087s	user 0.004s	sys 0.000s
I20260924 14:29:08.959143 26920 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:08.968674 26920 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:08.971881 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:08.972632 26328 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:08.979866 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:08.986287 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:09.005761 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:09.006609 26406 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:29:09.003197 26920 consensus_queue.cc:1046] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.026s
I20260924 14:29:09.029873 26920 raft_consensus.cc:3032] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:09.030385 26920 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:09.031145 26920 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:09.032263 26920 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:09.017189 26912 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:29:09.068020 26991 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260924 14:29:09.090950 26328 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:09.098644 26844 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:09.109740 26406 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:09.134696 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:09.157951 26906 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:29:09.181943 26906 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:09.182876 26906 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:09.202685 26848 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:29:09.234645 26129 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260924 14:29:09.254010 26997 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.003s
I20260924 14:29:09.328680 26079 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 1 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:09.692619 26848 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:09.723083 26848 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:29:10.370757 27005 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:10.371400 27005 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:10.373660 27005 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:10.374845 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:10.375783 26250 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:10.374490 26177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:10.376886 26177 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:10.377055 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:10.377925 27005 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:10.378273 27005 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:10.378600 27005 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:10.385242 27005 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:10.389595 26177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:10.390473 26177 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:10.391628 27005 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:10.391992 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:10.398711 26250 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:10.399745 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:10.400892 27005 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:10.401417 27005 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:10.402275 27005 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:10.417150 26079 catalog_manager.cc:5495] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 1 to 2, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 2 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:10.873430 26918 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:10.873971 26918 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:10.875973 26918 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:10.876996 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:10.877019 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:10.877916 26253 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:10.878855 26148 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:10.879916 26918 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:10.880390 26918 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:10.880743 26918 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:10.886529 26918 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:10.888463 26918 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:10.890576 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:10.892553 26175 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:10.894515 26966 consensus_queue.cc:1046] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:29:10.895607 27005 raft_consensus.cc:3032] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:10.895985 27005 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:10.896984 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:10.896804 27005 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:10.897531 26250 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:10.898146 27005 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:10.909056 26250 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:10.910259 26148 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:10.911334 26621 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:10.918042 26621 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:10.918913 26621 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:10.930519 26079 catalog_manager.cc:5495] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 3, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:11.013572 23135 tablet_server.cc:178] TabletServer@127.22.151.193:0 shutting down...
W20260924 14:29:11.038182 26221 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:29:11.061251 26221 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.081065 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:29:11.083987 23135 tablet_replica.cc:330] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.085235 23135 raft_consensus.cc:2229] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.086709 23135 raft_consensus.cc:2258] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.090965 23135 tablet_replica.cc:330] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.092716 23135 raft_consensus.cc:2229] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.093466 23135 raft_consensus.cc:2258] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.098344 23135 tablet_replica.cc:330] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.099571 23135 raft_consensus.cc:2229] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.100183 23135 raft_consensus.cc:2258] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.102514 23135 tablet_replica.cc:330] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.103399 23135 raft_consensus.cc:2229] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.104457 23135 raft_consensus.cc:2258] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.106974 23135 tablet_replica.cc:330] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.109025 23135 raft_consensus.cc:2229] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.110080 23135 raft_consensus.cc:2258] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.112690 23135 tablet_replica.cc:330] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.114720 23135 raft_consensus.cc:2229] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.115773 23135 raft_consensus.cc:2258] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:29:11.117662 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.118289 26369 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.118834 26369 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.119520 26369 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.125337 23135 tablet_replica.cc:330] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.126286 23135 raft_consensus.cc:2229] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.126780 23135 raft_consensus.cc:2258] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.128933 23135 tablet_replica.cc:330] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.129834 23135 raft_consensus.cc:2229] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.130685 23135 raft_consensus.cc:2258] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.132912 23135 tablet_replica.cc:330] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.133662 23135 raft_consensus.cc:2229] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.182166 23135 raft_consensus.cc:2258] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.205796 23135 tablet_replica.cc:330] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
W20260924 14:29:11.274252 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.275175 26369 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.283308 26369 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.284113 23135 raft_consensus.cc:2229] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.324033 23135 raft_consensus.cc:2258] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:29:11.340055 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.355407 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.356226 26399 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260924 14:29:11.359319 26399 pending_rounds.cc:76] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 1
I20260924 14:29:11.359650 26399 pending_rounds.cc:97] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:11.360116 26399 raft_consensus.cc:2866] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:29:11.361696 26295 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.368975 27029 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:11.358834 23135 tablet_replica.cc:330] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
W20260924 14:29:11.376355 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.378194 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.376665 23135 raft_consensus.cc:2229] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260924 14:29:11.393338 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.398665 26295 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.398875 26369 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.399405 23135 raft_consensus.cc:2258] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:29:11.416631 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.419826 26221 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.423107 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.423908 26221 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.424571 26221 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.429158 26221 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.433667 23135 tablet_replica.cc:330] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.437208 26253 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:11.442808 27020 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260924 14:29:11.448805 26221 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.450259 26295 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.452533 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:11.453190 26295 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.450810 23135 raft_consensus.cc:2229] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.455314 23135 raft_consensus.cc:2258] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:29:11.459825 26295 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.471711 23135 tablet_replica.cc:330] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.473397 23135 raft_consensus.cc:2229] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.474511 23135 raft_consensus.cc:2258] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.490653 23135 tablet_replica.cc:330] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.491809 23135 raft_consensus.cc:2229] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.492563 23135 raft_consensus.cc:2258] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.496560 23135 tablet_replica.cc:330] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.497524 23135 raft_consensus.cc:2229] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.498226 23135 raft_consensus.cc:2258] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.513811 23135 tablet_replica.cc:330] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.515628 23135 raft_consensus.cc:2229] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.516453 23135 raft_consensus.cc:2258] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.520006 23135 tablet_replica.cc:330] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.521220 23135 raft_consensus.cc:2229] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.522039 23135 raft_consensus.cc:2258] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.526227 23135 tablet_replica.cc:330] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.528249 23135 raft_consensus.cc:2229] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.529791 23135 raft_consensus.cc:2258] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.533108 23135 tablet_replica.cc:330] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.534305 23135 raft_consensus.cc:2229] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.535619 23135 raft_consensus.cc:2258] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.538669 23135 tablet_replica.cc:330] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.540078 23135 raft_consensus.cc:2229] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.540853 23135 raft_consensus.cc:2258] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.544039 23135 tablet_replica.cc:330] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.545486 23135 raft_consensus.cc:2229] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.546281 23135 raft_consensus.cc:2258] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.549912 23135 tablet_replica.cc:330] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.550825 23135 raft_consensus.cc:2229] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.551677 23135 raft_consensus.cc:2258] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.554967 23135 tablet_replica.cc:330] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.555913 23135 raft_consensus.cc:2229] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.556543 23135 raft_consensus.cc:2258] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.559811 23135 tablet_replica.cc:330] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.560617 23135 raft_consensus.cc:2229] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.561926 23135 raft_consensus.cc:2258] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.565538 23135 tablet_replica.cc:330] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.566510 23135 raft_consensus.cc:2229] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.567153 23135 raft_consensus.cc:2258] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.570662 23135 tablet_replica.cc:330] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.571647 23135 raft_consensus.cc:2229] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.572304 23135 raft_consensus.cc:2258] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:29:11.574019 26369 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:11.575590 23135 tablet_replica.cc:330] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.576543 23135 raft_consensus.cc:2229] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.577773 23135 raft_consensus.cc:2258] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.581032 23135 tablet_replica.cc:330] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.581859 23135 raft_consensus.cc:2229] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.582932 23135 raft_consensus.cc:2258] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.586935 23135 tablet_replica.cc:330] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.588002 23135 raft_consensus.cc:2229] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.588663 23135 raft_consensus.cc:2258] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.591624 23135 tablet_replica.cc:330] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.592442 23135 raft_consensus.cc:2229] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.593102 23135 raft_consensus.cc:2258] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.596575 23135 tablet_replica.cc:330] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.597468 23135 raft_consensus.cc:2229] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.598166 23135 raft_consensus.cc:2258] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.601632 23135 tablet_replica.cc:330] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.602604 23135 raft_consensus.cc:2229] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.603417 23135 raft_consensus.cc:2258] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.606634 23135 tablet_replica.cc:330] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.607573 23135 raft_consensus.cc:2229] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.608275 23135 raft_consensus.cc:2258] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.611613 23135 tablet_replica.cc:330] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.612459 23135 raft_consensus.cc:2229] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.613085 23135 raft_consensus.cc:2258] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.616114 23135 tablet_replica.cc:330] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.616895 23135 raft_consensus.cc:2229] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.617550 23135 raft_consensus.cc:2258] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.621019 23135 tablet_replica.cc:330] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.621861 23135 raft_consensus.cc:2229] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.623104 23135 raft_consensus.cc:2258] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.626729 23135 tablet_replica.cc:330] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.627822 23135 raft_consensus.cc:2229] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.628707 23135 raft_consensus.cc:2258] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.632149 23135 tablet_replica.cc:330] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.632982 23135 raft_consensus.cc:2229] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.633733 23135 raft_consensus.cc:2258] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.637377 23135 tablet_replica.cc:330] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.638227 23135 raft_consensus.cc:2229] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.638998 23135 raft_consensus.cc:2258] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.642179 23135 tablet_replica.cc:330] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.643019 23135 raft_consensus.cc:2229] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.643702 23135 raft_consensus.cc:2258] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.646535 23135 tablet_replica.cc:330] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.647281 23135 raft_consensus.cc:2229] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.648437 23135 raft_consensus.cc:2258] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.651629 23135 tablet_replica.cc:330] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.652382 23135 raft_consensus.cc:2229] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.653564 23135 raft_consensus.cc:2258] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.656666 23135 tablet_replica.cc:330] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.657580 23135 raft_consensus.cc:2229] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:29:11.658824 23135 raft_consensus.cc:2258] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.662118 23135 tablet_replica.cc:330] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.662878 23135 raft_consensus.cc:2229] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.663558 23135 raft_consensus.cc:2258] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:11.666705 23135 tablet_replica.cc:330] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: stopping tablet replica
I20260924 14:29:11.667399 23135 raft_consensus.cc:2229] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:29:11.668058 23135 raft_consensus.cc:2258] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:29:12.076025 27046 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.076572 27046 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.078372 27046 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.079422 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:12.081878 26295 leader_election.cc:334] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.082240 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.082888 27046 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.115777 26912 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.116287 26912 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.118245 26912 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.119202 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:12.122467 26295 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.122816 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.123383 26912 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.168267 27033 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.168846 27033 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.171097 27033 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.171928 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:12.176208 26369 leader_election.cc:334] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.176608 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:12.177313 27033 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.233096 26912 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.233724 26912 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.235917 26912 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.236584 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:12.237319 26404 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:12.238437 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:12.239223 26912 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.239584 26912 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.239905 26912 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:12.239979 26295 leader_election.cc:334] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.246090 26912 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.247772 26912 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.248564 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:12.249127 26404 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:12.251857 26295 leader_election.cc:334] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.255556 26404 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:12.256536 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.257021 27033 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.257270 26912 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:12.257802 26912 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:12.257591 27033 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.258819 26912 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.259845 27033 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.260521 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:12.266486 26369 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.267063 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:12.267828 27033 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.272927 26077 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:12.279666 27005 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.280189 27005 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.281935 27005 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.289793 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:12.298055 26369 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.300585 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.301472 27041 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.327142 27033 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.327812 27033 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.329811 27033 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.331693 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:12.332405 26325 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:12.333957 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:12.334893 27033 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260924 14:29:12.334947 26369 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.335269 27033 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.335691 27033 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.343248 26912 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.343501 27033 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.344221 26912 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.345381 27030 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.345652 27033 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.346050 27030 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.348242 27033 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.348883 27033 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:12.351617 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.352262 27033 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.353160 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:12.353618 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:12.354302 26327 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:12.354692 26912 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.353856 26325 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.351042 27030 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:12.356132 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:12.356783 26327 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
W20260924 14:29:12.357615 26369 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:29:12.358161 26369 leader_election.cc:334] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.359030 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:12.360023 26221 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.360198 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:12.360898 26258 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:12.361003 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.362212 27030 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.362584 27030 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.362995 27030 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.362671 27033 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.363986 27033 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.364926 27033 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.367544 26325 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:12.372294 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.373461 26912 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.373873 26912 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.374231 26912 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.374392 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.375340 26966 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:12.375852 26966 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:12.376878 26966 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.377868 27033 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.379931 27033 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.382635 26906 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.383322 26906 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.385492 26906 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:12.387419 26912 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.394688 26912 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:12.394734 26221 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.391377 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:12.390096 27030 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.398270 27030 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:12.400038 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.403158 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:12.403945 26325 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:12.404575 26369 leader_election.cc:334] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.406481 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:12.407375 27030 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.410195 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:12.408906 26077 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 2 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:12.411084 26327 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:12.412513 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:12.413331 26258 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 2: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:12.416435 26325 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:12.420634 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.425133 26966 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:12.425725 26966 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:12.426776 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.426712 26966 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.428138 26912 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260924 14:29:12.433004 26221 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.434988 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:12.436924 26848 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.442085 26912 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.442740 26912 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.445035 26912 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.444497 26077 catalog_manager.cc:5495] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 2 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:12.446890 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:12.447650 26258 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:12.448604 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:12.449527 26912 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.449957 26912 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.450417 26912 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:12.451900 26295 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.458690 26912 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.460839 26912 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.462100 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:12.462776 26258 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.471719 26258 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
W20260924 14:29:12.464952 26295 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.473351 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.474246 26912 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:12.474738 26912 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:12.475682 26912 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.503108 26077 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:12.549618 26848 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.550163 26848 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.552289 26848 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:12.557440 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:12.560087 26221 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.560531 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:12.561666 26912 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.561738 26906 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.562376 26912 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.565454 26912 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.566053 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:12.566748 26405 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:12.567934 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:12.570391 27046 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.570760 27046 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.571110 27046 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:12.576395 26295 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.580711 27046 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.582443 26848 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.583076 26848 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.585965 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:12.586668 26404 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:12.588013 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:12.588904 26906 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.589231 26848 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.589350 26906 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.589941 26906 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.590169 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:12.590806 26405 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:12.593808 26221 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.602945 26405 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:12.603981 26906 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.610584 27046 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.611868 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:12.614490 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:12.614709 26906 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.617025 26404 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:12.618791 26221 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.621886 27058 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:12.622439 27058 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:12.623355 27058 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.625182 26404 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:12.626536 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.627640 26906 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:12.628242 26906 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:12.629173 26906 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:12.631693 26295 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.657913 26077 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:12.695984 26077 catalog_manager.cc:5495] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
W20260924 14:29:12.700590 26295 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:12.707422 26405 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:12.709455 27058 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:12.736824 27005 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.737450 27005 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.739758 26966 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.740383 26966 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.742245 27033 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.742524 26966 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.743075 26966 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.742889 27033 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.744180 26966 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.745110 27033 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.747002 26966 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.748632 27005 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.751103 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:12.752296 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:12.753731 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:12.755456 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:12.765946 26848 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.766572 26848 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.767306 26325 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:12.773331 27005 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:12.776446 26369 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.776916 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.777647 27005 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.778080 26848 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:12.778549 26369 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.779059 26369 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.779788 27005 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:29:12.780815 26369 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:29:12.782608 26221 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.781244 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.784056 27005 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:29:12.787631 26369 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.788125 26369 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.788926 27005 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.791744 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:12.792564 26405 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:12.800909 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.803162 26848 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.803597 26848 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.803956 26848 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:12.821245 26369 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:12.824095 27005 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:12.825271 23135 thread.cc:536] Waited for 1000ms trying to join with diag-logger (tid 26204)
I20260924 14:29:12.824779 27005 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.827919 27005 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.836062 26848 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.838651 26848 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.841732 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:12.842420 26405 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:12.842975 26221 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:29:12.843588 26369 leader_election.cc:334] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.849400 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:12.851774 26405 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:12.851918 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:12.853593 27005 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.854686 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.855774 26848 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:12.856216 26848 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:12.857045 26848 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.862076 27056 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.862604 27058 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.862638 27056 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.863338 27058 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.865558 27058 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.868201 27030 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.868068 27056 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.868810 27030 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.871048 27030 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.871250 26966 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.872126 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:12.874058 26966 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.874841 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:12.876571 26966 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.878805 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:12.879724 26404 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:12.879784 26405 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
W20260924 14:29:12.880630 26295 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.882586 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:12.883047 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:12.883957 26295 leader_election.cc:334] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.884429 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.885115 27058 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:12.885188 27056 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.886047 27056 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.885488 26077 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:12.886514 27056 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:12.888851 26221 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.890352 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.891188 27030 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.891574 27030 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:12.891705 26369 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.891934 27030 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.892244 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.892989 26966 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:29:12.903506 26295 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:12.904359 27058 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.904982 27058 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.905951 27056 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.907636 27058 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.908772 27056 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.909504 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:12.910108 26405 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.912849 26966 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.913362 26906 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.913899 26906 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.915958 26906 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.916421 27030 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.913395 26966 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.930395 27030 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.930482 26966 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.918731 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:12.926294 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:12.932171 26258 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:12.932040 26404 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:12.919530 26848 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.933727 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:12.934010 26221 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.934414 26399 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.933794 26848 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.938057 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.938618 26848 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.939580 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:12.939816 26848 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.940237 26848 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.940641 26848 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.943715 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:12.944439 26404 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:12.948385 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:12.949342 27030 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.949791 27030 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:12.950043 26369 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.950385 27030 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.952350 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:12.924556 26295 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.953102 26966 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:12.923702 26405 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:12.954725 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:12.949642 26848 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.955597 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:12.957424 27058 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:12.957652 26325 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:12.957669 26848 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.958520 26848 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.955601 27056 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:12.959497 27056 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:12.958931 26848 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.960417 27056 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:12.961917 26848 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.961879 26966 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:12.963562 26369 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:12.957773 27058 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.966218 27058 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.974750 27058 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:12.978845 26221 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:29:12.979656 26221 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.980245 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:12.980921 26404 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:12.981170 26906 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.981750 26906 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:12.984167 26906 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:12.984185 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:12.985033 26405 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:12.987733 26253 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:12.988000 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:12.988857 26906 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:12.989221 26906 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.989959 26906 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:12.991780 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 67242.7us
W20260924 14:29:12.992443 26295 leader_election.cc:334] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:12.992363 26077 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:12.994550 26912 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:12.995015 26912 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:12.996037 26405 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:12.996829 26912 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:12.994551 26399 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:13.000810 27058 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:13.005231 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:29:13.005702 26295 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:13.007886 26848 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:13.007834 27030 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:13.008530 26848 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:13.008795 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:13.010314 26848 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:13.010440 27030 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:13.010776 26848 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:13.012923 26848 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:13.014221 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:13.016841 26399 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:13.011255 27030 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:13.018553 27030 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:13.018954 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:13.019629 26258 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:13.019486 27030 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:13.017277 26848 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:13.010777 27058 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260924 14:29:13.017848 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:13.022637 26253 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:13.027977 26258 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
W20260924 14:29:13.030047 26221 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:29:13.030669 26221 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:13.038421 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:13.039152 27058 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:13.039569 27058 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:13.039995 27058 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:13.040956 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:13.026719 26906 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:13.043629 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:13.045054 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:13.046043 26224 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:13.046826 27070 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:13.047354 27070 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:13.048225 27070 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:13.049309 26912 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:13.049815 26912 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:13.050594 26405 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:13.050679 26912 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:13.052989 27056 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:13.048568 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:13.053960 26848 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:13.054152 26399 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:13.054517 26848 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:13.054854 26848 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:13.051335 26221 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:13.063310 26906 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:13.064849 26079 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:13.068943 26848 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:13.072901 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:13.073535 26404 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:13.066932 27058 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:13.077718 26328 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:29:13.088481 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:29:13.090389 26221 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:29:13.090992 26221 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:13.093667 26399 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:13.094774 26848 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:13.095021 27058 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:13.048570 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:13.104350 26325 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:13.100739 26404 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
W20260924 14:29:13.091434 26221 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:29:13.143316 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 51900.9us
W20260924 14:29:13.146512 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:13.150497 26077 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:13.153708 26079 catalog_manager.cc:5495] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:13.155493 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:13.111682 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:13.111313 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:13.156136 26253 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:13.156503 26399 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:13.163640 26848 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:13.164072 26253 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:13.164248 26848 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:13.165210 26848 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:13.165829 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:13.167946 27070 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:13.168598 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:13.169554 27070 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:13.169917 27070 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:13.170265 27070 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:13.171312 26221 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:13.176282 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:13.177047 26912 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:13.177505 26912 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:13.178349 26912 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:13.180482 26221 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:13.182103 27070 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:13.184278 27070 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:13.185153 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 73559.6us
I20260924 14:29:13.185935 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:13.188848 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:13.189718 26328 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:13.190539 27069 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:13.191159 27069 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:13.191985 27069 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:13.192811 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:13.203780 26221 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:13.211537 26328 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:13.216312 26399 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:13.220559 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:13.222102 27069 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:13.222787 27069 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:13.223651 27069 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:13.230167 26224 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:13.232303 26404 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:13.235759 26906 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:13.239368 26077 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:13.252152 27030 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:13.253238 26906 raft_consensus.cc:686] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:13.254170 26906 consensus_queue.cc:237] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:13.259752 26404 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:13.263064 26848 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:13.266367 26079 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:13.293062 26077 catalog_manager.cc:5495] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 4, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 4 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:13.294958 26077 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:13.296916 26077 catalog_manager.cc:5495] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
W20260924 14:29:13.337160 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:13.359647 26221 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:29:13.393417 26404 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:13.394860 26912 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:13.414917 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:13.462929 26404 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:13.464366 27030 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:13.480607 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:13.492093 26404 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:13.493512 27030 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:13.519769 26221 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:13.545017 26295 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.559633 26221 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.562372 26221 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.580605 26295 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:13.612561 26253 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:13.613849 27058 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:13.628013 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:29:13.636998 26253 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:13.639369 26912 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:13.654917 26369 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.680542 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.695127 26221 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:13.701967 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:29:13.704089 26404 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:13.706851 27030 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260924 14:29:13.720852 26369 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:29:13.727178 26405 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260924 14:29:13.730108 26221 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:13.731593 26295 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:29:13.732205 26906 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260924 14:29:13.748208 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.751024 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:29:13.754438 26327 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:13.761408 26906 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:13.773690 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:13.803957 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:13.827823 23135 thread.cc:536] Waited for 2000ms trying to join with diag-logger (tid 26204)
W20260924 14:29:13.854729 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.866840 26369 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.870070 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.873857 26221 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.875753 26221 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.876058 26295 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:29:13.878539 26404 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260924 14:29:13.880208 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.880640 26369 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:29:13.881544 27030 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:13.885464 26369 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.899336 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.902612 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.935954 26295 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.949334 26369 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:13.963828 26369 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:14.021220 26221 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:14.150468 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:14.179418 26295 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:29:14.354286 27095 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:14.354874 27095 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:14.357291 27095 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:14.359505 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:14.360203 26253 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:14.361549 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:14.364821 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:14.365574 27095 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:14.371101 27095 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:14.373029 27095 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:14.381232 27095 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:14.383553 27095 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:14.386797 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:14.387552 26253 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:14.388983 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:29:14.395274 26253 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:14.396481 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:14.397257 27095 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:14.397735 27095 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:14.398427 27095 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:14.411176 26077 catalog_manager.cc:5495] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 3, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:14.825500 26258 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:14.827481 27095 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260924 14:29:14.828363 23135 thread.cc:536] Waited for 3000ms trying to join with diag-logger (tid 26204)
W20260924 14:29:14.905428 26295 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:29:15.372941 26295 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:15.387602 26369 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:15.457481 26295 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:15.577301 26369 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:15.702783 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:15.713187 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08000c6600 after lost signal to thread 23138
W20260924 14:29:15.714457 26279 debug-util.cc:398] Leaking SignalData structure 0x7b0800348ce0 after lost signal to thread 26115
W20260924 14:29:15.715893 26279 debug-util.cc:398] Leaking SignalData structure 0x7b0800169840 after lost signal to thread 26282
W20260924 14:29:15.717031 26279 debug-util.cc:398] Leaking SignalData structure 0x7b0800101240 after lost signal to thread 26356
W20260924 14:29:15.725883 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08000f7580 after lost signal to thread 26430
W20260924 14:29:15.731561 26221 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:15.746565 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:15.787213 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:15.802686 26221 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:15.817426 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:15.828894 23135 thread.cc:536] Waited for 4000ms trying to join with diag-logger (tid 26204)
W20260924 14:29:15.896911 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:15.908768 26295 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:15.914362 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:15.915746 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.051625 26295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111) [suppressed 427 similar messages]
W20260924 14:29:16.068780 26221 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:16.117542 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.121604 26295 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.120160 26369 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.127380 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.149196 26369 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.170535 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:16.186184 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.188454 26221 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:16.194236 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:16.213059 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.268548 26369 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.292503 26221 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.293218 26369 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.309904 26295 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.315785 26221 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.324625 26295 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:16.361999 26221 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:16.371210 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.383867 26295 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.421363 26369 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.424840 26369 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.457547 26295 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.505863 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:16.507819 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.583894 26221 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.622942 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.653141 26369 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.664613 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.772188 26221 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:16.829650 23135 thread.cc:536] Waited for 5000ms trying to join with diag-logger (tid 26204)
W20260924 14:29:17.605562 26295 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:29:17.830385 23135 thread.cc:536] Waited for 6000ms trying to join with diag-logger (tid 26204)
W20260924 14:29:17.910204 26295 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:17.950536 26295 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:18.023599 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:29:18.038455 23135 tablet_server.cc:194] TabletServer@127.22.151.193:0 shutdown complete.
W20260924 14:29:18.063192 26369 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:18.066872 26369 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:29:18.149325 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:29:18.161069 27113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:29:18.177248 27114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:29:18.201998 23135 server_base.cc:1034] running on GCE node
W20260924 14:29:18.213387 27116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:29:18.216050 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:29:18.216437 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:29:18.216648 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260158216636 us; error 0 us; skew 500 ppm
I20260924 14:29:18.217628 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260924 14:29:18.224299 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:18.227844 23135 webserver.cc:460] Webserver started at http://127.22.151.193:34861/ using document root <none> and password file <none>
I20260924 14:29:18.228804 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:29:18.229086 23135 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260924 14:29:18.241058 23135 fs_manager.cc:709] Time spent opening directory manager: real 0.011s	user 0.004s	sys 0.008s
W20260924 14:29:18.257688 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:29:18.259577 27122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:29:18.260672 23135 fs_manager.cc:725] Time spent opening block manager: real 0.009s	user 0.010s	sys 0.000s
I20260924 14:29:18.261067 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-0-root
uuid: "7e09fbebe29a49d291d23f9d395fe335"
format_stamp: "Formatted at 2026-09-24 14:28:55 on dist-test-slave-pm5x"
I20260924 14:29:18.261597 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260924 14:29:18.288540 26295 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.293879 26221 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:18.312342 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:29:18.314278 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:29:18.316754 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
W20260924 14:29:18.325382 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:29:18.328318 27129 ts_tablet_manager.cc:542] Loading tablet metadata (0/45 complete)
W20260924 14:29:18.397420 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:18.407621 26369 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.445627 26221 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:18.447412 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:18.479900 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.482921 26221 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.483642 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.518085 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:18.559003 26221 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:18.588625 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:18.648180 26369 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.651191 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.651273 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.676383 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.721833 26369 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:18.730943 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260924 14:29:18.731362 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.407s	user 0.013s	sys 0.000s
I20260924 14:29:18.731745 23135 ts_tablet_manager.cc:594] Registering tablets (0/45 complete)
W20260924 14:29:18.736754 26369 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:18.740211 27129 tablet_bootstrap.cc:492] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
W20260924 14:29:18.759552 26295 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.770187 26221 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:18.783396 27129 tablet_bootstrap.cc:492] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:18.784813 27129 tablet_bootstrap.cc:492] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:18.786202 27129 ts_tablet_manager.cc:1397] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.046s	user 0.018s	sys 0.013s
I20260924 14:29:18.789271 27129 raft_consensus.cc:348] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:18.791818 27129 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:18.792876 27129 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:18.806921 27129 ts_tablet_manager.cc:1428] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.020s	user 0.012s	sys 0.004s
I20260924 14:29:18.808096 27129 tablet_bootstrap.cc:492] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:18.847793 27129 tablet_bootstrap.cc:492] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:18.848955 27129 tablet_bootstrap.cc:492] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
W20260924 14:29:18.849558 26221 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:18.849821 27129 ts_tablet_manager.cc:1397] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.012s
I20260924 14:29:18.859174 27129 raft_consensus.cc:348] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:18.860317 27129 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:18.861290 27129 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:18.862728 26369 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:18.865676 27129 ts_tablet_manager.cc:1428] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260924 14:29:18.867087 27129 tablet_bootstrap.cc:492] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
W20260924 14:29:18.873080 26295 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:29:18.880312 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:18.886809 27129 tablet_bootstrap.cc:492] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:18.887990 27129 tablet_bootstrap.cc:492] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:18.888792 27129 ts_tablet_manager.cc:1397] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260924 14:29:18.891535 27129 raft_consensus.cc:348] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:18.892331 27129 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:18.893198 27129 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:18.895567 27129 ts_tablet_manager.cc:1428] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260924 14:29:18.896565 27129 tablet_bootstrap.cc:492] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:18.921732 27129 tablet_bootstrap.cc:492] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:18.922724 27129 tablet_bootstrap.cc:492] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:18.923475 27129 ts_tablet_manager.cc:1397] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.004s
I20260924 14:29:18.926278 27129 raft_consensus.cc:348] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:18.927263 27129 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:18.928191 27129 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:18.930644 27129 ts_tablet_manager.cc:1428] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:29:18.932154 27129 tablet_bootstrap.cc:492] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:18.955895 27129 tablet_bootstrap.cc:492] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:18.956915 27129 tablet_bootstrap.cc:492] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:18.957643 27129 ts_tablet_manager.cc:1397] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.004s
I20260924 14:29:18.960382 27129 raft_consensus.cc:348] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:18.961336 27129 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:18.962268 27129 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:18.965155 27129 ts_tablet_manager.cc:1428] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:29:18.966032 27129 tablet_bootstrap.cc:492] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
W20260924 14:29:18.974455 26295 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:18.986809 26295 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:18.991297 27129 tablet_bootstrap.cc:492] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:18.992466 27129 tablet_bootstrap.cc:492] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
W20260924 14:29:18.992733 26221 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:29:18.993156 27129 ts_tablet_manager.cc:1397] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.008s
I20260924 14:29:18.996929 27129 raft_consensus.cc:348] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:18.997797 27129 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:18.998961 27129 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.001616 27129 ts_tablet_manager.cc:1428] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260924 14:29:19.002470 27129 tablet_bootstrap.cc:492] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
W20260924 14:29:19.082358 26295 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:19.114470 26221 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:19.132154 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:19.154676 23135 ts_tablet_manager.cc:610] Registered 45 tablets
I20260924 14:29:19.166040 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.434s	user 0.335s	sys 0.000s
W20260924 14:29:19.167644 26369 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:19.168756 27129 tablet_bootstrap.cc:492] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.171144 27129 tablet_bootstrap.cc:492] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.171924 27129 ts_tablet_manager.cc:1397] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.170s	user 0.050s	sys 0.077s
W20260924 14:29:19.174527 26221 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:19.174427 27129 raft_consensus.cc:348] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.176429 27129 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.177793 27129 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.180747 27129 ts_tablet_manager.cc:1428] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260924 14:29:19.181586 27129 tablet_bootstrap.cc:492] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.208663 27129 tablet_bootstrap.cc:492] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.209754 27129 tablet_bootstrap.cc:492] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.210561 27129 ts_tablet_manager.cc:1397] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.010s
I20260924 14:29:19.213287 27129 raft_consensus.cc:348] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.214079 27129 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.214900 27129 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.217301 27129 ts_tablet_manager.cc:1428] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260924 14:29:19.218109 27129 tablet_bootstrap.cc:492] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.241122 27129 tablet_bootstrap.cc:492] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.242143 27129 tablet_bootstrap.cc:492] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.242827 27129 ts_tablet_manager.cc:1397] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.008s
I20260924 14:29:19.245364 27129 raft_consensus.cc:348] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.246114 27129 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.246894 27129 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.249495 27129 ts_tablet_manager.cc:1428] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260924 14:29:19.250317 27129 tablet_bootstrap.cc:492] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
W20260924 14:29:19.253806 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:19.274093 27129 tablet_bootstrap.cc:492] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.275081 27129 tablet_bootstrap.cc:492] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
W20260924 14:29:19.273895 26369 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:19.276582 27129 ts_tablet_manager.cc:1397] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.026s	user 0.021s	sys 0.004s
W20260924 14:29:19.277287 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:29:19.289179 27129 raft_consensus.cc:348] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.290119 27129 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.290964 27129 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.294865 27129 ts_tablet_manager.cc:1428] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260924 14:29:19.298971 27129 tablet_bootstrap.cc:492] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.329291 27129 tablet_bootstrap.cc:492] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.330343 27129 tablet_bootstrap.cc:492] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.331085 27129 ts_tablet_manager.cc:1397] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.003s
I20260924 14:29:19.336315 27129 raft_consensus.cc:348] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.337427 27129 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.338181 27129 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.340674 27129 ts_tablet_manager.cc:1428] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260924 14:29:19.341872 27129 tablet_bootstrap.cc:492] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.373179 27129 tablet_bootstrap.cc:492] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.374205 27129 tablet_bootstrap.cc:492] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.374989 27129 ts_tablet_manager.cc:1397] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.011s
I20260924 14:29:19.377601 27129 raft_consensus.cc:348] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.378705 27129 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.379530 27129 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.382074 27129 ts_tablet_manager.cc:1428] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260924 14:29:19.382974 27129 tablet_bootstrap.cc:492] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.405612 27129 tablet_bootstrap.cc:492] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.406621 27129 tablet_bootstrap.cc:492] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.407305 27129 ts_tablet_manager.cc:1397] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.003s
I20260924 14:29:19.409771 27129 raft_consensus.cc:348] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.410508 27129 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.411237 27129 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.413499 27129 ts_tablet_manager.cc:1428] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:29:19.414287 27129 tablet_bootstrap.cc:492] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.447727 27129 tablet_bootstrap.cc:492] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.448880 27129 tablet_bootstrap.cc:492] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.449616 27129 ts_tablet_manager.cc:1397] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.004s
I20260924 14:29:19.452240 27129 raft_consensus.cc:348] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.453044 27129 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.454198 27129 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.457125 27129 ts_tablet_manager.cc:1428] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:29:19.458458 27129 tablet_bootstrap.cc:492] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
W20260924 14:29:19.475380 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:19.491623 27129 tablet_bootstrap.cc:492] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.492744 27129 tablet_bootstrap.cc:492] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.493474 27129 ts_tablet_manager.cc:1397] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.035s	user 0.023s	sys 0.008s
I20260924 14:29:19.496282 27129 raft_consensus.cc:348] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.497084 27129 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.497798 27129 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.500222 27129 ts_tablet_manager.cc:1428] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260924 14:29:19.501082 27129 tablet_bootstrap.cc:492] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.524307 27129 tablet_bootstrap.cc:492] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.525353 27129 tablet_bootstrap.cc:492] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.526381 27129 ts_tablet_manager.cc:1397] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.004s
I20260924 14:29:19.529018 27129 raft_consensus.cc:348] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.529816 27129 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.530532 27129 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.533375 27129 ts_tablet_manager.cc:1428] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:29:19.534395 27129 tablet_bootstrap.cc:492] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.557197 27129 tablet_bootstrap.cc:492] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.558360 27129 tablet_bootstrap.cc:492] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.559190 27129 ts_tablet_manager.cc:1397] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.008s
I20260924 14:29:19.561878 27129 raft_consensus.cc:348] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.562848 27129 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.563726 27129 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.566138 27129 ts_tablet_manager.cc:1428] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:29:19.566905 27129 tablet_bootstrap.cc:492] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.589432 27129 tablet_bootstrap.cc:492] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.590510 27129 tablet_bootstrap.cc:492] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.591238 27129 ts_tablet_manager.cc:1397] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.007s
I20260924 14:29:19.593873 27129 raft_consensus.cc:348] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.594691 27129 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.595646 27129 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.598165 27129 ts_tablet_manager.cc:1428] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:29:19.599002 27129 tablet_bootstrap.cc:492] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.621167 27129 tablet_bootstrap.cc:492] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.622195 27129 tablet_bootstrap.cc:492] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.622905 27129 ts_tablet_manager.cc:1397] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.007s
I20260924 14:29:19.625506 27129 raft_consensus.cc:348] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.626271 27129 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.626984 27129 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.629787 27129 ts_tablet_manager.cc:1428] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260924 14:29:19.630733 27129 tablet_bootstrap.cc:492] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.653378 27129 tablet_bootstrap.cc:492] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.654485 27129 tablet_bootstrap.cc:492] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.655524 27129 ts_tablet_manager.cc:1397] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.008s
I20260924 14:29:19.658066 27129 raft_consensus.cc:348] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.658908 27129 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.659657 27129 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.662233 27129 ts_tablet_manager.cc:1428] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:29:19.663203 27129 tablet_bootstrap.cc:492] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.685525 27129 tablet_bootstrap.cc:492] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.686734 27129 tablet_bootstrap.cc:492] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.687513 27129 ts_tablet_manager.cc:1397] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.007s
I20260924 14:29:19.690084 27129 raft_consensus.cc:348] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.690969 27129 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.691824 27129 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.694173 27129 ts_tablet_manager.cc:1428] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260924 14:29:19.694983 27129 tablet_bootstrap.cc:492] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.717962 27129 tablet_bootstrap.cc:492] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.719127 27129 tablet_bootstrap.cc:492] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.720036 27129 ts_tablet_manager.cc:1397] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.007s
I20260924 14:29:19.722761 27129 raft_consensus.cc:348] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.723670 27129 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.724628 27129 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.727511 27129 ts_tablet_manager.cc:1428] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260924 14:29:19.728390 27129 tablet_bootstrap.cc:492] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.752063 27129 tablet_bootstrap.cc:492] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.753394 27129 tablet_bootstrap.cc:492] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.754243 27129 ts_tablet_manager.cc:1397] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.026s	user 0.023s	sys 0.000s
I20260924 14:29:19.757139 27129 raft_consensus.cc:348] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.758047 27129 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.758927 27129 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.761482 27129 ts_tablet_manager.cc:1428] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:29:19.762454 27129 tablet_bootstrap.cc:492] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.788992 27129 tablet_bootstrap.cc:492] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.790120 27129 tablet_bootstrap.cc:492] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.790971 27129 ts_tablet_manager.cc:1397] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.000s
I20260924 14:29:19.793586 27129 raft_consensus.cc:348] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.794368 27129 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.795061 27129 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:19.797608 27129 ts_tablet_manager.cc:1428] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260924 14:29:19.798483 27129 tablet_bootstrap.cc:492] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.823854 27129 tablet_bootstrap.cc:492] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.824884 27129 tablet_bootstrap.cc:492] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.825609 27129 ts_tablet_manager.cc:1397] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.008s
I20260924 14:29:19.829123 27129 raft_consensus.cc:348] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.830058 27129 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.831033 27129 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.833810 27129 ts_tablet_manager.cc:1428] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260924 14:29:19.834576 27129 tablet_bootstrap.cc:492] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.850793 27129 tablet_bootstrap.cc:492] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.852013 27129 tablet_bootstrap.cc:492] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.852671 27129 ts_tablet_manager.cc:1397] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.003s
I20260924 14:29:19.855400 27129 raft_consensus.cc:348] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.856338 27129 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.857257 27129 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:19.859582 27129 ts_tablet_manager.cc:1428] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260924 14:29:19.860448 27129 tablet_bootstrap.cc:492] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
W20260924 14:29:19.892755 26295 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:29:19.899194 27129 tablet_bootstrap.cc:492] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.900506 27129 tablet_bootstrap.cc:492] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.902154 27129 ts_tablet_manager.cc:1397] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.042s	user 0.031s	sys 0.004s
I20260924 14:29:19.906028 27129 raft_consensus.cc:348] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.906899 27129 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.907862 27129 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.910491 27129 ts_tablet_manager.cc:1428] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260924 14:29:19.911394 27129 tablet_bootstrap.cc:492] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.938153 27129 tablet_bootstrap.cc:492] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.940045 27129 tablet_bootstrap.cc:492] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.940783 27129 ts_tablet_manager.cc:1397] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.006s
I20260924 14:29:19.943321 27129 raft_consensus.cc:348] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.944180 27129 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:19.945384 27129 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:19.952179 27129 ts_tablet_manager.cc:1428] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260924 14:29:19.953212 27129 tablet_bootstrap.cc:492] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:19.996658 27129 tablet_bootstrap.cc:492] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:19.997851 27129 tablet_bootstrap.cc:492] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:19.998574 27129 ts_tablet_manager.cc:1397] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.018s
I20260924 14:29:20.001267 27129 raft_consensus.cc:348] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:20.002189 27129 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:20.002964 27129 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:20.005343 27129 ts_tablet_manager.cc:1428] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:29:20.006127 27129 tablet_bootstrap.cc:492] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:20.021533 27129 tablet_bootstrap.cc:492] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:20.022655 27129 tablet_bootstrap.cc:492] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:20.023401 27129 ts_tablet_manager.cc:1397] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260924 14:29:20.026136 27129 raft_consensus.cc:348] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:20.026995 27129 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:20.027760 27129 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:20.030320 27129 ts_tablet_manager.cc:1428] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:29:20.031354 27129 tablet_bootstrap.cc:492] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:20.040699 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.193:38943
I20260924 14:29:20.040903 27222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.193:38943 every 8 connection(s)
I20260924 14:29:20.060696 27132 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:20.061396 27132 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:20.100273 27132 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:20.161708 27132 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:20.162269 27132 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:20.162958 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:20.164712 27132 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:20.171224 27129 tablet_bootstrap.cc:492] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:20.172554 27129 tablet_bootstrap.cc:492] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:20.173310 27129 ts_tablet_manager.cc:1397] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.142s	user 0.026s	sys 0.003s
I20260924 14:29:20.171016 27190 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:20.176949 27129 raft_consensus.cc:348] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:20.177886 27129 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:20.178694 27129 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:20.179328 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:20.181267 27129 ts_tablet_manager.cc:1428] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260924 14:29:20.182864 27129 tablet_bootstrap.cc:492] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:20.183058 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:20.192565 27223 heartbeater.cc:347] Connected to a master server at 127.22.151.254:43789
I20260924 14:29:20.196667 27223 heartbeater.cc:464] Registering TS with master...
I20260924 14:29:20.197786 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:20.201376 27132 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:20.208631 27223 heartbeater.cc:511] Master 127.22.151.254:43789 requested a full tablet report, sending...
I20260924 14:29:20.212227 27129 tablet_bootstrap.cc:492] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:20.213660 27129 tablet_bootstrap.cc:492] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:20.214458 27129 ts_tablet_manager.cc:1397] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.007s
I20260924 14:29:20.218039 27129 raft_consensus.cc:348] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:20.219439 27129 raft_consensus.cc:729] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:20.220201 27129 consensus_queue.cc:260] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:20.238119 27129 ts_tablet_manager.cc:1428] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260924 14:29:20.239313 27129 tablet_bootstrap.cc:492] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:20.248185 26077 ts_manager.cc:194] Re-registered known tserver with Master: 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:20.420831 27234 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:20.421509 27234 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:20.422585 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 145305us
I20260924 14:29:20.423947 27234 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:20.427726 27189 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:20.439080 27190 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:20.447309 27194 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:20.457624 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:20.469578 27191 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:20.498225 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:20.500360 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:20.503068 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:20.504771 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:20.505645 27236 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:29:20.507175 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 69820.2us
I20260924 14:29:20.510401 27194 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:20.523624 27189 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:20.531117 27192 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:20.533300 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260924 14:29:20.612037 27186 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:20.612651 27182 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:20.628723 27129 tablet_bootstrap.cc:492] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:20.824259 27129 tablet_bootstrap.cc:492] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:20.825202 27129 ts_tablet_manager.cc:1397] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.586s	user 0.071s	sys 0.080s
I20260924 14:29:21.010179 27129 raft_consensus.cc:348] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:21.012449 27129 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:21.013428 27129 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:21.081744 27188 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:21.175303 27129 ts_tablet_manager.cc:1428] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.348s	user 0.016s	sys 0.004s
W20260924 14:29:21.171561 27255 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260924 14:29:21.176731 27256 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260924 14:29:21.195591 27189 consensus_meta.cc:287] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.671s	user 0.009s	sys 0.002s
W20260924 14:29:21.208355 27192 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.675s	user 0.013s	sys 0.000s
W20260924 14:29:21.210889 27186 consensus_meta.cc:287] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.598s	user 0.008s	sys 0.003s
I20260924 14:29:21.212662 27265 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:21.213300 27265 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:21.233775 27265 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:21.854841 26372 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.044s	user 0.065s	sys 0.139s
W20260924 14:29:21.855870 26372 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.047s	user 0.065s	sys 0.140s
I20260924 14:29:22.051962 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 14) took 1545 ms. Trace:
I20260924 14:29:22.052672 27194 rpcz_store.cc:276] 0924 14:29:20.506492 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.506760 (+   268us) service_pool.cc:227] Handling call
0924 14:29:20.510210 (+  3450us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:20.546631 (+ 36421us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:20.546664 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:20.547165 (+   501us) log.cc:842] Serialized 26 byte log entry
0924 14:29:20.587892 (+ 40727us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:21.127103 (+539211us) log.cc:842] Serialized 14 byte log entry
0924 14:29:22.050639 (+923536us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.050950 (+   311us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:22.051147 (+   197us) raft_consensus.cc:1671] finished
0924 14:29:22.051201 (+    54us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.051585 (+   384us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15650,"prepare.queue_time_us":219485,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":2703,"raft.queue_time_us":83943,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":206,"thread_start_us":382767,"threads_started":3,"wal-append.queue_time_us":29055}
I20260924 14:29:22.058389 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 11) took 1443 ms. Trace:
I20260924 14:29:22.058866 27194 rpcz_store.cc:276] 0924 14:29:20.614733 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:22.056269 (+1441536us) service_pool.cc:227] Handling call
0924 14:29:22.057046 (+   777us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:22.057802 (+   756us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:22.057866 (+    64us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:22.057981 (+   115us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:22.058032 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.058118 (+    86us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.058295 (+   177us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":190,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":302}
I20260924 14:29:22.062137 27194 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:22.072510 27181 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.019s	sys 0.011s
W20260924 14:29:22.074565 27181 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.020s	sys 0.012s
I20260924 14:29:22.076143 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 10) took 1461 ms. Trace:
I20260924 14:29:22.076694 27181 rpcz_store.cc:276] 0924 14:29:20.614359 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:21.166161 (+551802us) service_pool.cc:227] Handling call
0924 14:29:21.207769 (+ 41608us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:22.075332 (+867563us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:22.075411 (+    79us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:22.075557 (+   146us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:22.075659 (+   102us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.075778 (+   119us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.076044 (+   266us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3854,"raft.queue_time_us":33618,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":62,"thread_start_us":861587,"threads_started":1}
I20260924 14:29:22.080471 27259 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.081648 27259 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.089563 27258 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.090642 27258 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.091312 27260 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.092336 27260 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.094134 27258 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.094799 27260 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.112505 27259 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.114631 27278 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.115581 27278 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:22.116600 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 1.51152e+06us
I20260924 14:29:22.127792 27259 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.129041 27258 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.129071 27259 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.133459 27259 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.134042 27258 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.136684 27258 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.137219 27258 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.137490 27278 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.137977 27258 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.138572 27279 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.145773 27260 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.151620 27279 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.152230 27260 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:22.153012 27189 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.008s	sys 0.024s
W20260924 14:29:22.153855 27189 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.009s	sys 0.024s
I20260924 14:29:22.155195 27260 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.155701 27279 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.157334 27258 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:22.159981 27130 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.160072 27183 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.160913 27130 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.161206 27183 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.165325 27183 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.166157 27183 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.163491 27130 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.158911 27259 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.168254 27259 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.170677 27277 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.172758 27259 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:22.170905 27191 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.011s	sys 0.003s
I20260924 14:29:22.174827 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 13) took 1706 ms. Trace:
I20260924 14:29:22.158715 27278 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.179214 27278 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.167022 27183 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.182994 27268 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.184299 27278 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.169039 27279 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.193064 27279 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.200814 27280 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.202147 27280 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.205428 27280 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.206727 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.207895 26399 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.193602 27268 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.209668 27126 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.211146 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.211663 27268 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.175436 27191 rpcz_store.cc:276] 0924 14:29:20.468708 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.468893 (+   185us) service_pool.cc:227] Handling call
0924 14:29:20.469348 (+   455us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:20.505342 (+ 35994us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:20.505407 (+    65us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:20.516097 (+ 10690us) log.cc:842] Serialized 26 byte log entry
0924 14:29:20.615070 (+ 98973us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:21.169440 (+554370us) log.cc:842] Serialized 14 byte log entry
0924 14:29:22.174356 (+1004916us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.174504 (+   148us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:22.174562 (+    58us) raft_consensus.cc:1671] finished
0924 14:29:22.174578 (+    16us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.174771 (+   193us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":236939,"prepare.queue_time_us":324420,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":73385,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":162,"thread_start_us":1385196,"threads_started":3,"wal-append.queue_time_us":507362}
I20260924 14:29:22.178143 27277 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.214185 26401 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:22.217905 27238 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:21.198617 27129 tablet_bootstrap.cc:492] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:22.220619 27277 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.218757 27238 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.226320 27238 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.192377 27278 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.229071 27278 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.232298 27278 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.265470 27183 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:22.255134 27182 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.642s	user 0.018s	sys 0.023s
W20260924 14:29:22.267373 27188 consensus_meta.cc:287] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.172s	user 0.018s	sys 0.017s
W20260924 14:29:22.269848 27192 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.056s	user 0.001s	sys 0.012s
W20260924 14:29:22.269776 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 1.72428e+06us
I20260924 14:29:22.267697 27279 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:22.274068 27192 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.060s	user 0.003s	sys 0.013s
W20260924 14:29:22.274842 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65007.1us
I20260924 14:29:22.275727 27278 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.288038 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.288888 26399 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:22.289193 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 8) took 1675 ms. Trace:
I20260924 14:29:22.292253 27183 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.292660 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 9) took 1678 ms. Trace:
I20260924 14:29:22.293147 27183 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.300987 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.302800 26406 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:22.303092 27268 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.304015 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.304446 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.303948 27268 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.294822 27278 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.305063 26399 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.289898 27185 rpcz_store.cc:276] 0924 14:29:20.613714 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.665542 (+ 51828us) service_pool.cc:227] Handling call
0924 14:29:20.905986 (+240444us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:21.219665 (+313679us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:21.232356 (+ 12691us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:22.288297 (+1055941us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:22.288424 (+   127us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.288577 (+   153us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.288909 (+   332us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":275848,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":5484,"thread_start_us":230456,"threads_started":1}
I20260924 14:29:22.309425 27278 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.311169 27279 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.329306 27285 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.330394 27268 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.330163 27285 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.331512 27282 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:29:22.331954 26369 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.520s	user 0.094s	sys 0.107s
W20260924 14:29:22.332461 26369 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.521s	user 0.095s	sys 0.107s
I20260924 14:29:22.332350 27282 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:22.333039 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 1.52161e+06us
I20260924 14:29:22.335005 27268 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.336043 27268 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.338574 27268 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.333046 27285 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.339161 27268 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:22.339802 27285 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.338680 27245 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.352344 27245 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.353668 27284 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.354418 27284 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.293152 27190 rpcz_store.cc:276] 0924 14:29:20.614040 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.963652 (+349612us) service_pool.cc:227] Handling call
0924 14:29:21.025285 (+ 61633us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:22.291850 (+1266565us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b2c0009c918. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b76f5301 00007f09b770f8ba 00007f09b934781d 00007f09b934df6b 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:29:22.292029 (+   179us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:22.292066 (+    37us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:22.292128 (+    62us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:22.292182 (+    54us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.292261 (+    79us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.292475 (+   214us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":160040,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":10695,"spinlock_wait_cycles":4838144,"thread_start_us":77920,"threads_started":1}
I20260924 14:29:22.356779 27283 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.357547 27283 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.359674 27249 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.360432 27249 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.362677 27249 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:22.363234 27258 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.363410 27234 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:22.363966 27258 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.366406 27258 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.345505 27111 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.367662 27111 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.370740 27111 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.356817 27284 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:22.372149 27234 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.378348 27236 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.379807 27281 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.380616 27236 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.380771 27281 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.378934 27280 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.384001 27280 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.355155 27245 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.347344 27268 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:22.398782 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 94424.5us
I20260924 14:29:22.401505 27183 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.401979 27283 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:22.391274 27249 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.417248 26079 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55200 (request call id 121) took 1879 ms. Trace:
I20260924 14:29:22.402215 27236 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.417395 27249 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.417768 26079 rpcz_store.cc:276] 0924 14:29:20.537445 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.537594 (+   149us) service_pool.cc:227] Handling call
0924 14:29:22.416979 (+1879385us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1554773,"reported_tablets":1}
I20260924 14:29:22.402031 27234 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.419852 27249 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.421015 27234 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.402683 27280 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.406627 27132 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.391784 27281 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.407160 27290 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.405184 27268 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:22.422756 27132 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.407752 27248 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.407528 27285 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.396749 27282 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:22.349395 27279 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.423074 27290 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.414537 27278 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.436460 27281 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.457616 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.457432 27248 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.458472 26401 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:22.460216 27248 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.460464 27290 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.461318 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.463213 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.464287 27118 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.479787 27118 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.481662 27263 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.482528 27118 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:22.482403 27263 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.483827 27133 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.454183 27234 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.484687 27133 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.465211 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.465464 27246 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.454872 27111 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.465720 27244 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.466213 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.467312 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.468458 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.469926 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.470281 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.464784 27283 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.513242 27283 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.485126 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.514832 26401 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.486665 27285 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.469978 27279 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.453178 27280 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.511620 26405 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.515897 27283 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.516042 27280 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.516974 27244 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.518311 27280 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.493036 27248 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.493278 26399 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.493832 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 10) took 1970 ms. Trace:
I20260924 14:29:22.457532 27292 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.492259 27190 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:22.484752 27281 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.487248 27245 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.535879 27297 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.536162 27245 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.536650 27297 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.537089 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.537405 27133 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.538522 27245 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:22.485948 27234 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.538944 27297 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:22.539675 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 76562us
I20260924 14:29:22.540361 27263 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.508229 27249 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.509624 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.540598 27234 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.541877 27285 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:22.541038 27111 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.567418 27133 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.570263 27111 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.540421 27126 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.567773 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.572017 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.572284 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.572785 27190 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:22.572978 27181 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:22.573855 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.574043 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.574564 27190 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:22.565560 27294 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.576148 27252 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.576107 27294 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.576761 27281 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.564662 27272 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.564901 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.565552 27132 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.565665 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 12) took 2041 ms. Trace:
I20260924 14:29:22.566475 27301 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.566606 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.566987 27292 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.554880 27248 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.555325 27282 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.568835 27278 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.568449 27285 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.568514 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.575379 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.576893 27252 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:22.612905 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 103762us
I20260924 14:29:22.614720 27266 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.586990 27290 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:22.538957 27246 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.622330 27302 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.568303 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.588171 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.570257 27133 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.593871 27183 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:22.610837 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 67387.2us
I20260924 14:29:22.624257 27285 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:22.589497 27249 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.642777 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.647080 27191 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:22.648154 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.648891 27133 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:22.651115 27298 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260924 14:29:22.652598 27133 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.636524 27302 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.635982 27190 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:22.635674 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.637344 27295 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.660243 27294 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.631889 27290 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.640616 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.658627 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.625257 27272 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.647414 27248 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.651249 27277 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.643260 27244 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.666766 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.589182 27192 rpcz_store.cc:276] 0924 14:29:20.524374 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.530332 (+  5958us) service_pool.cc:227] Handling call
0924 14:29:20.530817 (+   485us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:22.278615 (+1747798us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:22.278674 (+    59us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:22.279420 (+   746us) log.cc:842] Serialized 26 byte log entry
0924 14:29:22.312508 (+ 33088us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:22.512890 (+200382us) log.cc:842] Serialized 14 byte log entry
0924 14:29:22.532575 (+ 19685us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.542104 (+  9529us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:22.545057 (+  2953us) raft_consensus.cc:1671] finished
0924 14:29:22.547791 (+  2734us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.565601 (+ 17810us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9146,"prepare.queue_time_us":25683,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":50821,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":7426,"thread_start_us":1274052,"threads_started":3,"wal-append.queue_time_us":64263}
I20260924 14:29:22.667490 27259 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.635849 27282 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.652904 27288 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.672646 27313 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.663834 27133 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.672919 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.673301 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 0) took 2054 ms. Trace:
I20260924 14:29:22.673534 27246 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.629618 27301 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.698779 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.699086 27301 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.673952 27300 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.675601 27290 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.660389 27295 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.677403 27292 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:22.680076 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 14) took 1872 ms. Trace:
I20260924 14:29:22.680238 27126 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.712932 27126 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.709578 26327 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.714138 27126 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.709473 27300 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.715734 27281 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:22.716217 27281 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.716611 27295 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.716787 27278 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.709672 27194 rpcz_store.cc:276] 0924 14:29:20.822236 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:22.060573 (+1238337us) service_pool.cc:227] Handling call
0924 14:29:22.061791 (+  1218us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:22.129970 (+ 68179us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:22.130812 (+   842us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:22.181005 (+ 50193us) log.cc:842] Serialized 26 byte log entry
0924 14:29:22.518415 (+337410us) spinlock_profiling.cc:243] Waited 6.03 ms on lock 0x7b1800347b00. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:29:22.612364 (+ 93949us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:22.640684 (+ 28320us) log.cc:842] Serialized 14 byte log entry
0924 14:29:22.655238 (+ 14554us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.657626 (+  2388us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:22.670561 (+ 12935us) raft_consensus.cc:1671] finished
0924 14:29:22.679767 (+  9206us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.679991 (+   224us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":38720,"prepare.queue_time_us":17937,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":65163,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":11753,"spinlock_wait_cycles":13863168,"thread_start_us":96420,"threads_started":2,"wal-append.queue_time_us":485049}
I20260924 14:29:22.681846 27249 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.682422 27185 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:22.726644 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.727969 27132 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.683076 27252 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.728700 27132 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.729146 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.729617 27252 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.729964 26251 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.730265 27252 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.730921 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.661141 27258 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.683801 27277 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.651746 27183 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.685460 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.657124 27302 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.641306 27266 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.689275 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.689174 27259 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.671865 26328 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.693296 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:22.694814 27303 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260924 14:29:22.695250 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.747159 27259 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.737002 27277 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.748664 27129 tablet_bootstrap.cc:492] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:22.699700 27312 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.754056 27312 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.695834 27186 rpcz_store.cc:276] 0924 14:29:20.610892 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.611359 (+   467us) service_pool.cc:227] Handling call
0924 14:29:20.611736 (+   377us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:22.321911 (+1710175us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:22.328335 (+  6424us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:22.381680 (+ 53345us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b1800217b40. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:29:22.441308 (+ 59628us) log.cc:842] Serialized 26 byte log entry
0924 14:29:22.543907 (+102599us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:22.544339 (+   432us) log.cc:842] Serialized 14 byte log entry
0924 14:29:22.563551 (+ 19212us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.594516 (+ 30965us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:22.651649 (+ 57133us) raft_consensus.cc:1671] finished
0924 14:29:22.654904 (+  3255us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.664016 (+  9112us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":32851,"prepare.queue_time_us":193,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":2359,"raft.queue_time_us":49981,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":4159,"spinlock_wait_cycles":2341884672,"thread_start_us":115163,"threads_started":2,"wal-append.queue_time_us":181348}
I20260924 14:29:22.700484 27308 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.697418 27133 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.759780 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.708762 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 2) took 2097 ms. Trace:
I20260924 14:29:22.680852 26326 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
W20260924 14:29:22.709856 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 155710us
I20260924 14:29:22.717231 27278 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.692027 27294 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.716620 27281 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:22.764416 27294 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.765130 27133 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.765743 27133 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.717334 27282 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:22.759778 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.531806 27189 rpcz_store.cc:276] 0924 14:29:20.522972 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.523094 (+   122us) service_pool.cc:227] Handling call
0924 14:29:20.523428 (+   334us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:22.154550 (+1631122us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:22.154600 (+    50us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:22.177055 (+ 22455us) log.cc:842] Serialized 26 byte log entry
0924 14:29:22.189091 (+ 12036us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:22.276462 (+ 87371us) log.cc:842] Serialized 14 byte log entry
0924 14:29:22.441655 (+165193us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.450012 (+  8357us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:22.451355 (+  1343us) raft_consensus.cc:1671] finished
0924 14:29:22.452402 (+  1047us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.493762 (+ 41360us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31877,"prepare.queue_time_us":116065,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":131006,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":199,"thread_start_us":1140768,"threads_started":4,"wal-append.queue_time_us":57052}
I20260924 14:29:22.731681 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.732018 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.733899 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.734189 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.735962 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.736229 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.772934 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.737067 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.737089 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.737810 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.738798 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.731216 27132 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.739470 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.798902 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:22.745908 26329 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:22.800724 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.800910 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.801679 27194 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:22.802021 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.745935 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.738615 27183 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.747431 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 13) took 2142 ms. Trace:
I20260924 14:29:22.803737 26328 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:22.739934 27258 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.804584 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.804836 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.805713 27258 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.803859 27188 rpcz_store.cc:276] 0924 14:29:20.610279 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.619584 (+  9305us) service_pool.cc:227] Handling call
0924 14:29:20.826167 (+206583us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:22.307315 (+1481148us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:22.310719 (+  3404us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:22.339575 (+ 28856us) log.cc:842] Serialized 26 byte log entry
0924 14:29:22.369142 (+ 29567us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b48001662d8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:29:22.448993 (+ 79851us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:22.457490 (+  8497us) log.cc:842] Serialized 14 byte log entry
0924 14:29:22.479133 (+ 21643us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.479229 (+    96us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:22.747109 (+267880us) raft_consensus.cc:1671] finished
0924 14:29:22.747221 (+   112us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.747351 (+   130us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":37536,"prepare.queue_time_us":5643,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":59034,"raft.run_cpu_time_us":2853,"raft.run_wall_time_us":88742,"spinlock_wait_cycles":17889664,"thread_start_us":100692,"threads_started":1,"wal-append.queue_time_us":190023}
I20260924 14:29:22.806334 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.743757 26324 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.748080 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.752316 27266 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.753289 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.807641 27300 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.695363 27288 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:22.808275 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 55185us
I20260924 14:29:22.734853 27290 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.759230 27312 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.760660 27129 tablet_bootstrap.cc:492] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:22.815021 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:22.815379 27288 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.815599 27129 ts_tablet_manager.cc:1397] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 1.617s	user 0.028s	sys 0.017s
I20260924 14:29:22.815810 27251 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.760082 27308 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.762771 27278 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.803325 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.762861 27252 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.763628 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.719952 27272 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.770833 26332 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:22.768282 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.773005 27069 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.771705 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.765178 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.773247 27287 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.767242 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.779975 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.787534 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.780431 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.793457 26251 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.791636 26258 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.836987 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.804180 27268 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:22.850741 27069 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.851841 27181 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:22.794425 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.794565 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.769704 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.793515 27311 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.787168 26400 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:22.795921 27133 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.786970 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.820266 27132 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:22.863314 27308 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.859424 27311 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.761593 27182 rpcz_store.cc:276] 0924 14:29:20.611668 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.611930 (+   262us) service_pool.cc:227] Handling call
0924 14:29:20.612355 (+   425us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:22.286635 (+1674280us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:22.296784 (+ 10149us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:22.316176 (+ 19392us) log.cc:842] Serialized 26 byte log entry
0924 14:29:22.400265 (+ 84089us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:22.400714 (+   449us) log.cc:842] Serialized 14 byte log entry
0924 14:29:22.401195 (+   481us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:22.401336 (+   141us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:22.708528 (+307192us) raft_consensus.cc:1671] finished
0924 14:29:22.708557 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:22.708697 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12025,"prepare.queue_time_us":225,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":16603,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":21201,"thread_start_us":27600,"threads_started":1,"wal-append.queue_time_us":101759}
I20260924 14:29:22.798404 27302 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.703630 27313 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.800786 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.715202 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.773061 27310 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.871249 26257 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
W20260924 14:29:22.871402 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 156276us
I20260924 14:29:22.852423 26253 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.823634 27319 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.852021 26397 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:22.824362 27290 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.807642 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.855144 27191 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:22.878664 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.878979 27313 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.879372 26400 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:22.879740 27290 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.855103 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.880826 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.880998 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.882287 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.882709 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.827076 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.883721 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.826107 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.884055 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.766592 27294 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:22.779718 27309 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.868359 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:22.885015 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 43968.9us
I20260924 14:29:22.782189 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.848078 26077 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34948
I20260924 14:29:22.860231 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.816413 27251 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:22.865900 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46246.5us
I20260924 14:29:22.832391 27129 raft_consensus.cc:348] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.870595 27311 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.870421 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.882319 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.873472 27132 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.890908 27132 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.891009 27186 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:22.891353 27132 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.893858 26077 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 1) took 2650 ms. Trace:
I20260924 14:29:22.894244 26077 rpcz_store.cc:276] 0924 14:29:20.243553 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:20.243708 (+   155us) service_pool.cc:227] Handling call
0924 14:29:22.893752 (+2650044us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1166568,"reported_tablets":45}
I20260924 14:29:22.850083 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.890928 26397 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.874429 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.855669 27287 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.875885 26325 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:22.873577 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.877471 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:22.875964 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.816970 27266 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.898088 26256 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.888023 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:22.899732 27266 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.900278 27266 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.900288 27234 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.900815 27234 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:22.901213 27234 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.884438 26406 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:22.898715 26405 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.875113 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.885776 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.884965 27309 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.889086 27129 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:22.890280 26324 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:22.877400 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.898576 26404 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.885418 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.909911 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.889706 27069 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.912701 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.913028 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:22.890072 27272 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:22.889366 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:22.917402 27259 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.917522 27309 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.919003 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.921125 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.906795 27129 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:22.906996 27251 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.907786 27308 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.919644 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:22.926687 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.929725 27263 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.930243 27263 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.930612 27263 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.921926 27259 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.930883 27125 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.931280 27259 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.931548 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.932626 27258 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.933015 27258 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.933362 27258 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.934335 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.935674 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:22.936335 27280 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.936420 26397 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:22.936676 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.936952 27129 ts_tablet_manager.cc:1428] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.121s	user 0.007s	sys 0.000s
I20260924 14:29:22.936892 27280 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:22.937403 27280 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.871662 27310 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.944597 27129 tablet_bootstrap.cc:492] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:22.945499 27310 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.945871 26404 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:22.946563 26254 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:22.946210 27308 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.947085 27266 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.949404 27266 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.918529 27272 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.952932 27272 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:22.952303 27126 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.954427 27126 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:22.955309 27279 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.955560 27296 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:22.955739 27279 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.956207 27279 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:22.959009 27183 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:22.959532 27183 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.959915 27183 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:22.961390 27263 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.946094 27223 heartbeater.cc:503] Master 127.22.151.254:43789 was elected leader, sending a full tablet report...
I20260924 14:29:22.965561 27263 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.967434 27123 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:22.955760 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.962873 27280 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:22.956076 27296 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:22.971032 27296 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:22.924100 27319 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:22.974812 26369 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:22.948761 27308 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:22.975945 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.019843 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:22.980991 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:22.989301 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.021572 27302 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.021708 27266 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.019629 27263 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:22.988826 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.079788 27266 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.079906 27263 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.080317 27266 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.080442 27263 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.019716 27268 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:22.986665 27132 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:23.058801 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 103119us
I20260924 14:29:23.033254 27281 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.086467 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.087466 27126 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:22.979238 26369 leader_election.cc:341] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:23.088327 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 109127us
W20260924 14:29:23.089128 27219 debug-util.cc:398] Leaking SignalData structure 0x7b080011c980 after lost signal to thread 23138
I20260924 14:29:23.089244 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:23.089658 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.090276 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:23.070760 26252 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:23.070568 27259 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.071586 27280 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:23.071835 27285 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.072671 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:23.092743 26369 leader_election.cc:341] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:23.075712 27290 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:23.068362 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 101099us
I20260924 14:29:23.082435 26404 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:22.980281 27277 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.080657 27296 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.087651 27281 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:23.090406 27219 debug-util.cc:398] Leaking SignalData structure 0x7b08000f33a0 after lost signal to thread 26115
I20260924 14:29:22.980295 27319 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:22.977306 27287 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:23.109015 26298 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:23.091418 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:22.983433 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:22.977372 27279 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.098336 27313 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.115614 27313 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.116073 27313 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.116503 27279 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.098711 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.098510 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.098999 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.092384 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.118947 26254 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:23.101039 27259 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.119668 27282 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.101997 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.120198 27282 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.120669 27282 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.094111 27260 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.092301 27249 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:23.123625 27260 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:23.124013 27249 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:23.124473 27249 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:23.126995 27132 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.129726 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.130496 27194 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:23.112242 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.132689 27296 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.095490 27268 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:23.133559 27268 raft_consensus.cc:686] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:23.134510 27268 consensus_queue.cc:237] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.134919 27278 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.079322 27272 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.116266 27319 consensus_queue.cc:1046] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:23.098492 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.102701 27277 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.103267 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.124073 27260 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.099403 27258 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.136598 27194 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.145844 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.146538 27192 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:23.147079 27258 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:23.108878 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:23.137508 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:23.149093 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:23.149979 27279 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.138558 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.144935 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:23.150400 27279 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:23.142897 27277 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.152168 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:23.154796 27258 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.155366 27258 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.155915 27258 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.156045 27294 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.156420 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.150980 26402 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.150327 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:23.142568 27132 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.155845 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.161895 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:23.138664 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 155296us
I20260924 14:29:23.167824 27132 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:23.164579 27285 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.151086 27279 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.165125 27319 raft_consensus.cc:3032] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:23.157125 27192 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
W20260924 14:29:23.162940 26369 leader_election.cc:341] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:23.160032 27296 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.157596 27294 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.183763 27294 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.183650 27279 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.186002 27272 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.155771 27290 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.171712 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:23.187997 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.188431 27290 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:23.173638 27319 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:23.189087 27290 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:23.189778 27290 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:23.169801 27321 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.190526 27321 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:23.190253 27319 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.190969 27321 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.176844 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.174244 27296 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:23.177459 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.180004 27191 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.181233 27281 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:23.204254 27281 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:23.190024 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:23.204769 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.205797 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.206444 27283 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.206871 27283 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:23.207217 27283 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.208248 27278 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.201846 27296 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.208864 27278 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.209221 27278 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.209568 27278 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.142249 27263 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:23.216346 27263 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:23.172590 27219 debug-util.cc:398] Leaking SignalData structure 0x7b08002b75a0 after lost signal to thread 26282
I20260924 14:29:23.220052 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:23.220638 27219 debug-util.cc:398] Leaking SignalData structure 0x7b0800328860 after lost signal to thread 26356
I20260924 14:29:23.201193 27313 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.204787 27281 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:23.221793 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.222491 27313 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.222879 27183 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.191709 27319 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.220234 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.224195 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.225028 27183 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.227022 27191 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.228256 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.229159 27194 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:23.229761 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.230067 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.230530 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.230893 26252 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.233045 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:23.221822 27219 debug-util.cc:398] Leaking SignalData structure 0x7b08002a6420 after lost signal to thread 26430
I20260924 14:29:23.233696 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.233768 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.217003 27279 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.234349 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.235024 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:23.234278 27219 debug-util.cc:398] Leaking SignalData structure 0x7b08002427e0 after lost signal to thread 27042
I20260924 14:29:23.237236 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.237577 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:23.237305 27219 debug-util.cc:398] Leaking SignalData structure 0x7b080020f5c0 after lost signal to thread 27222
I20260924 14:29:23.175612 27132 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.239163 27249 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.241075 27249 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:23.242264 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.243355 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.244943 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.245088 27282 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.246603 26325 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:23.247638 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.248087 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.249156 26324 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.250306 27194 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:23.237989 27281 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.252218 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.252607 27191 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:23.253875 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.254689 26404 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:23.254657 27192 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
W20260924 14:29:23.256465 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 79098us
I20260924 14:29:23.256590 26402 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:23.257504 27281 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.257534 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:23.259064 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:23.246096 27132 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:23.260967 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.274621 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.275221 26404 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.265362 26252 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:23.284315 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.265676 27294 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.266587 27259 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:23.267590 27282 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.268543 27234 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.260334 27263 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:23.283231 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.288401 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.283661 27248 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:23.283142 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.272082 27281 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:23.286676 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.287632 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:23.292059 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.293056 27123 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:23.296098 27234 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.298250 27125 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:23.305027 27296 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.296133 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.324224 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:23.307638 27283 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.313300 26404 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:23.316126 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.316555 26249 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.316613 27248 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.328756 27294 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.329109 27248 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.329577 27248 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.329448 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.330072 26397 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:23.330987 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.318044 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.318717 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.319118 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.347348 27258 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.318724 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.319942 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.348836 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.320624 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.321321 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.326625 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.321784 26402 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.327726 27183 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:23.328130 27259 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.317770 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:23.333743 27234 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:23.355902 27234 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:23.356066 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.356873 27234 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:23.333587 27321 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.329362 27277 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.335637 27281 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.353332 27309 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260924 14:29:23.360765 27281 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:23.361230 27281 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.361394 27309 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:23.361828 27277 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.318238 26369 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:23.338824 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.341149 27263 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:23.362699 26401 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:29:23.337301 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 131255us
I20260924 14:29:23.341619 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.342628 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.342999 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.364220 26403 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:23.364457 26397 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:23.353562 27259 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:23.343122 26325 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.365530 27259 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.349346 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.344381 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.365283 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.344455 27132 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:23.345894 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:23.370015 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 51848.9us
I20260924 14:29:23.321936 27278 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:23.350008 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.353916 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.361788 27309 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:23.363063 27263 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.376782 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.378124 27263 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.378675 27263 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:23.378786 26295 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:23.341388 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.339112 27266 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.376279 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:23.385259 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:23.372876 27249 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:23.379163 27263 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:23.390939 26369 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.390964 26295 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:23.380887 27277 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.392982 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:23.385687 27282 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:23.398034 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.393337 27319 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260924 14:29:23.391925 27310 consensus_queue.cc:1046] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260924 14:29:23.400394 27319 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:23.400774 27319 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:23.400825 27310 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:23.401301 27310 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:23.401640 27319 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.402086 27310 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.393410 27130 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.402946 27319 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:23.403090 27130 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:23.403357 27310 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.402555 27246 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:23.393343 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:23.403215 27312 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.407605 27126 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.407610 27252 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260924 14:29:23.387516 27308 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.410188 27312 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.387295 27313 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.393488 27295 consensus_queue.cc:1046] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260924 14:29:23.416136 27321 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.416424 26404 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.401171 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:23.418354 27252 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:23.429607 27252 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:23.429980 27295 raft_consensus.cc:3032] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:23.430413 27295 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:23.430639 27252 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.431239 27295 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.432068 27252 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.421471 27313 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:23.412827 27276 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.432544 27295 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.403673 27130 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.429214 27191 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:23.433027 27313 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.415421 27277 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.419207 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:23.416174 27258 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.421746 27312 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.407971 27309 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.421010 27260 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.445021 27129 tablet_bootstrap.cc:492] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:23.444661 27248 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.427117 27132 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:23.448406 27281 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.419889 27282 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260924 14:29:23.450258 26403 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:23.452843 26404 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.459523 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.461689 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.462307 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:23.440265 27276 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:23.439836 27278 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:23.467180 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 74253.2us
I20260924 14:29:23.422546 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.476876 26298 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:23.474954 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.466532 27296 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:23.473287 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 75314.6us
I20260924 14:29:23.475826 26405 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.480216 27129 tablet_bootstrap.cc:492] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:23.480855 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.481741 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.493520 27296 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.471653 27283 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:23.498696 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 87266.5us
I20260924 14:29:23.474190 27266 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.482934 27309 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.467489 27263 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.493990 26258 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.537563 27129 ts_tablet_manager.cc:1397] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.593s	user 0.028s	sys 0.025s
I20260924 14:29:23.499259 26406 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:23.496016 26404 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.477056 27132 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:23.496981 27281 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.540764 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:23.475436 27294 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:29:23.582648 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 100982us
I20260924 14:29:23.587641 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.477212 27276 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.482391 27324 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:23.497308 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.537416 27282 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.494186 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.498715 27260 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:23.497634 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.495417 27248 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:23.493772 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.589202 27330 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.481492 27290 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:23.592124 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.618213 27132 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.620342 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.602736 27324 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:23.584311 27126 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.627516 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.627660 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:23.607054 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:23.631664 26404 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:23.611003 27281 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:29:23.636165 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 155356us
I20260924 14:29:23.609735 27328 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:29:23.636889 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45228us
I20260924 14:29:23.644364 27292 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:23.644850 27292 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:23.645784 27292 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:23.647074 27292 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.648129 26406 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.648434 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:23.648661 27234 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:23.649416 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:23.649760 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:23.639583 26295 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:23.651186 27324 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.652413 26257 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:23.652747 26295 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.653468 27328 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.654335 27192 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.654840 26295 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.622210 27260 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260924 14:29:23.655661 27328 consensus_queue.cc:1046] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:23.655886 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 71669.2us
I20260924 14:29:23.654753 27276 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.656658 27328 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.656903 27126 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.657202 27328 raft_consensus.cc:3032] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:23.657742 27328 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:23.657997 27276 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:23.658238 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.659037 27234 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.659673 27234 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.659857 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:23.660984 26402 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:23.616590 27332 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.658504 27328 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.628193 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.671092 26402 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.668315 27129 raft_consensus.cc:348] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.689316 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.690236 27191 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:23.691020 26401 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 2: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:23.688838 27129 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:23.673676 26295 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.668053 27183 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.701081 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.693665 27129 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:23.706192 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 118643us
I20260924 14:29:23.687527 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:23.709097 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.662009 27281 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:23.706081 26295 leader_election.cc:341] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:23.721789 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.716723 26404 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:23.717243 26402 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.670291 27294 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260924 14:29:23.720741 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.688611 27295 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260924 14:29:23.708015 27282 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:23.735638 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:23.718287 27332 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.734571 26406 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:23.734301 27281 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:23.726311 26403 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.697161 27296 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
W20260924 14:29:23.815241 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 87002.8us
I20260924 14:29:23.730005 27328 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.808036 26400 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.728226 27246 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.725946 27283 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.737948 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.835156 27129 ts_tablet_manager.cc:1428] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.297s	user 0.023s	sys 0.015s
I20260924 14:29:23.835798 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:23.736855 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.806557 27277 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:23.844095 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 106295us
I20260924 14:29:23.867036 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.841169 27192 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.852057 27258 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:23.870325 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:23.871739 27341 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:23.843992 27336 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260924 14:29:23.809864 27259 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.806304 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.829881 27337 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
I20260924 14:29:23.894732 26369 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:23.856855 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.866561 27290 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:23.890647 27276 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.903311 27258 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:29:23.914716 27339 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260924 14:29:23.915715 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.895364 27341 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:23.919160 27290 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:23.924465 27290 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:23.925459 27290 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:23.920496 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.928244 27294 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:23.928385 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.928689 27294 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:29:23.914783 27310 consensus_meta.cc:287] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.000s
I20260924 14:29:23.929611 27294 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:23.928719 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:23.932159 27186 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.932370 27194 raft_consensus.cc:2379] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:23.937862 27182 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.941596 27259 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:23.950965 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.956470 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:23.957445 27192 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:23.958441 27341 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.959554 27186 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.961812 27186 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
W20260924 14:29:23.962719 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 233312us
I20260924 14:29:23.964004 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:23.964375 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:23.966253 27130 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.512s	user 0.006s	sys 0.004s
I20260924 14:29:23.966837 27296 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260924 14:29:23.971503 27312 consensus_meta.cc:287] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.005s
I20260924 14:29:23.971999 27312 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:23.973130 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:23.973388 27259 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:23.974123 27312 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:23.977165 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:23.978816 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:23.938831 27313 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.979673 27190 raft_consensus.cc:2379] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:23.974427 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.974902 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:23.982268 27313 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:23.931181 27294 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:23.975021 27258 raft_consensus.cc:3032] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:23.983151 27258 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:23.984086 27191 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:23.983949 27258 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:23.966874 27130 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:23.988570 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:23.992094 27332 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.992702 27130 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:23.995283 27130 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:23.997139 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:23.998808 27130 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:24.004153 26369 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:24.003382 27341 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.007680 27248 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.009406 27248 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:24.010747 27341 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:24.010354 27248 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:24.011799 27248 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.012625 27130 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.019810 27181 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:24.025676 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.027272 27310 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:24.028379 27310 raft_consensus.cc:3032] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.028694 27310 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.029824 27319 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.031613 27194 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.033401 27259 consensus_queue.cc:1046] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:24.034499 27277 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.034866 27277 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:24.035727 27277 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.037436 27277 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.039050 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:24.037653 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.040285 27309 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:24.039976 27310 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:24.041213 27310 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.039301 27194 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
W20260924 14:29:24.042009 26221 leader_election.cc:341] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:24.043984 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.044850 27309 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.045884 27194 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:24.046259 27309 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:24.047686 27308 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.048065 27308 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.048151 27130 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:24.049168 27248 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:24.048945 27308 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:24.049552 27248 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:29:24.052963 26221 leader_election.cc:341] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:24.053520 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.054291 27069 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:29:24.055347 26221 leader_election.cc:341] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:24.058682 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.058146 27319 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.050876 27248 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.060467 27248 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:24.060910 26397 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.062398 27336 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:24.062948 27336 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:24.063074 27069 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:24.063822 27336 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:24.064427 26221 leader_election.cc:341] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:24.069710 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:24.069912 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.070974 27319 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:24.072621 27319 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:24.072746 26221 leader_election.cc:341] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:24.073613 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:24.076494 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:24.073843 27069 raft_consensus.cc:3032] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.078632 27069 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.079490 27069 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:24.080787 27069 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.080797 26325 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.085127 26399 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.091876 26324 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.324390 27337 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:24.073925 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.325038 27337 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.327193 27337 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:24.328904 27337 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:24.330108 27348 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.330493 27348 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:24.331380 27348 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:24.340093 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 267355us
I20260924 14:29:24.342253 27354 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.349186 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.350677 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.348855 27353 consensus_queue.cc:1046] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:24.354719 27129 tablet_bootstrap.cc:492] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:24.358894 27355 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:24.360569 27353 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.361578 27353 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:24.362475 27353 raft_consensus.cc:3032] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.362943 27353 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.364240 27356 consensus_queue.cc:1046] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:24.366447 26251 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.368031 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:24.365428 27356 raft_consensus.cc:3032] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:24.368391 27353 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:24.369966 27353 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.370915 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:24.369124 27356 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.370913 27309 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.371711 27309 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.372493 27309 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:24.373493 27309 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.373833 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:24.375329 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:24.380867 27356 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:24.382273 27356 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:24.381392 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:24.387627 26324 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.388821 27287 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:24.389844 27287 consensus_queue.cc:1225] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260924 14:29:24.390425 27287 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.390759 27287 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.391994 27287 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:24.393584 27287 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.395980 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:24.396845 26258 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.403519 26251 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:24.416327 27355 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:24.417184 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:24.408026 26252 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:24.417866 27355 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:24.418264 27355 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.417248 27129 tablet_bootstrap.cc:492] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:24.419993 27309 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.419004 27355 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:24.419011 26224 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.421136 27355 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:24.422206 27129 tablet_bootstrap.cc:492] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:24.424957 27129 ts_tablet_manager.cc:1397] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.070s	user 0.018s	sys 0.003s
I20260924 14:29:24.422716 27309 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.421928 26224 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.431190 26224 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.434583 27309 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.435423 27309 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.438984 27129 raft_consensus.cc:348] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.439520 26258 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:24.440059 27129 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:24.441349 27129 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.443866 26224 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.448923 27355 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.451311 27355 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:24.452454 27355 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.452847 27355 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.453730 27355 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:24.455390 27355 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.458206 27188 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.460140 26224 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.462635 26224 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.464116 27129 ts_tablet_manager.cc:1428] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260924 14:29:24.465197 27129 tablet_bootstrap.cc:492] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:24.471025 27308 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.474882 27358 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.475646 27358 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.479372 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.480707 27308 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.482556 27358 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:24.483035 27358 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:24.485126 27358 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:24.488170 27308 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:24.488758 27308 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:24.491961 27353 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:24.492292 27308 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:24.492492 27353 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:24.493633 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:24.493295 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:24.499593 26324 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:24.504379 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:24.505831 27355 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:24.506215 27355 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:24.506596 27355 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.507356 27129 tablet_bootstrap.cc:492] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:24.508445 27129 tablet_bootstrap.cc:492] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:24.509198 27129 ts_tablet_manager.cc:1397] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.044s	user 0.021s	sys 0.002s
I20260924 14:29:24.511945 27333 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:24.513005 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:24.515743 27333 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.517954 27333 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:24.513504 27353 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:24.514348 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:24.516801 27355 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:24.520229 26224 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.522040 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.522404 27355 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:24.522326 27129 raft_consensus.cc:348] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:24.523303 27129 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:24.524138 27129 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:24.526125 27129 ts_tablet_manager.cc:1428] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260924 14:29:24.526868 27129 tablet_bootstrap.cc:492] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:24.528965 27353 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.539731 27355 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:24.540365 27355 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:24.540748 26328 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:24.542452 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:24.543402 27353 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:24.543617 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:24.545001 26401 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:24.546458 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:24.547299 27354 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:24.547767 27354 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:24.548096 27354 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:24.551173 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:24.552100 27188 raft_consensus.cc:2379] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:24.544057 27355 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:24.544994 27353 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.554101 27353 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:24.556255 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:24.557324 27354 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:24.558769 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:24.559727 27188 raft_consensus.cc:2379] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:29:24.561057 26221 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:24.561781 26221 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:24.564064 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:24.565083 26401 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:24.574496 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.015s	sys 0.012s
W20260924 14:29:24.575173 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.017s	sys 0.013s
I20260924 14:29:24.577646 27348 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:24.580355 26221 leader_election.cc:341] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:24.580859 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.581884 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:24.583737 27355 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.591974 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.601924 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.609375 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.610177 26324 raft_consensus.cc:2379] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:24.610848 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:24.611905 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.612960 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.614001 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.615048 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.623591 26324 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:24.623517 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:24.624238 26295 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.003s	sys 0.021s
I20260924 14:29:24.624111 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:24.624781 27354 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:24.626621 26295 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.003s	sys 0.021s
I20260924 14:29:24.631296 26401 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:24.632932 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.633805 27333 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.634591 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:24.634696 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.636358 27189 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.636596 27188 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.637346 27333 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.638772 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.639969 27333 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:24.619334 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:24.640282 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:24.640875 26249 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:24.641235 27194 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:24.643167 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:24.643932 27355 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:24.644739 27355 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:24.643957 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:24.643225 26399 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:24.645749 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:24.634626 27069 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.004s
I20260924 14:29:24.647100 27263 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:24.647301 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:24.648077 26332 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:24.648553 26401 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:24.648965 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.650476 27294 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:24.651067 27294 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.651471 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.653225 27294 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:24.656873 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.658675 26325 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:24.659394 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.659916 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.660066 26326 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:24.663867 27263 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:24.665324 27346 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:24.665699 27263 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.665747 27346 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:24.668372 27346 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.669518 27346 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:24.682134 27333 consensus_queue.cc:1046] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:24.687219 26332 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:24.700532 27181 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:24.701484 26249 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.688184 27333 raft_consensus.cc:3032] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.688602 26326 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:24.689149 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.703369 27333 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:24.704144 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.704239 27333 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:24.705389 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:24.690416 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.708083 26258 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:24.708563 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.709692 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.709915 26251 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:24.710239 26328 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.689301 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:24.711755 26325 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:24.712574 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:24.714160 26325 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:24.693049 26077 catalog_manager.cc:5495] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:29:24.647126 27355 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:24.696883 26079 catalog_manager.cc:5495] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 2, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:24.698869 27295 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:24.700088 27182 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:24.702893 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.705493 27333 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.716980 26297 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:24.721239 26332 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:24.729161 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 77814.7us
I20260924 14:29:24.732088 26403 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:24.734107 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:24.698230 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.749514 26250 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:24.748883 26328 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:24.753182 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:24.754696 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.756016 27069 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:24.756541 27069 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.758430 26332 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:24.718632 27295 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:24.698242 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:24.659471 26323 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:24.718662 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.771377 27367 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:24.778087 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:24.776764 26325 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.778793 26327 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:24.743350 27294 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:24.749635 27337 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:24.743005 26330 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
W20260924 14:29:24.765522 27310 consensus_meta.cc:287] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.714s	user 0.014s	sys 0.000s
I20260924 14:29:24.790174 26325 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.791332 27365 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.792917 26329 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:24.804565 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.805640 27365 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.795356 27189 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:24.809895 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.812697 26295 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:24.780088 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:24.787281 27336 consensus_queue.cc:1046] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260924 14:29:24.788645 27294 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:24.811482 26326 raft_consensus.cc:2379] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:24.795786 27367 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:24.818106 27348 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260924 14:29:24.807284 27337 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:24.829813 27129 tablet_bootstrap.cc:492] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:24.832924 26327 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:24.832131 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:24.828685 27348 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.822098 27368 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.816764 27188 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:24.853951 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:24.854524 26404 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:24.855363 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:24.836951 27367 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:24.855916 26402 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.860716 26328 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.847577 26325 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.849670 27069 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:24.845425 26249 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:24.872376 27348 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:24.872435 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:24.843291 26255 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:24.875909 26403 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4. Ops: []
W20260924 14:29:24.877516 27363 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260924 14:29:24.877645 26397 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:24.874656 27348 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:24.872000 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:24.893015 27129 tablet_bootstrap.cc:492] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:24.895931 27129 ts_tablet_manager.cc:1397] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.368s	user 0.030s	sys 0.052s
I20260924 14:29:24.835634 27370 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260924 14:29:24.871562 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:24.850554 26331 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:24.849998 27362 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:24.863220 27366 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:24.808794 27295 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:24.872340 27330 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.831499 27331 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260924 14:29:24.926476 27370 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
W20260924 14:29:24.932463 27364 log.cc:924] Time spent T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260924 14:29:24.933037 27374 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:24.928329 27294 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:24.935003 27346 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.935377 27346 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:24.935956 27331 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:24.936364 27331 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:24.936177 27346 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:24.890797 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 68006.4us
I20260924 14:29:24.926962 27330 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:29:24.958217 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 169042us
I20260924 14:29:24.960253 27346 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:24.979671 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 82823.4us
I20260924 14:29:24.990849 26257 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:24.992733 27350 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:24.953027 27331 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:24.954696 27337 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:25.006857 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 241769us
W20260924 14:29:25.133965 26369 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.953s (SENT) [suppressed 344 similar messages]
W20260924 14:29:24.985059 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 158353us
I20260924 14:29:25.148910 27129 raft_consensus.cc:348] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:25.175428 27129 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
W20260924 14:29:25.150460 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 132997us
W20260924 14:29:25.185694 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 224714us
I20260924 14:29:25.187032 27129 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:25.205444 26252 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:25.209441 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:25.217713 26255 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:25.221184 27069 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.221902 27069 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:25.261570 27069 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:25.004434 27370 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.196s
I20260924 14:29:25.267617 27333 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:25.268657 27333 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.274250 27373 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:25.275054 27373 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.277956 27373 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:25.276248 27333 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:25.281416 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:25.260155 27331 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:25.315725 26353 debug-util.cc:398] Leaking SignalData structure 0x7b08002469e0 after lost signal to thread 23138
I20260924 14:29:25.152269 27372 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:25.000144 27330 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:25.281445 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:25.326022 27258 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:25.402253 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 268317us
I20260924 14:29:25.379201 27129 ts_tablet_manager.cc:1428] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.483s	user 0.011s	sys 0.003s
W20260924 14:29:25.424774 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 151038us
I20260924 14:29:25.433204 27392 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:25.464982 27392 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:25.433668 27353 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.302408 27123 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:25.464152 26325 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:25.337333 27372 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:25.337742 27378 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:25.371259 27374 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:25.436221 27129 ts_tablet_manager.cc:1460] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Tablet startup took 539ms
I20260924 14:29:25.477911 27129 tablet_bootstrap.cc:492] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:25.474222 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:25.479030 26331 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
W20260924 14:29:25.468084 26353 debug-util.cc:398] Leaking SignalData structure 0x7b080025b640 after lost signal to thread 26115
I20260924 14:29:25.472808 27391 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.465919 27392 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.365885 27381 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.157s
I20260924 14:29:25.505769 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:25.509197 27379 log.cc:924] Time spent T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260924 14:29:25.495160 27378 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:25.493495 27390 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:25.513664 27069 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:25.476197 27353 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.484159 27374 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.556912 27069 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:25.502588 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 303679us
I20260924 14:29:25.559423 27069 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:25.553790 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:25.570430 26405 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:25.506933 27391 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.572345 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:25.575905 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:25.554438 27392 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:25.519665 27333 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.579401 27069 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:25.559995 26353 debug-util.cc:398] Leaking SignalData structure 0x7b0800308220 after lost signal to thread 26282
I20260924 14:29:25.587005 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:25.512043 27354 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260924 14:29:25.609912 27069 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:25.611045 27069 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.498095 27372 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:25.611613 27069 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:25.612026 26295 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260924 14:29:25.620779 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 114118us
I20260924 14:29:25.586206 27258 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.627066 27069 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:25.633177 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:25.633991 26328 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:24.937186 27283 consensus_queue.cc:1159] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e09fbebe29a49d291d23f9d395fe335" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:29:25.636539 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:25.654785 26353 debug-util.cc:398] Leaking SignalData structure 0x7b08003124a0 after lost signal to thread 26356
I20260924 14:29:25.627972 27258 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:25.612612 27123 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:25.655884 27354 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.621132 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:25.371053 27350 consensus_queue.cc:1159] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c22052275034c369b7f7dca086fbd1e" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:29:25.656845 27391 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:25.657038 27374 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:25.632115 27403 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.658000 27404 consensus_queue.cc:1046] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260924 14:29:25.663920 26353 debug-util.cc:398] Leaking SignalData structure 0x7b080029b840 after lost signal to thread 26357
I20260924 14:29:25.608321 27333 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:25.672320 26328 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:25.679519 27294 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:25.677228 27385 raft_consensus.cc:2741] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260924 14:29:25.703922 27263 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:25.705853 27407 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:25.708408 27374 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:25.693697 27354 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:25.727792 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:25.728513 26402 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
W20260924 14:29:25.711180 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 102316us
W20260924 14:29:25.730636 27395 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260924 14:29:25.699887 27403 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:25.732168 27258 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:25.732791 27405 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.733494 27403 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:25.733299 27405 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:25.734000 27403 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.734201 27417 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:25.734479 27403 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.734669 27417 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:25.718145 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 116820us
I20260924 14:29:25.716394 27381 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:25.723419 27388 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260924 14:29:25.731618 27391 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:25.716039 27409 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:25.727114 26353 debug-util.cc:398] Leaking SignalData structure 0x7b0800130b20 after lost signal to thread 26430
I20260924 14:29:25.736581 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:25.737589 27333 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:25.737888 27381 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.760140 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:25.755707 27354 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:25.762065 27381 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:25.736904 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:25.735436 27405 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:25.754294 27295 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:25.755460 27422 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.755929 27411 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.757576 27409 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.663704 27378 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:25.754184 27410 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.759649 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:25.763605 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 103904us
I20260924 14:29:25.759119 27330 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:25.736663 27417 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:25.775553 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:25.783596 27354 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:25.785135 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 200009us
I20260924 14:29:25.785905 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:25.786331 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:25.786604 26328 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:25.786988 27185 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:25.787818 27415 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:25.788957 27405 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:25.788895 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:25.789435 27405 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:25.790232 27405 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:25.795475 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:25.796288 27405 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:25.797113 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:25.797672 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:25.796468 27354 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:25.798336 27354 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:25.798691 27354 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:25.800134 27415 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:25.806161 26405 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:25.807907 27423 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:25.808442 27423 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:25.808773 27423 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:25.809758 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:25.810070 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:25.810691 26397 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:25.810480 27354 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:25.813938 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:25.814774 26331 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 3
W20260924 14:29:25.764779 26295 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260924 14:29:25.779624 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:25.780264 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:25.818243 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 53754.2us
I20260924 14:29:25.780751 26255 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:25.819298 27418 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:25.750137 27353 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:25.813314 27354 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:25.765553 27374 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:25.821043 27374 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:25.821568 27374 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:25.821717 27418 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:25.822968 27418 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:25.728724 27407 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:25.774102 27411 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:25.832628 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:25.832984 27407 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:25.833330 27411 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:25.833436 26251 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:25.835191 27409 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:25.836409 26328 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:25.836845 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:25.837046 26402 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:25.839859 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:25.840472 26328 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:25.839051 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:25.837690 26327 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:25.843580 26325 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:25.762866 27391 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:25.778517 27390 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:25.835526 27410 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.845515 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:25.846226 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:25.851776 26330 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:25.853906 26331 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:25.778450 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:25.861192 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:25.839116 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 102293us
I20260924 14:29:25.862771 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:25.863577 27324 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:25.864197 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:25.869480 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:25.834638 27407 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.007s
W20260924 14:29:25.857875 26353 debug-util.cc:398] Leaking SignalData structure 0x7b080008c600 after lost signal to thread 27222
I20260924 14:29:25.879225 26250 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:25.880131 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:25.880484 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:25.912376 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:25.913514 27194 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:25.915688 27189 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260924 14:29:25.916263 27189 pending_rounds.cc:76] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 2
I20260924 14:29:25.916558 27189 pending_rounds.cc:97] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260924 14:29:25.916913 27189 raft_consensus.cc:2866] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:25.880744 27391 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:25.860152 27390 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:25.771939 27422 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.889256 27410 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:25.890172 26405 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:25.891496 27403 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:25.901733 26401 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:25.866401 27409 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260924 14:29:25.866566 27417 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260924 14:29:25.871541 26326 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:25.921350 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:25.923257 27390 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:25.922633 27401 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:25.948633 27401 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:25.940619 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:29:25.950174 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 74650.6us
I20260924 14:29:25.950724 27069 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:25.951323 27069 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:25.951367 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:25.951879 27069 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:25.936954 27384 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260924 14:29:25.953632 26258 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:25.929289 27418 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:25.926715 27370 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:25.929275 27355 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:25.965638 26332 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:25.968612 26080 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 136) took 1092 ms. Trace:
I20260924 14:29:25.955191 27263 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:25.969027 26080 rpcz_store.cc:276] 0924 14:29:24.853976 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:24.855614 (+  1638us) service_pool.cc:227] Handling call
0924 14:29:25.945869 (+1090255us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260924 14:29:25.921626 27374 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.970362 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:25.969278 27418 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:25.971632 27370 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:29:25.937579 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 57173.1us
I20260924 14:29:25.956852 27409 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:25.979336 27333 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:25.982017 27411 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260924 14:29:25.981508 27418 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:25.981678 27403 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:25.977994 27188 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:25.973634 27423 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:25.983368 27192 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 3: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:25.993204 27069 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:25.967864 27417 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260924 14:29:25.992352 26295 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260924 14:29:25.974094 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:26.000567 27422 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:25.991803 27409 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:26.004447 27417 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:26.002028 27403 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:26.008726 27069 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:26.019330 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 322265us
I20260924 14:29:25.972023 27355 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:26.020784 27417 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:25.974808 27263 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:26.029758 27403 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:26.031661 27374 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:26.032157 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:26.033052 26256 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.035933 27403 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:26.038686 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:26.039541 26399 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:26.041993 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:26.042659 26399 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.050221 26256 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:26.050436 26399 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:26.090811 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:26.091287 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:26.091769 27185 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:25.992776 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:26.025336 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:26.093197 26326 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.093235 26397 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:26.094173 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:26.060581 27354 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260924 14:29:26.096433 27354 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:26.096872 27354 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:26.097556 27129 tablet_bootstrap.cc:492] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:26.098528 27129 tablet_bootstrap.cc:492] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:26.079342 27423 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:26.100246 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:26.101040 27194 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:26.102120 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:26.102789 26326 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:26.107987 27129 ts_tablet_manager.cc:1397] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.630s	user 0.029s	sys 0.031s
I20260924 14:29:26.021797 27409 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.055648 27069 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:26.023080 27252 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:26.056479 26257 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:26.057216 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:26.056957 27390 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:26.058190 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:26.139788 27421 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:26.058833 27370 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:26.140985 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 63802.7us
I20260924 14:29:26.141367 27069 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:26.141782 27069 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:26.142158 27069 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.063292 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:26.005918 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:26.019254 27410 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260924 14:29:26.075254 27263 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:26.147959 27185 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:26.075650 27373 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:26.077668 27401 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:26.079205 27336 raft_consensus.cc:3032] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:26.085659 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:26.065062 26402 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:26.142510 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:26.143203 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:26.143219 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50367.2us
W20260924 14:29:26.137775 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 154392us
I20260924 14:29:26.140338 27421 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:26.135776 27252 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:26.146394 27407 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:26.146493 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:26.147488 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 562) took 1523 ms. Trace:
I20260924 14:29:26.204293 27069 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:26.165596 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 44251.9us
I20260924 14:29:26.167909 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:26.143597 27129 raft_consensus.cc:348] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:26.205395 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 58987.5us
I20260924 14:29:26.206090 27129 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
W20260924 14:29:26.203363 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 117824us
I20260924 14:29:26.189812 27263 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:26.214159 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:26.205250 26332 rpcz_store.cc:276] 0924 14:29:24.613552 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:24.769006 (+155454us) service_pool.cc:227] Handling call
0924 14:29:24.769236 (+   230us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:25.970108 (+1200872us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:26.136205 (+166097us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400557dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:26.136480 (+   275us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2750964992}
I20260924 14:29:26.098937 27354 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:26.208520 26258 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:26.168594 27392 raft_consensus.cc:2741] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260924 14:29:26.222728 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:26.169816 27373 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:26.149389 27188 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:26.210266 27129 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:26.141881 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47819.2us
I20260924 14:29:26.147332 26401 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:26.205094 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:26.212791 27069 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:26.225646 27419 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260924 14:29:26.205880 27401 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:26.243086 27129 ts_tablet_manager.cc:1428] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.135s	user 0.006s	sys 0.001s
I20260924 14:29:26.221829 27407 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:26.168358 27189 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.190516 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:26.229465 26256 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:26.203859 27336 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:26.231586 27252 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:26.233934 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:26.228207 27354 raft_consensus.cc:2741] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260924 14:29:26.247262 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:26.227607 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 225324us
I20260924 14:29:26.248299 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:26.249498 27372 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:26.233287 27423 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:26.251084 27129 tablet_bootstrap.cc:492] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:26.254983 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:26.236393 27333 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:26.260799 26400 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:26.253293 27355 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:26.261255 26397 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:26.236269 27283 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260924 14:29:26.261858 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:26.262024 27373 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:26.243824 26251 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:26.277400 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:26.273834 26402 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.259820 26399 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.256424 27331 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:26.305740 26256 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:26.310829 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:26.266254 27365 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:26.264533 27390 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:26.250548 27401 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:26.312966 27384 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:26.346544 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:26.310105 27423 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:26.369560 27423 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:26.370335 27401 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:26.365792 27350 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260924 14:29:26.372289 27194 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:26.347605 27421 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:26.379913 26402 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:26.352867 27336 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:26.317335 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:26.343490 26406 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
W20260924 14:29:26.358961 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 81681.9us
I20260924 14:29:26.359676 27354 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:26.362437 27373 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.312045 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:26.360158 27365 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:26.398854 27405 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:26.399187 27354 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:26.403434 27374 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:26.433984 27365 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:26.374457 27384 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:26.387009 26398 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:26.338989 27407 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:26.418593 27390 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:26.367681 27417 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:26.442752 27354 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:26.412204 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 137083us
I20260924 14:29:26.454921 27336 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.457762 27390 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:29:26.437661 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 137692us
I20260924 14:29:26.461840 27365 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:26.442067 27405 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:26.494434 27393 thread.cc:636] apply [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.005s	sys 0.021s
W20260924 14:29:26.504176 27418 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.011s	sys 0.065s
I20260924 14:29:26.504208 27390 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:26.504222 27389 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:26.509541 27393 thread.cc:602] apply [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.006s	sys 0.021s
W20260924 14:29:26.530211 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 232428us
I20260924 14:29:26.512791 27411 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:26.516680 27418 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.011s	sys 0.066s
I20260924 14:29:26.610666 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:26.596796 27385 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:26.624186 26324 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:26.644587 27385 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:26.614197 27353 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:26.597203 27337 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:26.650050 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:26.661008 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:26.661674 26332 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:26.597579 27411 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:26.622424 27378 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:26.524291 27409 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:26.596982 27427 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:26.651054 27422 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:26.652096 27405 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:26.530294 27389 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:26.532434 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:26.655879 27337 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:26.708817 26332 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
W20260924 14:29:26.713424 27439 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260924 14:29:26.714932 27409 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:26.609571 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:26.721604 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:26.722512 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.021s	sys 0.043s
W20260924 14:29:26.722870 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.021s	sys 0.043s
I20260924 14:29:26.706252 27422 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:26.658531 27353 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:26.737494 26332 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:26.720319 27192 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.762517 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:26.767052 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:26.705916 27384 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:26.773403 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:26.724552 27417 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:26.702307 27378 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:26.706143 27427 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:26.784302 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:26.747752 27263 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:26.723130 27385 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:26.749573 27422 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:26.787030 27398 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:26.787549 27438 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:26.774179 26324 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
W20260924 14:29:26.775043 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 235055us
I20260924 14:29:26.865162 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:26.801890 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:26.788419 27252 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:26.873967 26258 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
W20260924 14:29:26.874464 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 141134us
I20260924 14:29:26.879757 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:26.793309 27263 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:26.788226 26399 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:26.792552 27355 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:26.863458 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:26.766258 27421 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260924 14:29:26.805254 27398 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:29:26.884403 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 83867.7us
I20260924 14:29:26.883903 27252 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:26.888815 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:26.893893 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:26.896618 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:26.897979 27398 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:26.901970 26331 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:26.871860 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 69235.3us
I20260924 14:29:26.872417 27353 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:26.887018 27263 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:26.884126 27410 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:26.902384 26258 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:29:26.910924 27189 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.654s	user 0.013s	sys 0.000s
I20260924 14:29:26.900938 27188 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
W20260924 14:29:26.998198 27443 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
W20260924 14:29:26.999127 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 65175.4us
I20260924 14:29:26.920855 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:26.921491 26323 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:26.923134 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:27.051605 26331 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:26.998826 27353 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
W20260924 14:29:27.046489 27354 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.004s
I20260924 14:29:27.037369 27331 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:26.865671 27438 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:26.996627 27189 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:27.055666 27337 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:27.054853 27410 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:29:27.073915 27444 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260924 14:29:27.074520 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 72) took 1074 ms. Trace:
I20260924 14:29:27.062363 27355 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:27.075016 27452 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
W20260924 14:29:27.082536 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 764810us
I20260924 14:29:27.084396 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:27.085371 27391 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:27.085780 27391 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:27.086148 27391 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:27.086495 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:27.088156 27456 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:29:27.088766 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:27.089461 27188 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:27.065330 27353 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:27.065821 27354 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:27.093427 27354 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.094063 26253 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:27.077200 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 74) took 1007 ms. Trace:
W20260924 14:29:27.096544 27449 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.375s	user 0.000s	sys 0.001s
I20260924 14:29:27.096686 27410 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.097803 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 593) took 1070 ms. Trace:
I20260924 14:29:27.067727 27389 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:27.075229 27448 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260924 14:29:27.074038 27194 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:29:27.105384 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 303637us
I20260924 14:29:27.046053 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:29:27.073931 27451 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260924 14:29:27.068373 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:27.094106 27336 consensus_meta.cc:287] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.000s
I20260924 14:29:27.126211 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:27.066821 27331 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:27.096356 27189 rpcz_store.cc:276] 0924 14:29:26.062819 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:26.062981 (+   162us) service_pool.cc:227] Handling call
0924 14:29:27.069910 (+1006929us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:27.105679 27378 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.144822 27129 tablet_bootstrap.cc:492] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:27.100286 27411 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:27.139808 27192 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:27.107303 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:27.099339 27355 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:27.159802 27464 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.133204 27427 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.135673 27407 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:27.132762 27354 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:27.174017 27129 tablet_bootstrap.cc:492] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:27.098183 27353 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:27.179386 27192 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:27.179924 26403 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:27.181074 27438 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:27.164060 27464 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:27.098174 26254 rpcz_store.cc:276] 0924 14:29:26.027466 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:26.028524 (+  1058us) service_pool.cc:227] Handling call
0924 14:29:26.058046 (+ 29522us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:26.058413 (+   367us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:26.058446 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:26.058895 (+   449us) log.cc:842] Serialized 26 byte log entry
0924 14:29:26.505243 (+446348us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:26.507955 (+  2712us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:26.513766 (+  5811us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:27.097574 (+583808us) raft_consensus.cc:1671] finished
0924 14:29:27.097597 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:27.097726 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":37819,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":443678,"threads_started":1,"wal-append.queue_time_us":656632}
I20260924 14:29:27.183209 27464 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:27.183670 27331 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:27.164484 27185 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:27.175977 27129 ts_tablet_manager.cc:1397] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.925s	user 0.024s	sys 0.003s
W20260924 14:29:27.147030 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 149432us
W20260924 14:29:27.175788 27373 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.774s	user 0.011s	sys 0.001s
I20260924 14:29:27.174289 27354 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:27.165262 27252 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:27.211140 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:27.189580 27407 raft_consensus.cc:686] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:29:27.228078 27462 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260924 14:29:27.233129 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 192457us
I20260924 14:29:27.074972 27181 rpcz_store.cc:276] 0924 14:29:26.000394 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:26.000566 (+   172us) service_pool.cc:227] Handling call
0924 14:29:26.001032 (+   466us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:26.001438 (+   406us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:26.001469 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:26.001908 (+   439us) log.cc:842] Serialized 26 byte log entry
0924 14:29:26.412872 (+410964us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:26.413925 (+  1053us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:26.437155 (+ 23230us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:27.072274 (+635119us) raft_consensus.cc:1671] finished
0924 14:29:27.074217 (+  1943us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:27.074436 (+   219us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":121,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":399097,"threads_started":1,"wal-append.queue_time_us":514322}
I20260924 14:29:27.236341 27463 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.223122 27354 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:27.236963 26406 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:27.199002 27411 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:27.169729 27425 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.065s
W20260924 14:29:27.235494 27372 consensus_meta.cc:287] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.963s	user 0.011s	sys 0.002s
I20260924 14:29:27.231930 27337 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.251694 27337 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.253749 27337 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.232595 27123 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:27.231877 27252 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:27.254791 27252 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:29:27.249161 27442 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.474s	user 0.003s	sys 0.000s
I20260924 14:29:27.234053 27330 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:27.256398 27411 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:27.257319 27330 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:27.257817 27391 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.237037 27389 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.258713 27411 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:27.259059 27389 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.250696 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:27.260164 26399 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:27.261139 27389 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:27.264396 27390 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.736s	user 0.005s	sys 0.005s
I20260924 14:29:27.263929 27252 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:27.264848 27390 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:27.267107 27390 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:27.259617 27355 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.268271 27391 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:27.268918 27407 consensus_queue.cc:237] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.271857 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:27.275969 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:27.231573 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:27.279570 27365 consensus_meta.cc:287] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.001s
I20260924 14:29:27.279358 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:27.296442 27463 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.296878 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:27.298497 26250 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:27.277051 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:27.298381 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 75502.2us
I20260924 14:29:27.321816 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:27.327973 27129 raft_consensus.cc:348] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.328895 27129 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:27.329484 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:27.336526 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:27.333935 27129 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.345964 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:27.337572 26332 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:27.322427 27392 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
W20260924 14:29:27.354180 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 98279.2us
I20260924 14:29:27.354909 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:27.358682 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:27.359521 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:27.360585 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:27.344978 27185 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:27.319726 26254 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 1.
I20260924 14:29:27.331156 27445 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:27.372509 27129 ts_tablet_manager.cc:1428] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.184s	user 0.006s	sys 0.001s
I20260924 14:29:27.385701 26402 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:27.330488 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:27.386962 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:27.387830 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:27.343109 27333 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.389824 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:27.321532 27182 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:27.383996 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:27.397981 26332 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:27.351217 27404 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.341830 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:27.402355 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:27.356170 27348 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.403801 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:27.406750 26404 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:27.413802 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:27.415150 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:27.355211 27387 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.386961 27392 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:27.416211 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:27.387125 27464 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:27.418864 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:27.386722 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:27.388568 27252 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:27.388023 27445 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:27.402325 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:27.411643 26323 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:27.431679 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:27.405074 27355 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260924 14:29:27.413147 27422 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.000s
I20260924 14:29:27.358274 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:27.333010 27398 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:27.408404 27330 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:27.443238 26404 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:27.452280 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:27.439534 27421 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:27.438555 27425 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:27.422341 27417 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:27.396682 27333 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.411845 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:27.447882 27473 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260924 14:29:27.423231 27475 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:27.421587 26399 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:27.440352 27445 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:27.464058 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61385.1us
I20260924 14:29:27.446434 27385 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:27.415022 27348 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.447703 27355 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:27.410638 27404 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:27.458631 27422 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:27.494527 27333 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.495043 26327 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:27.494589 27355 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:27.496374 27422 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:27.480537 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 145835us
I20260924 14:29:27.494529 27348 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.500128 27123 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:27.426314 27387 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.470068 27336 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:27.444088 27252 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:27.443200 27464 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.470366 27417 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:27.472692 27373 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:27.493404 26400 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:27.488124 27385 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:27.480495 26402 raft_consensus.cc:2354] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3.
I20260924 14:29:27.482152 27365 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.508693 26077 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 135) took 3094 ms. Trace:
I20260924 14:29:27.508827 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:27.510936 27390 raft_consensus.cc:2741] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260924 14:29:27.524919 27463 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.503825 27336 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:27.484990 27475 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:27.531111 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:27.531567 27475 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:27.527397 26326 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:27.481208 27421 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:27.519692 27373 raft_consensus.cc:686] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:27.517495 27417 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:27.520490 27385 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:27.503115 27263 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.542022 27354 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:27.528719 27365 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.545786 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:27.548867 26404 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:27.522266 27252 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:27.503345 27425 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:27.547981 27365 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.550575 27425 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:27.529531 27398 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:27.529889 27387 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.524587 26077 rpcz_store.cc:276] 0924 14:29:24.414466 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:24.419574 (+  5108us) service_pool.cc:227] Handling call
0924 14:29:24.489043 (+ 69469us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b540015fed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3af0 00007f09b9356f2a 00007f09bf2a8d6c 00007f09bf3a2c3e 00007f09bd6b4fb4 00007f09bd6b4f42 00007f09bd6b4e71 00007f09bd6b4dec 00007f09bd6b40a2 00007f09b777894c 00007f09b7777d86
0924 14:29:27.508631 (+3019588us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:29:25.005642 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:29:25.211860 (+206218us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:29:25.211895 (+    35us) write_op.cc:435] Acquired schema lock
0924 14:29:25.211907 (+    12us) tablet.cc:660] Decoding operations
0924 14:29:25.212249 (+   342us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:29:25.212310 (+    61us) write_op.cc:641] Partition lock acquired for write op
0924 14:29:25.212334 (+    24us) tablet.cc:683] Acquiring locks for 1 operations
0924 14:29:25.212517 (+   183us) tablet.cc:699] Row locks acquired
0924 14:29:25.212531 (+    14us) write_op.cc:260] PREPARE: finished
0924 14:29:25.212668 (+   137us) write_op.cc:270] Start()
0924 14:29:25.212778 (+   110us) write_op.cc:276] Timestamp: P: 1790260165212623 usec, L: 0
0924 14:29:25.212793 (+    15us) op_driver.cc:331] REPLICATION: starting
0924 14:29:25.213388 (+   595us) log.cc:842] Serialized 543 byte log entry
0924 14:29:26.617161 (+1403773us) op_driver.cc:444] REPLICATION: finished
0924 14:29:26.994800 (+377639us) write_op.cc:301] APPLY: starting
0924 14:29:26.999381 (+  4581us) tablet.cc:1365] starting BulkCheckPresence
0924 14:29:27.047768 (+ 48387us) tablet.cc:1368] finished BulkCheckPresence
0924 14:29:27.068745 (+ 20977us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:29:27.106600 (+ 37855us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:29:27.106751 (+   151us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 14:29:27.106796 (+    45us) write_op.cc:312] APPLY: finished
0924 14:29:27.107010 (+   214us) log.cc:842] Serialized 22 byte log entry
0924 14:29:27.107174 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:27.107252 (+    78us) write_op.cc:454] Released schema lock
0924 14:29:27.107325 (+    73us) write_op.cc:341] FINISH: Updating metrics
0924 14:29:27.135938 (+ 28613us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:27.140630 (+  4692us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":9763072,"child_traces":[["op",{"apply.queue_time_us":272382,"apply.run_cpu_time_us":2800,"apply.run_wall_time_us":235237,"mutex_wait_us":8691,"num_ops":1,"prepare.queue_time_us":205707,"prepare.run_cpu_time_us":2619,"prepare.run_wall_time_us":212620,"replication_time_us":1399045,"thread_start_us":111703,"threads_started":1}]]}
I20260924 14:29:27.531030 27336 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:27.489049 27392 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.528564 26079 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55200 (request call id 124) took 2898 ms. Trace:
W20260924 14:29:27.537986 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 110946us
I20260924 14:29:27.554402 27464 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.515981 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:27.626471 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 110600us
I20260924 14:29:27.535351 27391 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:27.629817 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:27.551782 27483 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:27.640748 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:27.642968 27404 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:27.643038 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:27.643764 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:27.646123 27181 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:27.648861 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:27.659791 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:27.655021 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:27.660715 26327 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 3
I20260924 14:29:27.655838 27129 tablet_bootstrap.cc:492] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:27.661613 26404 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
W20260924 14:29:27.654505 27453 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.764s	user 0.008s	sys 0.001s
I20260924 14:29:27.621116 27398 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.551615 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:27.628324 27283 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:27.631282 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:27.687431 27468 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:27.687880 27182 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:27.688138 27468 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:27.632659 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 385) took 1251 ms. Trace:
I20260924 14:29:27.666373 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:27.667874 27438 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:27.687457 27425 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:27.687873 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:27.672055 27404 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.620220 26079 rpcz_store.cc:276] 0924 14:29:24.630043 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:24.630201 (+   158us) service_pool.cc:227] Handling call
0924 14:29:27.528511 (+2898310us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:29:24.924598 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:29:24.925134 (+   536us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:29:24.925185 (+    51us) write_op.cc:435] Acquired schema lock
0924 14:29:24.925230 (+    45us) tablet.cc:660] Decoding operations
0924 14:29:24.925752 (+   522us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:29:24.925881 (+   129us) write_op.cc:641] Partition lock acquired for write op
0924 14:29:24.925926 (+    45us) tablet.cc:683] Acquiring locks for 1 operations
0924 14:29:24.928915 (+  2989us) tablet.cc:699] Row locks acquired
0924 14:29:24.928967 (+    52us) write_op.cc:260] PREPARE: finished
0924 14:29:24.929155 (+   188us) write_op.cc:270] Start()
0924 14:29:24.929432 (+   277us) write_op.cc:276] Timestamp: P: 1790260164929088 usec, L: 0
0924 14:29:24.929513 (+    81us) op_driver.cc:331] REPLICATION: starting
0924 14:29:24.930543 (+  1030us) log.cc:842] Serialized 543 byte log entry
0924 14:29:25.520800 (+590257us) op_driver.cc:444] REPLICATION: finished
0924 14:29:26.110120 (+589320us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b64085fe 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09bcd5e410
0924 14:29:26.727629 (+617509us) write_op.cc:301] APPLY: starting
0924 14:29:26.727729 (+   100us) tablet.cc:1365] starting BulkCheckPresence
0924 14:29:26.774529 (+ 46800us) tablet.cc:1368] finished BulkCheckPresence
0924 14:29:26.791076 (+ 16547us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:29:26.991108 (+200032us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:29:27.069497 (+ 78389us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0924 14:29:27.079006 (+  9509us) write_op.cc:312] APPLY: finished
0924 14:29:27.095971 (+ 16965us) log.cc:842] Serialized 22 byte log entry
0924 14:29:27.208819 (+112848us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:27.221929 (+ 13110us) write_op.cc:454] Released schema lock
0924 14:29:27.225331 (+  3402us) write_op.cc:341] FINISH: Updating metrics
0924 14:29:27.241414 (+ 16083us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:27.243098 (+  1684us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":272266,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":1141215,"apply.run_cpu_time_us":44681,"apply.run_wall_time_us":524852,"num_ops":1,"prepare.queue_time_us":141117,"prepare.run_cpu_time_us":5624,"prepare.run_wall_time_us":52444,"raft.queue_time_us":1203017,"raft.run_cpu_time_us":59042,"raft.run_wall_time_us":1211128,"replication_time_us":585832,"spinlock_wait_cycles":27171200,"thread_start_us":1290150,"threads_started":5,"wal-append.queue_time_us":139482,"wal-append.run_cpu_time_us":14957,"wal-append.run_wall_time_us":1610055}]]}
I20260924 14:29:27.689499 27263 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:27.715600 27374 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:27.716316 27374 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:27.717336 27374 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.664232 27483 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:27.704635 26324 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:27.669399 26326 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:27.631822 27421 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.689196 27468 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:27.673966 27398 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.692463 27126 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:27.710813 27188 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:27.738752 26327 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
W20260924 14:29:27.746238 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 71493.5us
I20260924 14:29:27.752341 27263 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:27.677034 27373 consensus_queue.cc:237] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.735752 27354 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:27.769101 27483 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:27.766391 27263 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260924 14:29:27.711236 27404 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:27.777695 27263 consensus_queue.cc:1225] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260924 14:29:27.826735 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:27.826484 26078 catalog_manager.cc:5495] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 1 to 2, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:27.693441 26401 rpcz_store.cc:276] 0924 14:29:26.373505 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:26.380368 (+  6863us) service_pool.cc:227] Handling call
0924 14:29:26.415496 (+ 35128us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:26.610114 (+194618us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:26.612358 (+  2244us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:26.696079 (+ 83721us) log.cc:842] Serialized 26 byte log entry
0924 14:29:26.769532 (+ 73453us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:26.783370 (+ 13838us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:27.542163 (+758793us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54006e69d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:27.543218 (+  1055us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:27.546613 (+  3395us) raft_consensus.cc:1671] finished
0924 14:29:27.548239 (+  1626us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:27.624715 (+ 76476us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2617,"raft.queue_time_us":2663,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":18661,"spinlock_wait_cycles":247548288,"thread_start_us":49209,"threads_started":1,"wal-append.queue_time_us":64608,"wal-append.run_cpu_time_us":1514,"wal-append.run_wall_time_us":1002096}
I20260924 14:29:27.829082 26078 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 3, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:27.771904 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:27.833374 26078 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 3, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:27.836982 27434 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:27.839082 26078 catalog_manager.cc:5495] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 1 to 2, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:27.839551 27434 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:27.845798 27483 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:27.848754 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 597) took 1056 ms. Trace:
I20260924 14:29:27.834506 27350 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:27.846382 27483 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:27.882104 27350 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:27.911144 27483 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.911338 27445 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:27.912990 26249 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:27.915570 27445 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:27.838922 27355 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:27.851692 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 87560.3us
I20260924 14:29:27.927778 27427 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:27.880017 27482 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:27.948182 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:27.952049 27355 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:27.952692 27482 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:27.932471 27350 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:27.954022 27482 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.959647 27483 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:27.963354 27417 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:27.849413 26255 rpcz_store.cc:276] 0924 14:29:26.792355 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:26.796674 (+  4319us) service_pool.cc:227] Handling call
0924 14:29:27.512031 (+715357us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:27.512334 (+   303us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:27.522851 (+ 10517us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:27.522885 (+    34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:27.522930 (+    45us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:27.522965 (+    35us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:27.848444 (+325479us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54002882d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:27.848509 (+    65us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:27.848675 (+   166us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":37957888}
I20260924 14:29:27.963047 27398 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.955693 27427 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:27.973896 26253 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.004s	sys 0.044s
W20260924 14:29:27.977398 26253 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.005s	sys 0.044s
I20260924 14:29:27.957386 27434 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:27.970753 27404 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:27.976905 27336 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:27.985236 27336 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:27.987031 27385 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:28.024199 26371 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.008s	sys 0.037s
I20260924 14:29:27.978118 27475 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:28.027484 26371 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.008s	sys 0.037s
I20260924 14:29:28.028475 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:28.033332 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 404451us
W20260924 14:29:28.016263 26397 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.000s	sys 0.034s
W20260924 14:29:28.022905 27192 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.003s	sys 0.030s
I20260924 14:29:28.054647 26298 leader_election.cc:398] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3.
I20260924 14:29:28.056358 26298 leader_election.cc:401] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260924 14:29:28.036940 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.017s	sys 0.013s
I20260924 14:29:28.026602 27370 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:28.040030 26397 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.000s	sys 0.035s
I20260924 14:29:28.034533 27346 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:28.059502 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:28.057487 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.018s	sys 0.014s
W20260924 14:29:28.054642 27192 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.004s	sys 0.030s
I20260924 14:29:28.062826 27372 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:28.062788 27488 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3.
I20260924 14:29:28.064064 27372 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:28.064460 27372 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:28.062449 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 604032us
I20260924 14:29:28.069375 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.070103 27188 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:28.061868 27346 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:28.061331 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:28.072777 27470 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.005s	sys 0.025s
I20260924 14:29:28.079906 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:28.081663 27182 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:28.062129 27370 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:28.074352 27346 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:28.083957 27470 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.005s	sys 0.026s
I20260924 14:29:28.075277 27417 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:28.077559 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:28.093801 26295 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.009s	sys 0.024s
W20260924 14:29:28.094192 26295 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.009s	sys 0.024s
I20260924 14:29:28.082242 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:28.082099 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:28.089689 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 497433us
I20260924 14:29:28.100402 27189 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:28.122741 27181 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:28.104269 27475 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:28.123242 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:28.124047 27417 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:28.124050 26327 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:28.124688 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:28.125686 26329 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:28.104926 27372 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:28.107923 27188 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:28.110919 26401 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:28.127918 27372 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:28.129176 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:28.130060 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:28.130697 26406 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:28.131167 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:28.131932 26399 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:28.132973 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:28.134608 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:28.093194 27427 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:28.151544 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:28.157464 26295 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:28.158586 26402 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:28.093524 27391 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:28.160508 26406 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:28.160322 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:28.137583 27385 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:28.167399 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:28.169298 26224 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:28.124686 27417 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:28.173058 27353 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:28.175547 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.162698 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:28.182427 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:28.187297 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:28.187981 26402 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:28.193719 26404 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:28.190299 26398 raft_consensus.cc:2354] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3.
I20260924 14:29:28.189822 27370 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:28.178653 26327 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:28.199226 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:28.241355 27411 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.020s	sys 0.028s
I20260924 14:29:28.189657 27390 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:28.326553 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:28.283988 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:28.292598 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:28.292941 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:28.269325 27407 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:28.293530 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:28.294135 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.271566 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:28.298710 26329 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:28.299389 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:28.300386 26402 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:28.301477 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:28.183090 27391 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:28.269382 27350 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260924 14:29:28.305083 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:28.305392 26406 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:28.308637 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:28.317111 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:28.311002 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:28.339073 26324 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:28.371356 26253 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
W20260924 14:29:28.331902 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 48017.3us
W20260924 14:29:28.324041 27411 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.020s	sys 0.029s
I20260924 14:29:28.337071 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:28.371183 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 66187.3us
W20260924 14:29:28.381815 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44818.6us
I20260924 14:29:28.307029 27346 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:28.322296 27483 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:28.343212 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:28.384310 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 67305.6us
W20260924 14:29:28.351331 26252 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.013s	sys 0.009s
W20260924 14:29:28.373598 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65049.1us
I20260924 14:29:28.341500 27189 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:28.339646 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:28.324628 27390 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:28.342700 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:28.348337 27398 raft_consensus.cc:2741] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260924 14:29:28.370687 27355 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:28.385788 27468 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:28.428119 27468 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:28.428043 27355 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:28.428670 27468 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:28.430274 27355 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:28.387012 26252 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.013s	sys 0.009s
I20260924 14:29:28.348078 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:28.431586 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 608) took 1082 ms. Trace:
I20260924 14:29:28.431903 26250 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:29:28.433133 27493 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260924 14:29:28.434496 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 603) took 1042 ms. Trace:
I20260924 14:29:28.434860 26397 rpcz_store.cc:276] 0924 14:29:27.392669 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:27.430941 (+ 38272us) service_pool.cc:227] Handling call
0924 14:29:27.431348 (+   407us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:27.451308 (+ 19960us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:27.455178 (+  3870us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:27.501610 (+ 46432us) log.cc:842] Serialized 26 byte log entry
0924 14:29:28.067855 (+566245us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:28.068583 (+   728us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:28.071082 (+  2499us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:28.434288 (+363206us) raft_consensus.cc:1671] finished
0924 14:29:28.434307 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:28.434435 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":560583,"threads_started":1,"wal-append.queue_time_us":687840}
I20260924 14:29:28.435930 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:28.436863 26254 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:28.437645 27390 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:28.328806 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:28.339480 26257 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:28.350602 27458 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:28.335227 27467 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260924 14:29:28.388422 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:28.388422 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:28.479720 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 91376.3us
W20260924 14:29:28.479918 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 91569.8us
I20260924 14:29:28.480846 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:28.344389 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.020s	sys 0.031s
I20260924 14:29:28.336606 27434 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260924 14:29:28.402017 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:28.353255 26398 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 4
I20260924 14:29:28.399096 27191 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:28.350687 26330 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:28.384294 27418 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:28.384632 26404 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:28.338830 27407 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:28.501255 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:28.515857 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:28.437704 26251 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:28.348755 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:28.533006 27407 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:28.492604 27355 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:28.508360 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106419us
I20260924 14:29:28.504925 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:28.558521 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:28.494382 27438 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:28.523902 27404 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:28.494228 27372 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:28.510422 27488 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:28.552742 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 86) took 1420 ms. Trace:
I20260924 14:29:28.553588 27482 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:28.519372 27385 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260924 14:29:28.558636 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.020s	sys 0.031s
I20260924 14:29:28.518626 27411 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:28.387552 27483 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:28.609163 27483 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:28.610163 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 1.06741e+06us
I20260924 14:29:28.611001 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:28.480299 27417 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:28.612624 27434 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:28.612979 27482 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:28.613848 27482 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:28.614244 27191 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:28.596594 27192 rpcz_store.cc:276] 0924 14:29:27.156064 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:27.163166 (+  7102us) service_pool.cc:227] Handling call
0924 14:29:27.179210 (+ 16044us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:27.442585 (+263375us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:27.445819 (+  3234us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:27.485677 (+ 39858us) log.cc:842] Serialized 26 byte log entry
0924 14:29:28.070724 (+585047us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:28.552051 (+481327us) log.cc:842] Serialized 14 byte log entry
0924 14:29:28.552401 (+   350us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:28.552482 (+    81us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:28.552524 (+    42us) raft_consensus.cc:1671] finished
0924 14:29:28.552539 (+    15us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:28.552679 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25299,"raft.queue_time_us":40509,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":34496,"thread_start_us":1012280,"threads_started":2,"wal-append.queue_time_us":632988}
I20260924 14:29:28.616406 27192 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:28.617534 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.538056 27350 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:28.536620 26399 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260924 14:29:28.556283 27507 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260924 14:29:28.535368 27418 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:28.520709 27409 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:28.564301 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:28.671466 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 107256us
I20260924 14:29:28.569360 27181 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 3: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:28.630764 26404 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:28.568455 27194 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:28.578210 27404 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:28.595850 27372 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:29:28.585246 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 248048us
I20260924 14:29:28.569119 27355 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:28.673103 27409 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:28.615056 27482 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:28.668011 27350 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:28.678785 27404 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:28.674954 27417 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:28.676952 27421 raft_consensus.cc:2741] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260924 14:29:28.606735 27488 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:28.683425 27185 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.009s	sys 0.043s
I20260924 14:29:28.681730 27182 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:28.679145 27372 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:28.698055 26400 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.022s	sys 0.026s
I20260924 14:29:28.663246 27468 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:28.695063 26401 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.024s	sys 0.032s
I20260924 14:29:28.693544 27409 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:28.715425 27458 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:28.715582 27355 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:28.716022 27458 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:28.716877 26324 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:28.691077 27346 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:28.692391 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.707417 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.720595 27191 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:29:28.713786 26401 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.025s	sys 0.032s
I20260924 14:29:28.713495 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.722201 27468 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:28.722916 26254 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:28.723232 27189 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:28.726076 26398 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:28.727535 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.727977 27385 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:28.728381 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.729295 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.728436 27385 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:28.729907 27495 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:28.730211 27190 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:28.730432 27495 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:28.732769 27191 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:28.734313 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.740162 27385 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:28.739888 27350 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:28.726572 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.744217 27186 raft_consensus.cc:2396] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260924 14:29:28.704916 27185 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.009s	sys 0.043s
I20260924 14:29:28.744755 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:28.431985 26252 rpcz_store.cc:276] 0924 14:29:27.349313 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:27.350768 (+  1455us) service_pool.cc:227] Handling call
0924 14:29:27.416394 (+ 65626us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:27.442976 (+ 26582us) raft_consensus.cc:1520] Early marking committed up to index 2
0924 14:29:27.832533 (+389557us) log.cc:842] Serialized 14 byte log entry
0924 14:29:28.431158 (+598625us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:28.431219 (+    61us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:28.431269 (+    50us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:28.431359 (+    90us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:28.431521 (+   162us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":49260,"prepare.run_cpu_time_us":216,"prepare.run_wall_time_us":216,"raft.queue_time_us":1489,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":924840,"threads_started":2}
I20260924 14:29:28.745597 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:28.747020 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:28.747274 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.748275 27186 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:28.768316 27129 tablet_bootstrap.cc:492] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260924 14:29:28.709376 26400 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.022s	sys 0.026s
I20260924 14:29:28.762176 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:28.761473 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:28.760979 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:28.766464 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:28.763092 27190 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:28.759605 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.744844 27488 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:28.747236 27495 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:28.839859 27129 tablet_bootstrap.cc:492] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:28.827982 27409 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:28.745648 26251 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:28.762123 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:28.815407 27470 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.023s	sys 0.066s
I20260924 14:29:28.771617 27191 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:28.854403 27129 ts_tablet_manager.cc:1397] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 1.199s	user 0.044s	sys 0.022s
I20260924 14:29:28.773080 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:28.818636 27427 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:28.773187 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:28.863374 27470 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.024s	sys 0.066s
I20260924 14:29:28.840771 27483 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:28.709467 27418 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:28.887997 27186 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:29:28.895031 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 379260us
W20260924 14:29:28.911795 26371 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.032s	sys 0.023s
I20260924 14:29:28.874585 27181 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
W20260924 14:29:28.913198 26371 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.032s	sys 0.023s
I20260924 14:29:28.928454 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:28.901571 27194 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:28.932526 27434 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:28.886440 27483 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:28.890690 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:28.911729 27475 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:29:28.968020 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 463174us
I20260924 14:29:28.980674 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:28.982551 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:28.984866 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 307227us
W20260924 14:29:28.928973 27477 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.007s	sys 0.015s
I20260924 14:29:28.943048 27438 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:28.943056 27492 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:29.006314 27492 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:29.007726 27516 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.117s	user 0.003s	sys 0.000s
I20260924 14:29:29.008440 27492 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:29.001004 27488 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:29.009390 27488 raft_consensus.cc:686] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:29.010272 27488 consensus_queue.cc:237] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.011715 27129 raft_consensus.cc:348] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:29.012470 27129 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:29.013085 27129 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:28.999294 27411 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:29.015658 27411 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:28.998096 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:29.005335 27438 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:29.018612 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:29.019898 27519 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260924 14:29:29.020677 27518 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260924 14:29:28.938493 27372 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:28.987110 27355 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:29.025368 27355 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:29.025751 27355 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:29.027671 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:28.944674 27434 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:29.033164 27434 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:28.983733 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:29.023341 26369 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:29.023658 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:29.015316 27129 ts_tablet_manager.cc:1428] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.153s	user 0.002s	sys 0.000s
I20260924 14:29:28.994802 27392 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:28.994812 27467 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:28.992691 27385 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:29.044345 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:29.049597 26331 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:28.992456 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 108118us
I20260924 14:29:29.045565 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:29.046748 26224 leader_election.cc:398] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3.
I20260924 14:29:29.053319 26224 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:29.047977 26332 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:28.988699 27475 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.040084 27421 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:29.023378 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.051597 27189 raft_consensus.cc:2396] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260924 14:29:29.065157 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260924 14:29:29.066027 27502 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:29.066363 27354 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:29.066622 27502 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:29.068877 27354 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:29.003996 27458 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:29.068953 27191 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:29.069273 27502 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:29.046862 27392 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:29.041131 26400 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:29.041647 27434 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260924 14:29:29.023208 27495 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260924 14:29:29.050122 27470 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:29.056766 27129 tablet_bootstrap.cc:492] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:29.078855 26331 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:29.052855 27409 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:29.037181 27417 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:29.070290 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:29.120428 26402 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:29.071484 27421 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:29:29.099803 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 54293.7us
W20260924 14:29:29.106303 27522 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260924 14:29:29.076402 27182 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
W20260924 14:29:29.144515 26221 leader_election.cc:341] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:29:29.111047 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.117136 27458 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:29.083678 27385 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:29.076515 27391 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:29.083588 27392 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:29.076699 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.093029 27526 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260924 14:29:29.109537 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:29.148473 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:29.095325 27354 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:29.117220 26330 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:29.143374 27421 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:29.164322 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 220768us
I20260924 14:29:29.193750 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:29.146355 27470 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:29.210451 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:29.211201 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:29.212105 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 67639us
I20260924 14:29:29.179603 27483 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:29.196991 27434 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:29.167963 27385 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:29.227375 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:29.201699 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.246536 26399 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:29.240098 27434 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:29.246950 27470 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:29.247771 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:29.248875 27434 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:29.249606 26405 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260924 14:29:29.250831 27532 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260924 14:29:29.251139 26397 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:29.251904 26404 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:29.252857 27475 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:29.238931 27531 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260924 14:29:29.253472 27182 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260924 14:29:29.253896 27182 pending_rounds.cc:76] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 2
I20260924 14:29:29.238991 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:29.212401 27492 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:29.254820 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.255007 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.255626 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.246389 26400 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:29.255046 27492 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:29.256090 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.257290 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.257243 27492 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.259791 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:29.260207 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:29.260309 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:29.261019 26329 raft_consensus.cc:2354] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3.
I20260924 14:29:29.261658 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:29.261790 27185 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:29.261951 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:29.262245 27185 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:29.262383 26328 raft_consensus.cc:2354] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3.
I20260924 14:29:29.207759 27354 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:29.211887 27372 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:29.223637 27404 consensus_meta.cc:287] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.004s
I20260924 14:29:29.263528 27372 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:29.263106 27185 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.263592 27404 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.264078 27372 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:29.264510 27185 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:29.264909 27372 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.265661 27404 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:29.254154 27182 pending_rounds.cc:97] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260924 14:29:29.266986 27182 raft_consensus.cc:2866] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:29.261067 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:29.268898 26331 raft_consensus.cc:2354] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3.
I20260924 14:29:29.174194 27391 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:29.270565 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:29.271292 27391 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:29.271821 26324 raft_consensus.cc:2379] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:29.272552 27391 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:29.273130 27391 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:29.262844 27385 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:29.275739 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:29.277267 27192 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:29.281319 27391 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:29.280053 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:29.277465 26400 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:29.281342 26398 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:29.284627 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:29.285034 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:29.284664 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:29.208091 27417 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:29.240527 27355 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:29.290824 26295 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:29.326059 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:29.306140 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 78878.6us
W20260924 14:29:29.249827 27524 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260924 14:29:29.336638 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:29.337733 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:29.338706 26254 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:29.342144 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:29.345041 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:29.314724 27353 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
W20260924 14:29:29.348654 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 303737us
I20260924 14:29:29.350936 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:29.321574 27385 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:29.356021 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:29.356920 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:29.324307 26403 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:29.237811 27527 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260924 14:29:29.347276 27185 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260924 14:29:29.363188 26406 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:29.361953 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:29.340510 27458 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:29.365433 26331 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:29.362197 27185 pending_rounds.cc:76] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 1
I20260924 14:29:29.374189 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:29.356078 27385 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:29.331682 27482 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260924 14:29:29.378000 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:29.361330 27353 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:29.381136 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:29.361034 27534 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:29.388947 27470 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260924 14:29:29.403728 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:29.373342 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:29.404959 27528 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:29.405462 27528 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:29.405807 27528 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:29.407277 27392 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:29.369356 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 85806.8us
I20260924 14:29:29.411458 27482 consensus_queue.cc:1046] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:29.378330 27185 pending_rounds.cc:97] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:29.421655 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:29.386101 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:29.386312 27385 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:29.413367 27185 raft_consensus.cc:2866] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:29.390507 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:29.414340 27534 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:29.452060 27355 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:29.471787 27552 raft_consensus.cc:3032] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:29.471501 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.440376 27470 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:29.420032 27477 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260924 14:29:29.472461 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:29.473280 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:29.474056 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.450721 27336 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:29.472105 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.484089 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:29.484418 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:29.485769 27391 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:29.453027 27427 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260924 14:29:29.487821 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:29.447324 27350 raft_consensus.cc:2741] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260924 14:29:29.492920 26295 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:29.453966 27534 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:29.489775 27552 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:29.452181 27533 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260924 14:29:29.522951 27552 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.442600 27125 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:29.500265 27470 raft_consensus.cc:686] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:29.514143 27188 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:29.514633 27191 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:29.490484 26397 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:29.504863 27189 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:29.526257 27188 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:29.475289 27404 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260924 14:29:29.530493 27336 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:29.530929 27336 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:29.531529 27336 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:29.511940 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:29.532087 26401 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:29.525931 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:29.498674 27392 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:29.533548 27392 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:29.534060 27392 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.501442 27182 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:29.530109 26403 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:29.530843 27188 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.496004 27409 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
W20260924 14:29:29.541028 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 477235us
I20260924 14:29:29.493953 27353 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:29.524324 27552 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:29.565446 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 68513.5us
I20260924 14:29:29.566668 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:29.525441 27554 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260924 14:29:29.553211 27539 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:29.569129 27129 tablet_bootstrap.cc:492] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260924 14:29:29.543279 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 98539.5us
I20260924 14:29:29.561038 27495 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260924 14:29:29.558732 27188 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:29.581235 27470 consensus_queue.cc:237] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.576166 27477 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:29.573954 27561 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:29.595985 27126 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:29.594975 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:29.597863 26332 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:29.600085 27129 tablet_bootstrap.cc:492] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:29.602273 27129 ts_tablet_manager.cc:1397] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.554s	user 0.023s	sys 0.004s
I20260924 14:29:29.608114 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:29.611112 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:29.612036 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:29.579846 27392 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:29.613075 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 197002us
I20260924 14:29:29.616255 27468 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:29.618933 27562 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:29.612876 27560 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:29.614096 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:29.624082 27565 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:29.614503 27477 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:29.632900 27562 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:29.646366 27529 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.434s	user 0.003s	sys 0.009s
I20260924 14:29:29.646677 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:29.648044 27495 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:29.647526 27560 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:29.643961 27391 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:29.654076 27551 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260924 14:29:29.631880 27392 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:29.656172 27421 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.672585 27535 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:29.657845 26401 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:29.682924 27477 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.705955 27573 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:29.695966 27565 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:29.711131 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:29.642396 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46506.4us
I20260924 14:29:29.716034 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:29.694355 27495 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:29.727589 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:29.727185 27576 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:29.737802 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:29.710078 26324 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:29.722513 27186 raft_consensus.cc:2354] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:29.692602 27533 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:29.745669 26258 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:29.746376 26402 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:29.746201 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:29.742483 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 241188us
I20260924 14:29:29.715598 27392 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:29.735908 27535 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:29.684290 27385 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:29.745502 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:29.747524 27576 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:29.729813 27190 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260924 14:29:29.762301 27392 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:29.783514 26255 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:29:29.785246 27562 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:29.786497 26330 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:29.788198 27421 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:29.791136 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:29.791887 27427 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:29.792282 27427 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:29.792626 27565 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:29.776384 26369 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:29.780889 27576 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:29.715602 27528 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:29.786538 27129 raft_consensus.cc:348] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:29.799031 27129 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:29.763167 27570 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:29.799949 27129 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:29.795359 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:29.815732 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:29.822023 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:29.788200 27182 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:29.824198 26323 raft_consensus.cc:2354] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:29.824425 27129 ts_tablet_manager.cc:1428] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.222s	user 0.006s	sys 0.001s
I20260924 14:29:29.824910 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:29.776567 27350 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:29.793119 27427 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:29.829103 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:29.832615 26401 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:29:29.761148 27533 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:29.740476 27573 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:29.798707 27418 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:29.801685 27417 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:29.869599 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:29.836432 26258 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:29.870693 26224 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:29.818778 27523 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:29.837034 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 88655.5us
I20260924 14:29:29.816828 27565 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:29.834429 27409 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:29.849673 27404 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:29.885746 26224 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:29.951040 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65417.6us
I20260924 14:29:29.813862 27534 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:29.853895 27468 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:29.864346 27562 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260924 14:29:30.094558 27470 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.017s	sys 0.057s
W20260924 14:29:30.095090 27470 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.017s	sys 0.057s
I20260924 14:29:29.777252 27495 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:29.851193 27421 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.105s
I20260924 14:29:29.860249 27560 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:29.864660 27536 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:29.849994 27336 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:29.825923 27392 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:29.864621 27350 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:29.948359 27547 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:29.946255 27354 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:30.110141 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:30.166669 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:29.826287 27570 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:30.090728 27427 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:30.067819 27546 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:29.874326 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:30.106277 27438 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:30.091369 27434 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:29.842439 27535 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:30.110134 27526 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:30.092737 27562 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:30.201057 27572 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260924 14:29:29.873240 27123 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:30.106629 27467 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:29.955895 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 81733.9us
I20260924 14:29:29.957262 27530 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:30.236430 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 69882.8us
I20260924 14:29:29.873306 27418 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:30.107237 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:29.946388 27571 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.335s
I20260924 14:29:30.089962 27565 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:30.071600 27404 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:30.186784 27547 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:30.186177 27495 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:30.106590 27468 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:30.203776 27483 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:30.289422 27533 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:30.267395 27354 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:30.323226 27528 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:30.177831 27536 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:30.308691 27562 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:30.179337 27578 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.154s
I20260924 14:29:30.301682 27438 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:30.301263 27188 consensus_meta.cc:287] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.722s	user 0.011s	sys 0.000s
I20260924 14:29:30.295553 27470 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:30.349007 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:30.338747 27535 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:30.339560 27523 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:30.343436 27560 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:30.372854 26328 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.000s	sys 0.012s
W20260924 14:29:30.455964 26328 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.001s	sys 0.012s
I20260924 14:29:30.459403 27129 tablet_bootstrap.cc:492] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:30.347904 27547 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:30.305044 27434 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:30.273126 27350 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:30.313802 27594 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:30.412925 27565 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:30.309567 27526 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:30.376672 26400 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:30.296244 27385 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:30.344130 27482 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260924 14:29:30.359845 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:30.385138 27527 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.005s	sys 0.011s
I20260924 14:29:30.420620 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:30.498363 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:30.500149 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:30.501040 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:30.359616 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 502528us
I20260924 14:29:30.425211 27534 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:30.322206 27467 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:30.425479 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:30.431715 27585 raft_consensus.cc:3032] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:30.431903 27336 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:30.433394 27188 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260924 14:29:30.433468 27181 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:30.434968 27418 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:30.339941 27530 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:30.439994 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.443256 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 413) took 1233 ms. Trace:
I20260924 14:29:30.533123 27188 pending_rounds.cc:76] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 2
I20260924 14:29:30.447521 26325 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:30.448349 27392 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:30.449759 27182 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:30.385104 27417 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:30.451072 27570 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:30.454485 26399 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:30.377774 27409 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:30.567549 27409 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:30.466061 26328 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:30.569892 27409 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:30.571259 26328 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:30.571393 27434 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:30.385918 27573 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:30.452680 27185 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:30.370967 27354 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:30.577066 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 192095us
I20260924 14:29:30.549906 27123 leader_election.cc:398] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3.
I20260924 14:29:30.473109 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:30.489253 27468 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:30.397773 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:30.460177 27523 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:30.503268 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:30.523365 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 102841us
W20260924 14:29:30.504192 27527 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.005s	sys 0.012s
I20260924 14:29:30.585366 26401 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:30.533722 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:30.537034 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:30.490262 27600 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:30.365454 27483 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:30.546171 26402 rpcz_store.cc:276] 0924 14:29:29.209434 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.210183 (+   749us) service_pool.cc:227] Handling call
0924 14:29:30.442906 (+1232723us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54006e6ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:30.443137 (+   231us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1230411136}
I20260924 14:29:30.488442 27594 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:30.522902 27585 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:30.552196 26325 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:30.602646 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:30.603070 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:30.603356 26325 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:30.603865 27182 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:30.595738 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 107) took 1014 ms. Trace:
I20260924 14:29:30.604691 27185 rpcz_store.cc:276] 0924 14:29:29.592633 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.595532 (+  2899us) service_pool.cc:227] Handling call
0924 14:29:30.451919 (+856387us) spinlock_profiling.cc:243] Waited 6.08 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eebce 000000000034aea9 00007f09be21aa30 00007f09be2e07b4 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:30.452155 (+   236us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.577054 (+124899us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:30.590748 (+ 13694us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13990784}
I20260924 14:29:30.607466 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:30.551719 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:30.562477 27536 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:30.566833 27467 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:30.551605 27188 pending_rounds.cc:97] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 3.3
W20260924 14:29:30.473083 26332 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.002s	sys 0.012s
I20260924 14:29:30.559508 26324 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:30.582530 26254 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:30.583772 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:30.588699 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:30.557418 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:30.591773 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 54829.1us
I20260924 14:29:30.593554 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.590027 27409 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:30.597836 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:30.580657 27123 leader_election.cc:401] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:29:30.600291 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:30.611219 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:30.598079 27592 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:30.620167 26221 leader_election.cc:398] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:30.620586 26295 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:30.645985 26221 leader_election.cc:401] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260924 14:29:30.572372 27417 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:30.592435 27404 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:30.596204 27438 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:30.620034 27188 raft_consensus.cc:2866] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:30.633095 26325 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
W20260924 14:29:30.600919 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 219525us
I20260924 14:29:30.634714 26256 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
W20260924 14:29:30.636744 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 103614us
W20260924 14:29:30.637163 27577 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.464s	user 0.003s	sys 0.000s
I20260924 14:29:30.609685 27585 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:30.638940 27526 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:30.641296 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:30.669448 26401 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:30.670598 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:30.670867 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:30.607018 27570 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election won for term 4
W20260924 14:29:30.644685 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 44482us
I20260924 14:29:30.646205 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.647744 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:30.615672 26332 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.002s	sys 0.012s
I20260924 14:29:30.626869 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:30.652962 27590 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:30.653227 27467 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:30.649519 27392 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 3
I20260924 14:29:30.648342 27409 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:30.653936 27191 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:30.654126 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:30.624701 27591 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:30.673471 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52955.4us
I20260924 14:29:30.677881 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:30.649300 26397 raft_consensus.cc:1261] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:30.647586 27353 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:30.613917 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:30.649022 27355 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:30.681934 27404 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:30.674775 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:30.600021 27600 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:30.653466 27571 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:30.699378 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:30.651566 27398 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
W20260924 14:29:30.729918 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 130123us
I20260924 14:29:30.690584 27570 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:30.729923 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:30.691047 27585 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:30.734270 27186 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:30.734843 27600 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:30.714823 27547 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:30.653995 27536 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260924 14:29:30.699955 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.737267 27547 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:30.737354 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 354) took 1389 ms. Trace:
I20260924 14:29:30.691567 27468 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260924 14:29:30.679641 26403 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:30.698392 27608 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:30.703620 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 121) took 1216 ms. Trace:
I20260924 14:29:30.732170 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:30.735560 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 108) took 1142 ms. Trace:
I20260924 14:29:30.727689 26402 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260924 14:29:30.740648 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65965.1us
I20260924 14:29:30.703157 27354 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:30.752662 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:30.752869 27186 raft_consensus.cc:1261] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260924 14:29:30.735594 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:30.757421 26298 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:30.753712 27612 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260924 14:29:30.753568 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:30.749261 27547 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:30.757383 27186 pending_rounds.cc:76] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 1
I20260924 14:29:30.704792 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:30.772693 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:30.730672 27562 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:30.757727 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:30.696244 27592 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260924 14:29:30.741920 27404 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:30.782845 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.747298 27608 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:30.765537 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 113) took 1170 ms. Trace:
I20260924 14:29:30.762782 27601 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:30.766151 27409 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:30.766304 27482 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:30.754324 27573 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:30.766779 27547 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:30.800273 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:30.737802 26329 rpcz_store.cc:276] 0924 14:29:29.347868 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.349345 (+  1477us) service_pool.cc:227] Handling call
0924 14:29:29.574577 (+225232us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eebce 000000000034aea9 00007f09be21aa30 00007f09be2e07b4 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:29.580281 (+  5704us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:29.595859 (+ 15578us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:29:30.429964 (+834105us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b18003ac540. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b8fdd82c 00007f09b64089dc 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6423155 00007f09bcd23167 00007f09b9358807 00007f09b9367f0f
0924 14:29:30.430147 (+   183us) log.cc:842] Serialized 14 byte log entry
0924 14:29:30.719671 (+289524us) log.cc:842] Serialized 14 byte log entry
0924 14:29:30.737014 (+ 17343us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:30.737065 (+    51us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:30.737113 (+    48us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:30.737192 (+    79us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:30.737299 (+   107us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":23589,"prepare.queue_time_us":270204,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":21008,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":27430656,"thread_start_us":481719,"threads_started":3,"wal-append.queue_time_us":265904}
I20260924 14:29:30.773604 26405 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:30.802767 27547 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:30.755152 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 84374.3us
I20260924 14:29:30.782341 27530 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:30.771700 27594 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:30.771404 27560 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:30.742065 27502 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:30.780763 27483 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:30.762851 27597 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:30.744040 27599 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:30.772186 27186 pending_rounds.cc:97] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:30.822932 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:30.757689 27602 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:30.771166 27570 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:30.756829 27191 rpcz_store.cc:276] 0924 14:29:29.593085 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.595826 (+  2741us) service_pool.cc:227] Handling call
0924 14:29:30.653374 (+1057548us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:30.653592 (+   218us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.707982 (+ 54390us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:30.735501 (+ 27519us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3618688}
I20260924 14:29:30.699695 27590 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:30.814229 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:30.791142 27608 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:30.819154 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.796839 27523 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:30.805707 26400 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:30.750842 27188 rpcz_store.cc:276] 0924 14:29:29.487179 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.492409 (+  5230us) service_pool.cc:227] Handling call
0924 14:29:29.513988 (+ 21579us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.673490 (+1159502us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:30.700612 (+ 27122us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:30.832597 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:30.808164 27482 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260924 14:29:30.804848 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 112435us
I20260924 14:29:30.836299 27617 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260924 14:29:30.840127 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.814843 27502 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:30.823904 27186 raft_consensus.cc:2866] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:30.823490 27417 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:30.849156 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.850594 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.852806 26402 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
W20260924 14:29:30.821677 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 51218.2us
I20260924 14:29:30.856812 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.824920 27597 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:30.830897 27594 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:30.859817 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:30.865094 27613 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260924 14:29:30.828815 27602 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:30.826748 27355 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.029s
I20260924 14:29:30.872296 27562 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:30.871991 27594 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:30.873320 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:30.790428 27192 rpcz_store.cc:276] 0924 14:29:29.595020 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.596259 (+  1239us) service_pool.cc:227] Handling call
0924 14:29:29.618419 (+ 22160us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:29.698546 (+ 80127us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:29.698588 (+    42us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:29.699431 (+   843us) log.cc:842] Serialized 26 byte log entry
0924 14:29:29.776308 (+ 76877us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:29.827820 (+ 51512us) log.cc:842] Serialized 14 byte log entry
0924 14:29:29.849322 (+ 21502us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:29.856434 (+  7112us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:30.636612 (+780178us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:29:30.763705 (+127093us) raft_consensus.cc:1671] finished
0924 14:29:30.765314 (+  1609us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:30.765475 (+   161us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":4694,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":2243,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":160,"spinlock_wait_cycles":35690368,"thread_start_us":153850,"threads_started":2,"wal-append.queue_time_us":247813}
I20260924 14:29:30.957419 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 631) took 1237 ms. Trace:
I20260924 14:29:30.958029 26257 rpcz_store.cc:276] 0924 14:29:29.719983 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.722169 (+  2186us) service_pool.cc:227] Handling call
0924 14:29:30.426826 (+704657us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eebce 000000000034aea9 00007f09be21aa30 00007f09be2e07b4 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:30.427010 (+   184us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.548136 (+121126us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:30.550595 (+  2459us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:30.559908 (+  9313us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:30.565447 (+  5539us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:30.957065 (+391618us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54002a1554. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:30.957146 (+    81us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:30.957342 (+   196us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":278684800}
I20260924 14:29:30.847497 27123 leader_election.cc:398] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3.
I20260924 14:29:30.848395 27502 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:29:30.933077 27427 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.656s	user 0.005s	sys 0.007s
I20260924 14:29:30.934450 27590 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:30.935475 26398 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:30.948329 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 356) took 1593 ms. Trace:
I20260924 14:29:30.948928 27602 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:30.949944 27597 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:30.951226 27189 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:30.975011 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 130893us
I20260924 14:29:30.829330 27404 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:30.944547 27491 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:30.977238 27129 tablet_bootstrap.cc:492] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:30.963871 27192 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260924 14:29:30.974578 27495 consensus_meta.cc:287] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.625s	user 0.013s	sys 0.000s
W20260924 14:29:30.965083 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 130302us
I20260924 14:29:30.975669 26323 rpcz_store.cc:276] 0924 14:29:29.354563 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.357914 (+  3351us) service_pool.cc:227] Handling call
0924 14:29:30.943956 (+1586042us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54007d3cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:30.948163 (+  4207us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":855940864}
I20260924 14:29:30.988778 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.014312 27129 tablet_bootstrap.cc:492] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:31.015565 27129 ts_tablet_manager.cc:1397] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.562s	user 0.026s	sys 0.001s
W20260924 14:29:31.025048 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 200359us
I20260924 14:29:31.025780 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:31.027882 27434 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:31.028477 27434 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:31.029438 27434 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:31.032497 27599 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:31.033072 27599 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:31.034133 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.039264 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 114) took 1444 ms. Trace:
I20260924 14:29:31.020651 27597 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:31.044014 27597 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:31.046147 27597 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:31.039733 27194 rpcz_store.cc:276] 0924 14:29:29.595349 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.595616 (+   267us) service_pool.cc:227] Handling call
0924 14:29:31.033530 (+1437914us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:31.035393 (+  1863us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:31.035650 (+   257us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:31.035698 (+    48us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:31.038964 (+  3266us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:31.039022 (+    58us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.039092 (+    70us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:31.039210 (+   118us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":50184576}
I20260924 14:29:31.005165 27392 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3.
I20260924 14:29:31.043140 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:30.986588 27491 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:31.031432 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:31.034904 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:31.046779 26249 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260924 14:29:31.046705 27123 leader_election.cc:398] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3.
I20260924 14:29:31.060174 27191 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:31.061035 27491 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:31.061232 27592 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:31.063110 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.063025 26295 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:31.064312 26221 leader_election.cc:398] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:31.067984 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:31.069304 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:31.070636 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 637) took 1115 ms. Trace:
I20260924 14:29:31.071024 26258 rpcz_store.cc:276] 0924 14:29:29.955228 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.955494 (+   266us) service_pool.cc:227] Handling call
0924 14:29:30.162862 (+207368us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.533418 (+370556us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:30.533448 (+    30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:30.533487 (+    39us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:30.533514 (+    27us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.070393 (+536879us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54003822d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:31.070441 (+    48us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:31.070587 (+   146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":851742592}
I20260924 14:29:30.762593 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:29:31.059185 26249 pending_rounds.cc:76] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 1
W20260924 14:29:31.098343 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 72953.4us
I20260924 14:29:31.126751 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.062706 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.127887 26398 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:31.128039 27523 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:31.128649 27523 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:31.131042 27523 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:31.131805 26255 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:29:31.125191 27491 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:31.132238 27592 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:31.133766 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.134699 27491 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:31.137177 27491 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:31.062489 27181 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
W20260924 14:29:31.141824 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 78927.3us
W20260924 14:29:31.145372 26295 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260924 14:29:31.140327 26249 pending_rounds.cc:97] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:31.158356 26295 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:31.139514 27353 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:31.143594 27627 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.085s	user 0.004s	sys 0.000s
I20260924 14:29:31.145889 27592 consensus_queue.cc:1046] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260924 14:29:31.169032 26405 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:31.134649 27477 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:31.140839 27629 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:31.150326 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.176056 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:31.171504 27192 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:31.157584 26249 raft_consensus.cc:2866] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:31.142728 27585 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260924 14:29:31.157197 27591 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:31.150279 27547 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:31.151753 27502 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:31.180578 27629 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:31.193069 27592 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:31.179355 26257 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:31.183969 27192 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:31.183503 27353 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:31.207744 27353 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:31.208714 27353 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:31.210089 27353 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:31.181138 27630 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260924 14:29:31.204351 27581 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:31.213722 26254 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:29:31.216630 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 58357.5us
I20260924 14:29:31.195137 27591 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:31.225449 26327 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:31.238008 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 365) took 1511 ms. Trace:
I20260924 14:29:31.238586 26327 rpcz_store.cc:276] 0924 14:29:29.727050 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.753801 (+ 26751us) service_pool.cc:227] Handling call
0924 14:29:31.224922 (+1471121us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:31.225199 (+   277us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:31.237604 (+ 12405us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.237906 (+   302us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6274944}
I20260924 14:29:31.211679 27581 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:31.209179 27192 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:31.244081 27633 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260924 14:29:31.213161 27608 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:31.252907 27608 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:31.250267 27491 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:31.249889 27591 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:31.253554 27634 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260924 14:29:31.290948 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:31.267418 27192 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:31.248203 27592 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:31.291340 27630 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:31.269726 27629 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:31.272081 27491 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:31.314489 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 157293us
I20260924 14:29:31.314805 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.267899 27599 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:31.329141 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:31.329720 26328 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:31.329856 26405 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:31.320560 27502 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:31.291199 27591 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:31.314404 27547 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:31.307689 27630 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:31.324533 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:31.331590 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 123) took 1168 ms. Trace:
I20260924 14:29:31.338389 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.334776 27546 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:31.340464 27581 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:31.340150 26327 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:31.344358 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:31.345055 27491 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:31.345569 27602 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:31.346000 27602 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:31.346349 27602 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:31.346746 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 626) took 1638 ms. Trace:
W20260924 14:29:31.348438 27639 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260924 14:29:31.348675 27630 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:31.347146 26332 rpcz_store.cc:276] 0924 14:29:29.708593 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.709498 (+   905us) service_pool.cc:227] Handling call
0924 14:29:29.709870 (+   372us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:29.762582 (+ 52712us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:29.762625 (+    43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:29.773017 (+ 10392us) log.cc:842] Serialized 26 byte log entry
0924 14:29:31.346227 (+1573210us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b1800237220. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b8fdd82c 00007f09b64089dc 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411
0924 14:29:31.346364 (+   137us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:31.346396 (+    32us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.346454 (+    58us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:31.346495 (+    41us) raft_consensus.cc:1671] finished
0924 14:29:31.346506 (+    11us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:31.346697 (+   191us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":33719,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":9952,"spinlock_wait_cycles":6011392,"thread_start_us":903128,"threads_started":2,"wal-append.queue_time_us":872485}
I20260924 14:29:31.338902 27182 rpcz_store.cc:276] 0924 14:29:30.184253 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:30.604750 (+420497us) service_pool.cc:227] Handling call
0924 14:29:30.605125 (+   375us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.621788 (+ 16663us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:30.639575 (+ 17787us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:30.639664 (+    89us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:30.639707 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.319072 (+679365us) spinlock_profiling.cc:243] Waited 6.83 ms on lock 0x7b540001b8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:31.321617 (+  2545us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:31.328955 (+  7338us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":15705344}
I20260924 14:29:31.353293 26330 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:31.348778 27546 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:31.358245 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:31.361233 27583 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.751s	user 0.008s	sys 0.000s
I20260924 14:29:31.363124 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 119) took 1569 ms. Trace:
I20260924 14:29:31.363682 27190 rpcz_store.cc:276] 0924 14:29:29.793558 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.798289 (+  4731us) service_pool.cc:227] Handling call
0924 14:29:29.854274 (+ 55985us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.309961 (+455687us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:30.314916 (+  4955us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:30.320304 (+  5388us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:30.328076 (+  7772us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.362808 (+1034732us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b540073c654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:31.362892 (+    84us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:31.363052 (+   160us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1007265152}
I20260924 14:29:31.365571 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 634) took 1259 ms. Trace:
I20260924 14:29:31.366024 26400 rpcz_store.cc:276] 0924 14:29:30.106374 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:30.702754 (+596380us) service_pool.cc:227] Handling call
0924 14:29:30.742143 (+ 39389us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.832102 (+ 89959us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.365276 (+533174us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b54001c1454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:31.365506 (+   230us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":114234752}
I20260924 14:29:31.370555 27594 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:31.355618 27636 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:31.370633 27418 raft_consensus.cc:3032] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:31.373013 27640 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:31.377069 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 356) took 1581 ms. Trace:
I20260924 14:29:31.358275 26405 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:31.378762 27591 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:31.384518 27606 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.013s	sys 0.019s
I20260924 14:29:31.379647 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.381623 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:31.383996 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 626) took 1514 ms. Trace:
W20260924 14:29:31.386009 26369 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 1.520s (SENT)
I20260924 14:29:31.393143 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.387194 27418 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:29:31.391599 27606 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.013s	sys 0.019s
I20260924 14:29:31.401530 27599 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:31.404744 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:31.405413 26404 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:31.393772 26406 rpcz_store.cc:276] 0924 14:29:29.869426 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.871310 (+  1884us) service_pool.cc:227] Handling call
0924 14:29:30.089979 (+218669us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.416168 (+326189us) raft_consensus.cc:1520] Early marking committed up to index 2
0924 14:29:30.601242 (+185074us) log.cc:842] Serialized 14 byte log entry
0924 14:29:30.745485 (+144243us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:30.745556 (+    71us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:30.745613 (+    57us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.383750 (+638137us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b5400335754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:31.383803 (+    53us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:31.383936 (+   133us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":280655,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":54,"raft.queue_time_us":6453,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":3565,"spinlock_wait_cycles":749480704,"thread_start_us":285435,"threads_started":2,"wal-append.queue_time_us":182248}
I20260924 14:29:31.407521 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:29:31.408281 26369 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 2.027s (SENT) [suppressed 9 similar messages]
I20260924 14:29:31.348029 27592 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:31.409550 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 630) took 1436 ms. Trace:
I20260924 14:29:31.390074 27488 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260924 14:29:31.391345 27636 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:31.391378 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:31.416899 27636 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:31.419528 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:31.402812 27418 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:31.420754 27546 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:31.401429 26400 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:31.388746 26249 rpcz_store.cc:276] 0924 14:29:29.795868 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.798854 (+  2986us) service_pool.cc:227] Handling call
0924 14:29:29.852316 (+ 53462us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:31.192029 (+1339713us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.376740 (+184711us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002cbfd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:31.376991 (+   251us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2624334720}
I20260924 14:29:31.421136 27418 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:31.409998 26404 rpcz_store.cc:276] 0924 14:29:29.972803 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.973052 (+   249us) service_pool.cc:227] Handling call
0924 14:29:31.405045 (+1431993us) spinlock_profiling.cc:243] Waited 96.2 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:31.405240 (+   195us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:31.409278 (+  4038us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.409499 (+   221us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":221342464}
I20260924 14:29:31.412169 27353 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:31.422088 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:31.395383 27594 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260924 14:29:31.420799 26255 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:31.438874 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:31.439438 27596 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:31.439868 27596 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:31.440657 27596 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:31.439868 27576 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:31.443877 27576 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:31.444242 27576 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:31.448289 27602 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:31.449015 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 624) took 1741 ms. Trace:
I20260924 14:29:31.449388 26331 rpcz_store.cc:276] 0924 14:29:29.707133 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.707256 (+   123us) service_pool.cc:227] Handling call
0924 14:29:29.707641 (+   385us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:29.948988 (+241347us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:29.949019 (+    31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:29.949063 (+    44us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:29.949090 (+    27us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.448795 (+1499705us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b540001efd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:31.448840 (+    45us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:31.448961 (+   121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":554567936}
I20260924 14:29:31.450565 27602 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:31.451918 27630 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:31.452374 27630 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:31.453224 27630 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:31.454512 27630 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:31.455303 27192 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260924 14:29:31.455799 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 123) took 1217 ms. Trace:
I20260924 14:29:31.456212 27185 rpcz_store.cc:276] 0924 14:29:30.301160 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:30.610147 (+308987us) service_pool.cc:227] Handling call
0924 14:29:30.651572 (+ 41425us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.673413 (+ 21841us) raft_consensus.cc:1520] Early marking committed up to index 2
0924 14:29:30.754402 (+ 80989us) log.cc:842] Serialized 14 byte log entry
0924 14:29:31.455268 (+700866us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b1800032b00. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09b6261990 00007f09b6423864 00007f09b6423155 00007f09b93463d0 00007f09b9358576 00007f09b93690d7 00007f09b9369058
0924 14:29:31.455427 (+   159us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:31.455497 (+    70us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:31.455551 (+    54us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.455617 (+    66us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:31.455739 (+   122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":384,"prepare.queue_time_us":107683,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"spinlock_wait_cycles":3123968,"thread_start_us":88223,"threads_started":2,"wal-append.queue_time_us":80651}
I20260924 14:29:31.455705 27192 pending_rounds.cc:76] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 1
I20260924 14:29:31.458303 27192 pending_rounds.cc:97] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:31.458465 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 78) took 2213 ms. Trace:
I20260924 14:29:31.458608 27192 raft_consensus.cc:2866] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:31.451584 27353 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:31.458847 26399 rpcz_store.cc:276] 0924 14:29:29.245115 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.248631 (+  3516us) service_pool.cc:227] Handling call
0924 14:29:30.454256 (+1205625us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.563703 (+109447us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.458261 (+894558us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54006e6ed0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:31.458390 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2771954560}
W20260924 14:29:31.460273 27565 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260924 14:29:31.461277 27353 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:31.465564 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 86) took 1177 ms. Trace:
I20260924 14:29:31.466140 26403 rpcz_store.cc:276] 0924 14:29:30.287708 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:30.311173 (+ 23465us) service_pool.cc:227] Handling call
0924 14:29:31.465343 (+1154170us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54001c1e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:31.465487 (+   144us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":415723904}
I20260924 14:29:31.467839 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 126) took 1014 ms. Trace:
I20260924 14:29:31.468422 27188 rpcz_store.cc:276] 0924 14:29:30.474532 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:30.833932 (+359400us) service_pool.cc:227] Handling call
0924 14:29:30.834124 (+   192us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:31.467386 (+633262us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:31.467419 (+    33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:31.467488 (+    69us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:31.467523 (+    35us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.467595 (+    72us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:31.467762 (+   167us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":632928}
I20260924 14:29:31.473317 26401 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260924 14:29:31.476383 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 68097.3us
I20260924 14:29:31.485188 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:31.530342 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:31.532459 26224 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.531225 26331 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:31.546917 27502 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:31.547734 27502 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:31.550402 27502 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:31.552380 27129 raft_consensus.cc:348] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:31.553479 27129 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:31.554320 27129 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:31.534401 27392 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:31.529548 27630 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:31.539394 27594 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260924 14:29:31.534991 27607 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260924 14:29:31.569623 26406 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:31.551100 27502 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:31.572201 26404 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:31.572835 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:31.573530 26298 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:31.580113 27576 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:31.575023 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 727168us
I20260924 14:29:31.582130 27576 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:31.582698 27576 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:31.583117 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:31.561686 27630 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:31.584436 27630 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:31.576006 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:31.564811 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:31.573333 27594 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:31.586403 26253 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260924 14:29:31.586802 26253 pending_rounds.cc:76] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 1
I20260924 14:29:31.586571 26224 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.587040 26253 pending_rounds.cc:97] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:31.587327 26253 raft_consensus.cc:2866] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:31.595379 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:31.597451 26331 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:31.597954 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:31.600823 27129 ts_tablet_manager.cc:1428] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.585s	user 0.010s	sys 0.000s
W20260924 14:29:31.601497 27129 ts_tablet_manager.cc:1460] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Tablet startup took 585ms
I20260924 14:29:31.606799 27129 tablet_bootstrap.cc:492] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:31.598968 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:31.605136 27589 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:31.609434 27491 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:31.610026 27491 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:31.610980 27491 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:31.618160 26323 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:31.619853 27417 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:31.621137 27427 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:31.621577 27427 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:31.609151 27589 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:31.622467 27427 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:31.626173 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.619110 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.629464 27597 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:31.628440 27589 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:31.590678 27626 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:31.586397 27594 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:29:31.555858 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 429167us
I20260924 14:29:31.642052 27588 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
W20260924 14:29:31.649049 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47449.7us
I20260924 14:29:31.653748 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:31.658320 26369 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 1.793s (SENT)
I20260924 14:29:31.662345 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.601164 27609 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260924 14:29:31.636554 27372 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260924 14:29:31.677557 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:31.660240 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 77209.4us
I20260924 14:29:31.671742 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:31.672794 27491 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260924 14:29:31.661882 27123 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.683835 27581 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:31.686775 27602 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:31.706691 27649 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:31.695313 27570 consensus_queue.cc:1046] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260924 14:29:31.695940 27523 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260924 14:29:31.712942 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:31.702497 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:31.701702 27589 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:31.694561 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 423) took 2060 ms. Trace:
I20260924 14:29:31.708555 26326 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260924 14:29:31.718941 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:31.735081 27647 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260924 14:29:31.735929 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:31.736428 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 80959.8us
I20260924 14:29:31.742520 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 633) took 1768 ms. Trace:
I20260924 14:29:31.746657 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:31.747174 26401 rpcz_store.cc:276] 0924 14:29:29.973934 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:30.680857 (+706923us) service_pool.cc:227] Handling call
0924 14:29:31.472800 (+791943us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:31.474053 (+  1253us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.742250 (+268197us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54001c1e50. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:31.742443 (+   193us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1820340480}
I20260924 14:29:31.726962 26397 rpcz_store.cc:276] 0924 14:29:29.634037 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.634185 (+   148us) service_pool.cc:227] Handling call
0924 14:29:30.425828 (+791643us) spinlock_profiling.cc:243] Waited 7.02 ms on lock 0x7b54004c00a8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2c7434 00007f09be28f877 00007f09be28f2c7 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:30.425968 (+   140us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:30.721053 (+295085us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.683037 (+961984us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b540073dcd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:31.690416 (+  7379us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":434402048}
I20260924 14:29:31.751111 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.751283 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:31.731134 27589 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:31.748725 27502 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:31.752538 27418 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260924 14:29:31.777179 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:31.777611 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:31.777285 27654 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:31.753870 27192 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:31.749238 27526 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260924 14:29:31.772760 27502 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:31.797485 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:31.771225 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:31.779314 27438 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:31.808843 26325 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260924 14:29:31.787572 27570 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:31.815747 27194 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260924 14:29:31.800930 27502 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:31.829174 27652 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.230s	user 0.003s	sys 0.000s
I20260924 14:29:31.805508 27354 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260924 14:29:31.816708 26397 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:31.818367 27125 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:31.809407 27438 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:31.815840 27596 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:31.830271 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:31.809973 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 120) took 2013 ms. Trace:
W20260924 14:29:31.857329 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 166559us
I20260924 14:29:31.809746 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:31.791440 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.873211 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.863011 27547 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:31.831171 27658 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260924 14:29:31.869329 27654 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:31.928061 26399 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:31.878537 27547 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:31.947009 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:31.950177 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 77096.9us
I20260924 14:29:31.913872 27565 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:31.876197 27185 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
W20260924 14:29:31.879649 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 60205.6us
I20260924 14:29:31.982167 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:31.983520 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:31.958214 27637 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260924 14:29:31.964432 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:31.961602 27573 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:29:31.965853 27655 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
W20260924 14:29:31.933471 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 149505us
I20260924 14:29:31.955770 27495 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260924 14:29:31.954041 27602 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:31.866649 27189 rpcz_store.cc:276] 0924 14:29:29.793962 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.798617 (+  4655us) service_pool.cc:227] Handling call
0924 14:29:30.950639 (+1152022us) spinlock_profiling.cc:243] Waited 8.57 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:30.950973 (+   334us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:31.610104 (+659131us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:31.610137 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:31.610602 (+   465us) log.cc:842] Serialized 26 byte log entry
0924 14:29:31.690981 (+ 80379us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:31.728779 (+ 37798us) log.cc:842] Serialized 14 byte log entry
0924 14:29:31.748652 (+ 19873us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:31.762481 (+ 13829us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:31.763848 (+  1367us) raft_consensus.cc:1671] finished
0924 14:29:31.786673 (+ 22825us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:31.807534 (+ 20861us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":398891,"prepare.queue_time_us":79147,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":1657,"raft.queue_time_us":405925,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":209,"spinlock_wait_cycles":19730176,"thread_start_us":164148,"threads_started":3,"wal-append.queue_time_us":80495}
I20260924 14:29:31.967550 26399 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:31.967904 27547 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:31.968341 27609 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260924 14:29:31.975672 26369 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 2.119s (SENT)
I20260924 14:29:31.979144 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:31.979763 27589 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:31.980191 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:31.981251 27190 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:31.981583 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:31.994999 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:32.048913 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 54009.6us
I20260924 14:29:31.987197 27427 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:31.931217 27438 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:32.049885 27597 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:32.051630 26252 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260924 14:29:32.051879 27438 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:32.049108 27547 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:31.994722 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:31.987603 27591 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260924 14:29:31.979332 27594 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:32.064639 27654 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:32.073524 27565 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:32.074853 27565 raft_consensus.cc:3032] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:32.075417 27565 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:32.065861 27194 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:32.076683 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.077667 26295 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:32.079017 27404 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:32.074818 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:31.991482 27573 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:32.060091 27188 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:32.076174 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:32.086162 27578 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:32.086460 27594 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:32.071796 27654 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:32.056530 27353 consensus_queue.cc:1159] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "45ab97b763804767846e9d27a01b561f" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:29:32.087090 27182 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:32.079537 27404 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:32.087839 26258 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:32.089366 27601 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:32.092083 27502 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:32.036048 27125 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:32.076232 27565 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:32.079210 27580 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260924 14:29:32.079000 27194 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:32.058642 27630 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:32.119482 27649 raft_consensus.cc:3032] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:32.119967 27649 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:32.120306 27502 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:32.120705 27630 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:32.113816 27404 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:32.120798 27649 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:32.121464 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 129985us
I20260924 14:29:32.122316 27191 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:29:32.122718 27560 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:32.123554 27565 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:32.125708 26331 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:32.107379 26404 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:32.126921 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:32.069474 27589 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260924 14:29:32.100394 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 372) took 1280 ms. Trace:
I20260924 14:29:32.102360 27624 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:32.138064 26326 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:32.134204 26329 rpcz_store.cc:276] 0924 14:29:30.817790 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:30.820987 (+  3197us) service_pool.cc:227] Handling call
0924 14:29:31.962796 (+1141809us) spinlock_profiling.cc:243] Waited 8.61 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:32.098512 (+135716us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":19811200}
I20260924 14:29:32.125424 27194 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:32.141299 27194 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:32.142382 27194 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:32.143822 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:32.145324 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 129) took 1412 ms. Trace:
I20260924 14:29:32.145803 27186 rpcz_store.cc:276] 0924 14:29:30.755537 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:30.864870 (+109333us) service_pool.cc:227] Handling call
0924 14:29:31.816543 (+951673us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:31.816772 (+   229us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:31.820464 (+  3692us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:31.820514 (+    50us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:31.820563 (+    49us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:31.820609 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:32.145078 (+324469us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b540000d054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:32.145129 (+    51us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:32.145266 (+   137us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":36916992}
I20260924 14:29:32.147485 27589 consensus_queue.cc:1225] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer 6c22052275034c369b7f7dca086fbd1e log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260924 14:29:32.149397 27129 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.109s	user 0.003s	sys 0.000s
I20260924 14:29:32.149662 27623 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:32.149971 27129 tablet_bootstrap.cc:492] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:32.151022 27129 tablet_bootstrap.cc:492] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:32.115818 27601 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:32.159199 27654 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:32.150375 27623 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:32.161262 27623 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:32.164520 27129 ts_tablet_manager.cc:1397] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.558s	user 0.028s	sys 0.018s
I20260924 14:29:32.166364 27573 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:32.166896 27649 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:32.106652 27182 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:32.172600 27129 raft_consensus.cc:348] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:32.173456 27129 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:32.176856 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:32.177892 26404 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:32.181221 27129 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:32.190534 27129 ts_tablet_manager.cc:1428] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.001s
I20260924 14:29:32.191514 27129 tablet_bootstrap.cc:492] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap starting.
I20260924 14:29:32.229907 27601 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:32.236877 27601 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:32.105599 26221 leader_election.cc:341] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Illegal state: must be running to vote when last-logged opid is not known
W20260924 14:29:32.285804 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 180097us
I20260924 14:29:32.287762 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:32.292716 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:32.297148 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:32.299590 26249 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:32.299140 26079 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:32.304819 26255 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:32.304662 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:32.306275 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:32.307598 26257 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
W20260924 14:29:32.309728 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 134709us
I20260924 14:29:32.318315 26400 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:32.307140 26295 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:32.320732 26404 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:29:32.324442 26397 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:32.326287 27355 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:32.327405 27570 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:32.351914 27353 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:32.352747 27353 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:32.353389 27353 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:32.355088 26079 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 4, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 4 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:32.358171 27560 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:32.360015 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:32.196828 27404 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:32.370136 26401 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:32.373243 26254 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:32.371079 27404 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:32.182003 27182 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:32.378327 27182 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:32.380322 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:32.381321 26298 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:32.383598 27560 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:32.384547 27350 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:32.385402 27350 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:32.387626 26079 catalog_manager.cc:5495] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:32.387732 27350 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:32.341543 26400 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:32.398977 27570 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:32.390359 27547 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:32.399009 27355 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:32.404495 27547 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:32.418722 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 112138us
I20260924 14:29:32.424103 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:32.399149 27502 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:32.425999 27502 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:32.429250 27502 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:32.431699 27346 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:32.431485 26400 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:32.412513 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 652) took 1011 ms. Trace:
I20260924 14:29:32.434434 26400 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:32.434929 26257 rpcz_store.cc:276] 0924 14:29:31.385922 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:31.386876 (+   954us) service_pool.cc:227] Handling call
0924 14:29:32.306309 (+919433us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:32.306915 (+   606us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:32.342900 (+ 35985us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:32.396074 (+ 53174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4181248}
I20260924 14:29:32.436398 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:32.403285 27570 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:32.443790 26298 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:32.465901 26399 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:29:32.436990 27654 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260924 14:29:32.434778 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:32.482713 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:32.482012 27336 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:32.484215 27336 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:32.472101 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:32.491322 27336 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:32.503427 27609 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:32.494938 26079 catalog_manager.cc:5495] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 3, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 3 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:32.506763 27560 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:32.507870 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.527195 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:32.531592 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.533733 27181 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:32.534849 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:32.524931 27654 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260924 14:29:32.545323 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 100868us
W20260924 14:29:32.548118 27681 log.cc:924] Time spent T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260924 14:29:32.626240 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.549283 27547 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:32.563390 26079 catalog_manager.cc:5495] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:32.627768 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:32.625955 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 191429us
I20260924 14:29:32.629137 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.628961 26079 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:32.629892 27654 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:32.630457 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.566351 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.632004 27186 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:32.632993 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.575011 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.623837 27188 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:32.634101 27186 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:32.633082 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.577585 26078 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 2) took 7870 ms. Trace:
I20260924 14:29:32.665057 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:32.665822 27186 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:32.666013 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.630831 26079 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:32.634915 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:32.635387 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.674374 27190 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:32.675580 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:32.675694 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:32.677855 27188 raft_consensus.cc:2396] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260924 14:29:32.677485 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.680531 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.681326 27190 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:32.664431 26078 rpcz_store.cc:276] 0924 14:29:24.707129 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:24.712106 (+  4977us) service_pool.cc:227] Handling call
0924 14:29:32.577288 (+7865182us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:29:28.552692 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:29:29.067281 (+514589us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b5c00058d0c. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf6d7501 00007f09bf6d73f1 00007f09bf6d212c 00007f09bcd71a59 00007f09bcd5d603 00007f09bcd5d2b8 00007f09bcd61ba1 00007f09bcd61b59 00007f09bcd61ae9 00007f09bcd61ab1 00007f09bcd60dad
0924 14:29:29.067361 (+    80us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:29:29.067391 (+    30us) write_op.cc:435] Acquired schema lock
0924 14:29:29.067406 (+    15us) tablet.cc:660] Decoding operations
0924 14:29:29.068035 (+   629us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:29:29.068092 (+    57us) write_op.cc:641] Partition lock acquired for write op
0924 14:29:29.068115 (+    23us) tablet.cc:683] Acquiring locks for 4 operations
0924 14:29:29.068474 (+   359us) tablet.cc:699] Row locks acquired
0924 14:29:29.068491 (+    17us) write_op.cc:260] PREPARE: finished
0924 14:29:29.068649 (+   158us) write_op.cc:270] Start()
0924 14:29:29.068856 (+   207us) write_op.cc:276] Timestamp: P: 1790260169068605 usec, L: 0
0924 14:29:29.068884 (+    28us) op_driver.cc:331] REPLICATION: starting
0924 14:29:29.069525 (+   641us) log.cc:842] Serialized 1717 byte log entry
0924 14:29:29.709721 (+640196us) op_driver.cc:444] REPLICATION: finished
0924 14:29:30.346331 (+636610us) write_op.cc:301] APPLY: starting
0924 14:29:30.359471 (+ 13140us) tablet.cc:1365] starting BulkCheckPresence
0924 14:29:30.407890 (+ 48419us) tablet.cc:1368] finished BulkCheckPresence
0924 14:29:30.416704 (+  8814us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:29:30.494603 (+ 77899us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:29:30.494782 (+   179us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0924 14:29:30.494828 (+    46us) write_op.cc:312] APPLY: finished
0924 14:29:30.495123 (+   295us) log.cc:842] Serialized 40 byte log entry
0924 14:29:30.495299 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:30.495399 (+   100us) write_op.cc:454] Released schema lock
0924 14:29:30.499576 (+  4177us) write_op.cc:341] FINISH: Updating metrics
0924 14:29:30.531983 (+ 32407us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:30.540361 (+  8378us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":23376,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":606116,"apply.run_cpu_time_us":4972,"apply.run_wall_time_us":204692,"num_ops":4,"prepare.queue_time_us":646360,"prepare.run_cpu_time_us":21497,"prepare.run_wall_time_us":801640,"raft.queue_time_us":218361,"raft.run_cpu_time_us":70307,"raft.run_wall_time_us":460772,"replication_time_us":639283,"spinlock_wait_cycles":15217408,"thread_start_us":1345979,"threads_started":4,"wal-append.queue_time_us":374684,"wal-append.run_cpu_time_us":38545,"wal-append.run_wall_time_us":2111961}]]}
I20260924 14:29:32.685783 26079 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:32.667387 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.688887 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:32.690029 27192 raft_consensus.cc:2396] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260924 14:29:32.693606 26329 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:32.695996 26079 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:32.702872 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:32.704296 26257 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:32.658700 27182 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260924 14:29:32.523383 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 226543us
I20260924 14:29:32.678407 27346 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:32.678879 27601 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:32.691130 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.710731 27601 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:32.710829 27346 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:32.682319 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.631783 27565 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:32.698246 26079 catalog_manager.cc:5495] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:32.711956 27182 pending_rounds.cc:76] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 1
I20260924 14:29:32.711694 27547 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:32.743221 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:32.738571 26079 catalog_manager.cc:5495] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 2 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:32.717360 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.718461 27565 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:32.719743 27189 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:32.739305 27182 pending_rounds.cc:97] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:32.775000 26400 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260924 14:29:32.775490 27190 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:32.775810 26400 pending_rounds.cc:76] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 1
I20260924 14:29:32.765329 27547 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:32.776235 26400 pending_rounds.cc:97] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260924 14:29:32.776712 26400 raft_consensus.cc:2866] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:32.777912 27547 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:32.777980 27523 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260924 14:29:32.778365 27186 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:32.769107 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:32.784473 27438 consensus_meta.cc:287] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.705s	user 0.016s	sys 0.004s
I20260924 14:29:32.787855 27189 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260924 14:29:32.787249 26326 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:32.789563 27189 pending_rounds.cc:76] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 1
I20260924 14:29:32.790042 27189 pending_rounds.cc:97] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:32.790484 27189 raft_consensus.cc:2866] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:32.767885 27346 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:32.791311 26329 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260924 14:29:32.793356 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.794453 27192 raft_consensus.cc:2354] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4.
I20260924 14:29:32.796149 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:32.797497 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:32.799825 27601 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:32.782161 27182 raft_consensus.cc:2866] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:29:32.810933 27483 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260924 14:29:32.822180 27601 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:32.819177 27186 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:32.828117 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:32.775333 27565 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:32.820396 27355 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:32.820227 27346 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
W20260924 14:29:32.843933 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 62335.4us
I20260924 14:29:32.858074 27336 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:32.860021 27438 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:32.860293 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:32.860716 27438 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:32.861457 27547 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:32.863770 27194 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.009s
I20260924 14:29:32.863991 27438 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:32.864605 27194 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260924 14:29:32.865067 27194 pending_rounds.cc:76] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 1
I20260924 14:29:32.865360 27194 pending_rounds.cc:97] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260924 14:29:32.865684 27194 raft_consensus.cc:2866] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:32.866591 27336 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:32.868146 27601 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:32.868005 27336 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:32.842864 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:32.872301 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:32.873433 26331 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:32.874545 27649 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:32.868929 27355 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:32.877977 27355 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:32.879281 27626 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:32.879887 27626 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:32.880343 27626 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:32.874938 27181 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:32.883571 27182 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 3
I20260924 14:29:32.902987 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:32.905164 27601 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:32.906694 27438 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:32.907688 27438 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:32.911748 27547 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:32.912919 27547 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:32.914129 27547 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:32.919667 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:32.920743 27192 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:32.923039 27654 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:32.926362 27565 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:32.950654 27353 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.596s	user 0.009s	sys 0.007s
I20260924 14:29:33.011595 27123 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.015704 27547 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:33.022082 27129 tablet_bootstrap.cc:492] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 14:29:33.023404 27129 tablet_bootstrap.cc:492] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Bootstrap complete.
I20260924 14:29:33.024345 27129 ts_tablet_manager.cc:1397] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Time spent bootstrapping tablet: real 0.833s	user 0.026s	sys 0.015s
I20260924 14:29:33.025485 27570 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:33.026315 27570 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.029042 27570 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:32.974265 27654 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.037079 27654 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.038493 27654 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:33.038753 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 151300us
I20260924 14:29:33.039961 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.041199 27483 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:33.042451 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.044395 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.054692 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:33.060312 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.061447 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:33.074865 26399 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:33.076239 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.003525 27353 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.063732 27346 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:33.006737 27626 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.064926 27649 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 14:29:33.083978 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 227881us
I20260924 14:29:33.092962 27346 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:33.095705 27438 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.103700 27346 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:29:33.091612 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 49233.2us
I20260924 14:29:33.119596 27438 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.123395 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.124214 27626 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.124248 26402 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
W20260924 14:29:33.129206 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 295882us
I20260924 14:29:33.108263 27346 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:33.137692 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 218408us
I20260924 14:29:33.109396 27353 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.123348 27438 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.148270 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.123361 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.165627 26295 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.169047 27372 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:33.169586 27372 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:33.169973 27372 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.171598 26325 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:33.136819 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:33.180362 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53798.7us
I20260924 14:29:33.181289 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.182127 27565 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:33.182641 27649 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260924 14:29:33.183308 27695 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260924 14:29:33.182636 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.184513 27649 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:33.184568 27372 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.185947 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:33.186750 27372 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.186771 27649 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:33.187531 27649 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:33.188050 27649 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:33.196954 27523 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:33.199187 26401 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:33.192598 27353 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:29:33.217229 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 141091us
W20260924 14:29:33.222887 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 74807us
I20260924 14:29:33.224301 27353 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.233023 27495 consensus_queue.cc:1046] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260924 14:29:33.234929 27353 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.220389 27404 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.227494 27626 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.255539 27350 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.245501 27626 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.256675 27350 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.258147 27438 raft_consensus.cc:3032] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:33.258811 27438 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:33.246147 27129 raft_consensus.cc:348] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.262233 27129 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Initialized, Role: FOLLOWER
I20260924 14:29:33.259374 27626 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:33.273643 27438 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.275221 27438 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:33.277804 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.278954 27715 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:33.280036 27715 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.280752 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:33.281589 27188 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 2: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:33.282459 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.282891 27715 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.279037 27194 raft_consensus.cc:2396] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260924 14:29:33.285832 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.286238 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:33.288144 27699 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260924 14:29:33.288446 27404 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.291574 27711 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:33.292198 27711 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.302857 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.306767 27181 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260924 14:29:33.310331 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 94852.7us
I20260924 14:29:33.270989 27129 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.270149 27523 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:33.330123 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 86433.6us
I20260924 14:29:33.317126 27570 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.336719 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:33.333015 27350 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.316636 27123 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.353648 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.357149 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:33.342386 27570 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.340721 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.361570 27129 ts_tablet_manager.cc:1428] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Time spent starting tablet: real 0.306s	user 0.010s	sys 0.000s
I20260924 14:29:33.349563 27573 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.349601 27495 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:33.352843 27353 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.360425 27565 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:33.365542 27350 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:33.360306 27718 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:33.384020 27573 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:33.386003 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 55081us
I20260924 14:29:33.387665 27720 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:33.375031 27711 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.337723 27523 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:33.361002 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 59573.9us
I20260924 14:29:33.360141 27404 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.393736 27560 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.397816 27718 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:33.397123 27350 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:33.381986 27495 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:33.415617 27698 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260924 14:29:33.418996 27523 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.405078 27720 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:33.395967 27353 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.423830 27350 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.424440 27720 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.421157 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.428714 27560 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.430001 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.426482 27570 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.431010 27560 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:33.435617 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 71859.6us
I20260924 14:29:33.436493 26298 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:33.435060 27654 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:33.431336 27353 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.441725 27404 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:33.415056 27719 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260924 14:29:33.457484 26401 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:33.458717 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.452921 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.464170 26403 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
W20260924 14:29:33.464377 27721 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260924 14:29:33.465798 27126 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:33.453006 27565 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:33.468062 27722 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260924 14:29:33.466897 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.475593 27718 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.468089 27717 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.470182 27404 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:33.466861 27707 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.466678 27560 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:33.455137 27495 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.466435 27353 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:33.483935 27717 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.478161 27346 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.490368 27346 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.481534 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:33.472539 27438 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:33.490796 27350 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:33.487308 27560 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.534812 27560 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:33.535586 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.536347 26401 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:33.535907 27404 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:33.539366 27714 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260924 14:29:33.540223 27730 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260924 14:29:33.541101 27725 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260924 14:29:33.541213 27601 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.541805 27601 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.543359 27573 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.545312 27350 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:33.545833 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 64435.3us
W20260924 14:29:33.528656 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 162531us
I20260924 14:29:33.527863 27353 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.533831 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:33.513183 27570 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:33.490384 27717 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.487881 27502 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.550848 27438 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.481545 27707 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.575268 26249 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:33.563758 26397 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.518385 27718 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.552532 27495 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:33.492493 27565 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.614044 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.614765 27185 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
W20260924 14:29:33.611126 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 81994.2us
I20260924 14:29:33.614914 27696 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.615684 27696 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.610874 27720 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:33.616890 27720 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.617497 26295 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.617818 27696 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.618104 27182 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:33.621731 27735 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:33.622143 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.622911 27188 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:33.622196 27735 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:33.624346 26295 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.624678 27735 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:33.624402 27654 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:33.638768 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 298084us
I20260924 14:29:33.640198 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.640583 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.640937 26401 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:33.640108 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:33.565042 27502 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:33.643354 26402 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.624706 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:33.625434 27591 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:33.637836 27654 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.647078 27565 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.647437 27729 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:33.624251 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 89) took 1082 ms. Trace:
I20260924 14:29:33.653007 26369 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:33.656790 26298 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.657653 27346 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:33.658048 27346 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.658401 27346 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:33.626688 27495 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.659775 27353 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.661074 27495 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.661764 27718 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.661088 27707 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.662382 27729 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.662849 27729 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:33.662949 27654 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.651639 27483 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.652063 27591 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.663926 27591 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:33.663800 27483 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.665938 27483 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.666673 27346 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.667604 27732 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.670081 27707 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.670662 27707 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.611775 27649 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.671919 27720 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.672232 27346 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.672986 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.673055 27649 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.671981 27194 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.676229 27502 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:33.673190 27727 log.cc:924] Time spent T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Append to log took a long time: real 0.087s	user 0.000s	sys 0.003s
I20260924 14:29:33.628383 27601 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.655602 27186 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.686079 27523 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:33.642617 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 91181.8us
I20260924 14:29:33.698661 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.699633 26406 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:33.704666 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:33.705551 26404 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:33.707835 27707 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.708175 26298 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.709069 27707 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:33.709467 27707 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.709849 27707 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.710132 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.710847 26251 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:33.713210 26253 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:33.713809 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.712975 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.714658 26257 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:33.714825 26400 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:33.717065 26253 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:33.720450 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.722316 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.723012 26258 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:33.696060 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.723680 27192 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:33.724840 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.726013 27717 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:33.726110 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.726605 27717 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.679594 27190 rpcz_store.cc:276] 0924 14:29:32.541416 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:32.777455 (+236039us) service_pool.cc:227] Handling call
0924 14:29:32.799211 (+ 21756us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:32.886200 (+ 86989us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:32.950281 (+ 64081us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:32.974683 (+ 24402us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:33.043969 (+ 69286us) log.cc:842] Serialized 26 byte log entry
0924 14:29:33.216668 (+172699us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:33.220626 (+  3958us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:33.220782 (+   156us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:33.542100 (+321318us) raft_consensus.cc:1671] finished
0924 14:29:33.542129 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:33.624038 (+ 81909us) spinlock_profiling.cc:243] Waited 84.6 ms on lock 0x7b54000028d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:33.624175 (+   137us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":20878,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":22914,"spinlock_wait_cycles":202118656,"thread_start_us":165358,"threads_started":1,"wal-append.queue_time_us":328801}
I20260924 14:29:33.727146 27717 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:33.728102 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.729159 26400 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 1.
I20260924 14:29:33.731096 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:33.732674 27738 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:33.733088 27738 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.733426 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.733945 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.735819 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.735994 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.736609 26406 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:33.733435 27738 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:33.740180 27126 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:33.741320 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:33.741952 26249 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.735374 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:33.743723 26257 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.743989 27736 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:33.744585 27736 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:33.745069 27736 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.748682 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.749922 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.754925 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.757367 26253 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:33.759874 26402 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:33.760119 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:33.760833 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 115594us
I20260924 14:29:33.761041 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.764753 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:33.765988 26401 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:33.767410 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.768102 26258 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:33.668184 27732 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.771291 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:33.765126 27696 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:33.755604 27194 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:33.779585 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.780614 27726 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:33.781188 27726 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:33.782014 27726 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:33.783728 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:33.784503 26331 raft_consensus.cc:2354] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4.
I20260924 14:29:33.789337 26404 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:33.776428 27438 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.792035 26298 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.793026 27716 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:33.793701 27716 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:33.793962 27186 raft_consensus.cc:2396] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260924 14:29:33.794641 27716 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.779186 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:33.832803 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 58913.7us
I20260924 14:29:33.795786 27560 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.797853 27732 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.836257 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:33.837494 27192 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:33.839990 27717 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.841629 27733 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.841346 27729 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.842125 27733 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.843147 27717 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.800328 27743 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:33.855599 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.858395 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.859170 26325 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:33.779340 27696 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:33.859787 27696 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.854275 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.860719 26400 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:33.861951 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:33.862870 27483 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:33.863281 27483 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:33.863662 27483 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:33.864593 26323 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:33.872249 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.873219 27696 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.874679 26324 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:33.875819 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.876657 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:33.876668 27696 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.810974 26295 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:33.877991 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 67106.5us
I20260924 14:29:33.878096 27696 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:33.878913 27696 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:33.879520 27743 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:33.812367 26397 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:33.814201 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.880198 27696 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.815670 27601 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:33.882998 27601 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:33.883339 27601 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:33.884331 27601 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.885923 27601 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:33.818883 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:33.819343 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:33.889353 27346 consensus_queue.cc:1046] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:33.890697 27731 raft_consensus.cc:3032] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:33.891045 27731 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:33.824847 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.892458 27190 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:33.892014 27731 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.893397 27731 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:33.893141 27736 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.801466 27738 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.828927 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:33.830482 27735 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:33.831632 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 60441.2us
I20260924 14:29:33.841391 27502 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:33.843936 27729 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.844839 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.905498 27654 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.845296 27649 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.845655 27560 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.848119 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:33.848443 26249 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:33.911017 27735 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.911221 26403 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:33.847939 27591 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.799748 26257 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:33.912498 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.849982 27707 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.914161 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:33.851318 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:33.915622 27743 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:33.916293 27743 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:33.916972 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:33.917685 26397 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:33.918507 27591 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.919512 26329 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:33.920043 27346 consensus_queue.cc:1046] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:33.921160 27346 raft_consensus.cc:3032] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:33.921518 27346 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:33.852007 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.922298 26255 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:33.922339 27346 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.854733 27733 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.876886 26323 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:33.924597 27707 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:33.898983 27736 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.921180 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:33.926069 27591 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:33.808182 27740 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.898144 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:33.927811 26258 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.927708 27740 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.929109 27123 leader_election.cc:398] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4.
I20260924 14:29:33.929955 27740 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.929701 27123 leader_election.cc:401] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260924 14:29:33.827240 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.906095 27654 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.906245 27649 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:33.931743 26221 connection.cc:232] RPC call timeout handler was delayed by -1.44694663700011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1723s for a response
I20260924 14:29:33.935727 27181 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:33.945453 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:33.948321 26258 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:33.944208 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.906868 27502 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.943001 26078 catalog_manager.cc:5495] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:33.943396 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:33.952904 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.950968 26404 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:33.930661 27591 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260924 14:29:33.957505 26078 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:33.959426 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:33.942905 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:33.944463 27720 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.962148 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.945868 27649 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.944695 27654 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.964506 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:33.951184 27502 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:33.943351 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.967123 26402 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.968015 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:33.951552 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:33.968683 26404 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.959653 26078 catalog_manager.cc:5495] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 3, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:33.969849 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.970592 26325 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:33.961211 26400 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:33.971875 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:33.972262 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.972730 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.972870 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.973202 27182 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:33.971894 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.974097 27732 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:33.976519 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.964920 27126 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:33.977702 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.979604 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.981923 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.963207 27372 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:33.982678 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.983690 27654 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:33.984112 27654 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:33.984489 27654 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:33.985399 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:33.986220 27649 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:33.986799 27649 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:33.963423 27719 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:33.958884 27736 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:33.987731 27649 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:33.988376 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:33.988462 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:33.989176 27191 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 5
I20260924 14:29:33.989295 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:33.987821 27736 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:33.992204 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50407.3us
I20260924 14:29:33.993058 27736 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:33.964814 27346 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:33.995096 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:33.995867 26401 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:33.967157 27720 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:33.995162 27372 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:33.996799 26255 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:33.997889 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:33.997888 27720 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:33.997947 27372 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:33.999874 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.968719 26406 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:34.001720 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:34.002442 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:33.991119 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.003917 27126 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:34.004753 27560 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:34.005421 27560 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:34.005867 27560 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.005261 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:33.970326 26078 catalog_manager.cc:5495] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:33.962046 27740 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4.
W20260924 14:29:34.015838 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 70478.9us
I20260924 14:29:34.019024 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:34.019315 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:34.032582 27125 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:34.032877 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.033710 27565 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:34.034107 27565 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:34.034183 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:34.034580 27565 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.034797 26397 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:34.035516 27570 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:34.035889 27570 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:34.036150 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:34.037173 27730 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:34.037618 27730 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:34.038192 26401 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:34.038736 27732 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:34.038465 27730 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:34.039112 27732 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:34.039512 27732 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:34.041031 27743 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:34.029008 27560 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:34.031373 27716 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:34.044521 27716 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:34.044862 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.045819 27191 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:34.046563 27716 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:34.047067 27738 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:34.047993 26332 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:34.048172 27729 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:34.036202 27570 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:34.041098 26404 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:34.001864 26402 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:34.053303 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:34.054598 26298 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:34.015064 26079 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 136) took 4613 ms. Trace:
I20260924 14:29:34.062457 26295 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:34.063977 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 90417.5us
I20260924 14:29:34.067108 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:34.069463 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 46922.7us
I20260924 14:29:34.061983 26079 rpcz_store.cc:276] 0924 14:29:29.401698 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:29.411694 (+  9996us) service_pool.cc:227] Handling call
0924 14:29:34.014996 (+4603302us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:29:33.238368 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:29:33.238982 (+   614us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:29:33.239310 (+   328us) write_op.cc:435] Acquired schema lock
0924 14:29:33.239340 (+    30us) tablet.cc:660] Decoding operations
0924 14:29:33.335975 (+ 96635us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:29:33.349108 (+ 13133us) write_op.cc:641] Partition lock acquired for write op
0924 14:29:33.350564 (+  1456us) tablet.cc:683] Acquiring locks for 11 operations
0924 14:29:33.384175 (+ 33611us) tablet.cc:699] Row locks acquired
0924 14:29:33.384229 (+    54us) write_op.cc:260] PREPARE: finished
0924 14:29:33.384458 (+   229us) write_op.cc:270] Start()
0924 14:29:33.384655 (+   197us) write_op.cc:276] Timestamp: P: 1790260173384376 usec, L: 0
0924 14:29:33.384704 (+    49us) op_driver.cc:331] REPLICATION: starting
0924 14:29:33.385679 (+   975us) log.cc:842] Serialized 4454 byte log entry
0924 14:29:33.811908 (+426229us) op_driver.cc:444] REPLICATION: finished
0924 14:29:33.866870 (+ 54962us) write_op.cc:301] APPLY: starting
0924 14:29:33.866985 (+   115us) tablet.cc:1365] starting BulkCheckPresence
0924 14:29:33.867276 (+   291us) tablet.cc:1368] finished BulkCheckPresence
0924 14:29:33.867421 (+   145us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:29:33.870994 (+  3573us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:29:33.871546 (+   552us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0924 14:29:33.871632 (+    86us) write_op.cc:312] APPLY: finished
0924 14:29:33.872057 (+   425us) log.cc:842] Serialized 82 byte log entry
0924 14:29:33.873166 (+  1109us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:33.873310 (+   144us) write_op.cc:454] Released schema lock
0924 14:29:33.873381 (+    71us) write_op.cc:341] FINISH: Updating metrics
0924 14:29:33.931748 (+ 58367us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:33.934230 (+  2482us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":555278,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":54600,"apply.run_cpu_time_us":4444,"apply.run_wall_time_us":83972,"num_ops":11,"prepare.queue_time_us":378675,"prepare.run_cpu_time_us":23279,"prepare.run_wall_time_us":206897,"raft.queue_time_us":133592,"raft.run_cpu_time_us":3414,"raft.run_wall_time_us":51203,"replication_time_us":427078,"thread_start_us":406928,"threads_started":4,"wal-append.queue_time_us":87835}]]}
I20260924 14:29:34.077314 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.086120 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:34.072206 27717 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:34.072060 27404 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:34.091993 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:34.071307 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:34.098694 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.099972 27649 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:34.099587 27502 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.101032 26398 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:34.102108 27502 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:34.074656 27483 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.102977 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:34.104413 26406 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:34.106050 27483 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:34.106592 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:34.107249 26402 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 6
I20260924 14:29:34.103654 27182 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:34.086244 27730 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:34.117241 26253 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:34.118629 27711 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260924 14:29:34.114621 27523 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 14:29:34.119427 27756 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260924 14:29:34.120095 27711 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:34.120493 27711 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:34.121407 27711 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:34.110284 27717 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:34.118647 27438 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260924 14:29:34.123673 27717 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:34.124643 27438 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:34.125056 27438 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:34.125972 27438 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:34.110064 27719 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260924 14:29:34.112892 27404 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:34.130489 27186 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:34.132014 27181 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:34.122627 27711 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:34.106848 26400 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:34.140208 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:34.147261 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.156231 26258 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:34.147992 26402 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:34.153353 26331 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:34.162151 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:34.167196 27483 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:34.167773 27483 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.169782 27483 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:34.173485 27609 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.174255 27609 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:34.188293 27609 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:34.190281 27350 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.190950 27350 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.193410 27719 consensus_queue.cc:1225] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260924 14:29:34.194429 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.196192 27601 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.196976 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.196764 27601 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.198107 27591 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.198931 27601 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:34.198661 27591 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.200035 27404 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:34.201447 27591 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:34.203596 27438 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:34.205114 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.203617 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.207285 27350 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:34.210371 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:34.212662 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.152187 27730 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:34.158077 27736 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:34.230958 26258 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:34.313719 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.218983 27736 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:34.221052 27495 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:34.231566 27483 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:34.247951 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:34.245894 26403 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:34.216619 26323 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:34.237224 26398 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:34.330368 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.332427 26328 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:34.324791 27736 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.341313 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:34.230728 27565 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.199268 27732 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.335952 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:34.336277 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.350509 26256 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.336994 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:34.337309 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:34.337980 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.339077 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:34.340283 27372 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.340730 27731 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.341277 27580 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.342020 27719 consensus_queue.cc:1046] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:29:34.343901 26257 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:34.228900 27350 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260924 14:29:34.344856 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.345292 26406 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:34.325204 27495 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:34.320461 27743 consensus_queue.cc:1046] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260924 14:29:34.355794 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.317083 27654 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.357522 26324 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:34.365893 27372 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:34.368472 27495 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:34.369256 27740 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:34.357686 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.333179 27735 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.344405 27483 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:34.380643 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.216606 27730 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:34.342855 27502 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.364516 26401 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:29:34.365309 26249 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:34.373389 27743 consensus_queue.cc:1225] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer 6c22052275034c369b7f7dca086fbd1e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260924 14:29:34.374374 27654 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:34.374895 27735 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.378504 26250 raft_consensus.cc:2396] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260924 14:29:34.386046 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 311977us
I20260924 14:29:34.402443 27763 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:34.369818 27495 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:34.369776 27740 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:34.421607 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.426373 27732 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:34.437217 26258 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:34.440557 27565 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:34.442283 26403 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:34.389417 27502 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.443763 27483 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.445331 26256 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:34.445953 27502 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:34.447033 27372 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:34.447929 26332 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:34.366074 27580 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:34.454435 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:34.457908 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52739.8us
W20260924 14:29:34.394917 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 58031us
I20260924 14:29:34.365530 27731 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:34.448125 27743 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:34.466367 26324 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:34.475960 27182 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:34.455631 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:34.474169 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.466177 27350 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:34.477346 27186 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:34.475632 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.477644 27350 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:34.478085 27350 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:34.476636 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:34.478971 26403 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:34.478919 27350 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.480271 27350 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:34.474933 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:34.481997 26401 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.483651 27502 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.493968 27502 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.496016 27502 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:34.499029 27735 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:34.499879 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:34.500603 26324 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:34.501796 26403 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:34.503772 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:34.504034 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:34.505169 26401 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:34.475541 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:34.519616 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.511946 27502 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:34.522517 27502 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.522924 27502 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.524638 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:34.525717 27194 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:34.527060 27731 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:34.512449 27729 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:34.529163 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:34.529525 27729 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.529953 27729 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:34.469486 27126 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:34.532655 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 63281.8us
I20260924 14:29:34.533694 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:34.534802 27654 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:34.535403 27654 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:34.536592 27654 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.484409 27728 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260924 14:29:34.481508 27570 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:34.550850 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.560407 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:34.585799 26332 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:34.562183 26253 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260924 14:29:34.549894 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 205079us
I20260924 14:29:34.542706 26369 leader_election.cc:398] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4.
I20260924 14:29:34.608659 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.609117 26369 leader_election.cc:401] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:29:34.574209 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.555263 26295 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:34.611020 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:29:34.611618 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 56471us
I20260924 14:29:34.613483 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.622991 26256 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:34.540995 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.557767 27185 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260924 14:29:34.551335 27718 raft_consensus.cc:3032] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:34.540181 27649 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:34.543583 27182 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:34.628952 27743 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:34.546602 27570 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:34.542883 27580 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:34.544428 27732 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
I20260924 14:29:34.545512 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.548427 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:34.546756 27191 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:34.612797 27495 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:34.633595 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.626737 27720 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:34.630489 27185 pending_rounds.cc:76] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 1
I20260924 14:29:34.616158 27716 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:34.623694 27719 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:34.632057 27718 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:34.539986 27126 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.539250 27502 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.641696 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.636312 27735 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:34.644351 27185 pending_rounds.cc:97] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:34.651712 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:34.541221 27740 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.642284 27765 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:34.642805 27570 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260924 14:29:34.641129 27649 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:34.668366 27735 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.658509 27495 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260924 14:29:34.670488 27495 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:34.670369 27649 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.644587 27189 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:34.671257 27735 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:34.632687 27438 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260924 14:29:34.673758 27438 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:34.674118 27438 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:34.656286 26221 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:34.659584 27181 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 5: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:34.662482 27483 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:34.675364 27763 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:34.675817 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:34.676492 26323 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.658500 27185 raft_consensus.cc:2866] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:34.644665 27716 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.680934 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:34.681523 27716 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:34.681831 27185 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:34.682659 27711 raft_consensus.cc:3032] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:34.683103 27711 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:34.683956 27711 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.685148 27711 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.658579 27720 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:34.687549 27483 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:34.665750 27736 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:34.688194 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 145628us
I20260924 14:29:34.689018 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.689534 27736 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:34.666412 27502 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:34.670910 27495 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:34.648329 27719 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:34.690418 27719 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.690832 27720 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:34.690913 27495 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.692314 27495 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:34.693373 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:34.694416 27717 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:34.698132 26078 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55200 (request call id 125) took 2001 ms. Trace:
I20260924 14:29:34.655597 27125 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:34.698930 27502 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election won for term 4
W20260924 14:29:34.699035 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 53460.2us
I20260924 14:29:34.699434 27502 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:34.700654 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:34.700421 27502 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:34.701151 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:34.701937 27717 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:34.702322 27717 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:34.702687 27717 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.675954 27763 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:34.662990 27718 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.697939 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.710089 26078 rpcz_store.cc:276] 0924 14:29:32.696445 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:32.699084 (+  2639us) service_pool.cc:227] Handling call
0924 14:29:34.698043 (+1998959us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:29:34.097279 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:29:34.221781 (+124502us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:29:34.221832 (+    51us) write_op.cc:435] Acquired schema lock
0924 14:29:34.221856 (+    24us) tablet.cc:660] Decoding operations
0924 14:29:34.222679 (+   823us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:29:34.222780 (+   101us) write_op.cc:641] Partition lock acquired for write op
0924 14:29:34.222818 (+    38us) tablet.cc:683] Acquiring locks for 4 operations
0924 14:29:34.223313 (+   495us) tablet.cc:699] Row locks acquired
0924 14:29:34.223338 (+    25us) write_op.cc:260] PREPARE: finished
0924 14:29:34.223547 (+   209us) write_op.cc:270] Start()
0924 14:29:34.223694 (+   147us) write_op.cc:276] Timestamp: P: 1790260174223495 usec, L: 0
0924 14:29:34.223721 (+    27us) op_driver.cc:331] REPLICATION: starting
0924 14:29:34.224518 (+   797us) log.cc:842] Serialized 1717 byte log entry
0924 14:29:34.313824 (+ 89306us) op_driver.cc:444] REPLICATION: finished
0924 14:29:34.329990 (+ 16166us) write_op.cc:301] APPLY: starting
0924 14:29:34.333616 (+  3626us) tablet.cc:1365] starting BulkCheckPresence
0924 14:29:34.350070 (+ 16454us) tablet.cc:1368] finished BulkCheckPresence
0924 14:29:34.361016 (+ 10946us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:29:34.375260 (+ 14244us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:29:34.375462 (+   202us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0924 14:29:34.375513 (+    51us) write_op.cc:312] APPLY: finished
0924 14:29:34.375734 (+   221us) log.cc:842] Serialized 40 byte log entry
0924 14:29:34.375909 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:34.376008 (+    99us) write_op.cc:454] Released schema lock
0924 14:29:34.376057 (+    49us) write_op.cc:341] FINISH: Updating metrics
0924 14:29:34.407789 (+ 31732us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:34.413519 (+  5730us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":39262,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":4655,"apply.run_cpu_time_us":3200,"apply.run_wall_time_us":96031,"num_ops":4,"prepare.queue_time_us":15116,"prepare.run_cpu_time_us":4192,"prepare.run_wall_time_us":161338,"replication_time_us":85000,"thread_start_us":15577,"threads_started":1}]]}
I20260924 14:29:34.720307 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:34.716084 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:34.733004 27372 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:34.733507 27372 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.733913 27372 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:34.737943 27181 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:34.740201 26080 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 137) took 8247 ms. Trace:
I20260924 14:29:34.674903 27438 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.740578 27483 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:34.740721 27731 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:34.741186 27438 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:34.741271 27731 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:34.741724 27731 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:34.708949 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 169081us
I20260924 14:29:34.730602 27523 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260924 14:29:34.744232 27126 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:34.745456 27126 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:34.747226 26258 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:34.749010 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:34.752029 27570 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:34.752689 27570 raft_consensus.cc:686] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:34.752844 27565 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:34.753037 27764 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:34.753440 27764 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:34.754472 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:34.740625 26080 rpcz_store.cc:276] 0924 14:29:26.509394 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:26.516402 (+  7008us) service_pool.cc:227] Handling call
0924 14:29:27.433607 (+917205us) spinlock_profiling.cc:243] Waited 99.8 ms on lock 0x7b540015fed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3af0 00007f09b9356f2a 00007f09bf2a8d6c 00007f09bf3a2c3e 00007f09bd6b4fb4 00007f09bd6b4f42 00007f09bd6b4e71 00007f09bd6b4dec 00007f09bd6b40a2 00007f09b777894c 00007f09b7777d86
0924 14:29:34.740029 (+7306422us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":145253,"reported_tablets":7,"spinlock_wait_cycles":229639936}
W20260924 14:29:34.761075 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45079.7us
I20260924 14:29:34.728430 27732 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:34.768260 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:34.654799 27730 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.101s
I20260924 14:29:34.768733 26332 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:34.708607 27763 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.781423 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:34.758453 27743 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4.
I20260924 14:29:34.758018 27764 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.798089 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:34.799139 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 69306us
I20260924 14:29:34.802480 26323 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
W20260924 14:29:34.758201 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57147.9us
W20260924 14:29:34.804032 27770 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260924 14:29:34.731091 27760 consensus_queue.cc:1046] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260924 14:29:34.806740 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:34.752882 27495 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260924 14:29:34.821223 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:34.823113 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:34.825043 26401 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:34.805207 27696 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:34.804032 26398 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260924 14:29:34.824167 27649 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:34.828753 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.792209 27720 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:34.796096 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.763614 27719 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.842537 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:34.753649 27570 consensus_queue.cc:237] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:34.792277 27735 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260924 14:29:34.817257 27743 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:34.848971 26404 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:34.846735 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:34.850447 26399 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:34.790908 27729 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.781451 27765 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260924 14:29:34.856915 26406 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260924 14:29:34.850543 27649 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:34.862812 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:34.863732 26402 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:34.853417 27696 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:34.876753 26406 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 6. Ops: []
I20260924 14:29:34.878614 26399 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:34.895757 26404 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:34.898802 26402 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:34.905697 26404 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260924 14:29:34.908949 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 127646us
I20260924 14:29:34.870004 27780 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:34.917990 26257 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260924 14:29:34.920742 26257 pending_rounds.cc:76] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 2
I20260924 14:29:34.921058 26257 pending_rounds.cc:97] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260924 14:29:34.921370 26257 raft_consensus.cc:2866] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:34.911113 26255 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:34.897002 27565 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:34.928984 26254 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:34.929337 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.928987 26403 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:34.930037 26399 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:34.930234 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.931018 26401 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:34.931797 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:34.932016 26404 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:34.950599 26404 raft_consensus.cc:1226] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:34.932694 26402 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:34.935874 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:34.937587 27773 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260924 14:29:34.944259 26251 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:34.944299 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.959648 26406 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:34.906306 26400 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:34.962917 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.964516 26406 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:34.895282 27716 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.972693 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.983953 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.985069 26402 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:34.987987 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.986393 27372 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:34.994541 27495 consensus_queue.cc:1225] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260924 14:29:34.938539 27696 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:35.004925 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.009259 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.000519 27716 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:35.010978 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 89306.3us
I20260924 14:29:35.011410 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:34.984678 27720 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:29:35.013896 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 156819us
I20260924 14:29:34.995467 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:34.911942 27649 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:35.044440 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:35.006608 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.048036 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:35.011253 27495 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:35.054190 27765 consensus_queue.cc:1225] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260924 14:29:35.006451 27719 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:35.056474 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:35.017421 27729 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:35.057202 27783 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
I20260924 14:29:35.054890 27495 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:35.061969 27711 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:35.062984 27720 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:35.065893 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.066952 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.076905 26402 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:35.068341 27372 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:35.043761 27717 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:35.068056 27731 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:35.010939 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46776.9us
I20260924 14:29:35.089015 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:35.089666 27775 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260924 14:29:35.043026 27716 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260924 14:29:35.077849 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.093554 26402 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:35.088263 26405 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:35.070562 26406 raft_consensus.cc:2362] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4. Re-sending same reply.
I20260924 14:29:35.072113 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:35.099531 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.101420 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:35.102778 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:35.103107 27495 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:35.103536 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:35.103838 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:35.098508 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:35.086681 27719 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:35.074376 27780 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:35.073565 26397 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
W20260924 14:29:35.119355 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 357177us
I20260924 14:29:35.120503 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:35.121362 27731 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:35.110212 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61312.3us
I20260924 14:29:35.123311 27483 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:35.124639 27483 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:35.125605 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:35.110908 26332 raft_consensus.cc:2396] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260924 14:29:35.132913 27781 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:35.133039 27580 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:35.133430 27781 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:35.133514 27346 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:35.134320 27781 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:35.136065 27346 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:35.137111 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.137938 26404 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:35.114382 26406 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:35.098415 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:35.096017 27733 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:35.139377 27181 raft_consensus.cc:2354] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4.
I20260924 14:29:35.139344 27733 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:35.140108 26250 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:35.140479 26250 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:35.140370 27733 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:35.141134 27781 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:35.141300 26250 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:35.142491 26250 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:35.146384 27483 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:35.147881 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:35.148561 26332 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:35.151574 27346 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:35.152590 27346 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:35.154186 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:35.154317 27346 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:35.156476 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:35.158814 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:35.161868 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:35.166961 27483 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:35.167917 27483 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:35.168517 27580 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:35.168913 27728 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:35.168557 27483 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:35.169571 27728 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:35.171949 27728 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:35.177891 26332 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:35.181036 27502 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:35.182402 27502 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:35.182847 27502 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:35.183807 27502 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:35.185096 27502 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:35.188501 27792 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:35.189123 27792 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:35.103828 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:35.192732 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 67704.3us
I20260924 14:29:35.200516 26258 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:35.202068 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:35.111768 27719 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:35.223165 26250 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:35.225308 27194 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:35.228125 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:35.075110 27565 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:35.225132 27523 consensus_queue.cc:1046] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:35.103158 27782 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
I20260924 14:29:35.241835 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:35.105211 27778 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.109s
I20260924 14:29:35.206002 27792 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:35.256801 27404 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260924 14:29:35.313426 27780 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:35.331964 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 79272.1us
W20260924 14:29:35.332969 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 72182.8us
I20260924 14:29:35.333312 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:35.206984 27763 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260924 14:29:35.367468 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:35.375001 26251 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:35.252213 27729 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260924 14:29:35.326089 27736 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:35.370863 26224 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:35.332252 27718 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:35.228364 27787 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260924 14:29:35.333492 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.365279 26397 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260924 14:29:35.377952 27769 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260924 14:29:35.413651 26256 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:35.414152 26256 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:29:35.416435 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45697.1us
I20260924 14:29:35.417526 26224 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:35.340240 27372 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260924 14:29:35.436982 26224 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:35.438794 26224 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:35.422427 26256 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:35.414335 27765 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:35.415062 27726 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:35.416177 27717 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:35.407132 26399 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:54910
I20260924 14:29:35.468695 26399 raft_consensus.cc:595] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:35.448731 27783 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:35.462735 27495 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:35.375489 27791 consensus_queue.cc:1046] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260924 14:29:35.408210 27736 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:35.463105 27438 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.721s	user 0.013s	sys 0.001s
W20260924 14:29:35.498907 27803 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260924 14:29:35.499567 27802 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260924 14:29:35.547996 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 109324us
I20260924 14:29:35.551059 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:35.553076 26224 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:35.560421 27796 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
W20260924 14:29:35.561055 27798 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
W20260924 14:29:35.562512 27806 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260924 14:29:35.554785 27791 consensus_queue.cc:1046] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260924 14:29:35.549400 27793 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:35.563510 27188 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: []
W20260924 14:29:35.571033 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 185356us
I20260924 14:29:35.559623 27789 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:35.573756 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:35.601392 27736 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:35.569903 27483 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:35.602130 27777 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:35.602530 26224 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:35.602692 27777 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:35.602751 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:35.603674 27483 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:35.603888 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:35.605600 26369 leader_election.cc:398] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4.
I20260924 14:29:35.605980 27787 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:35.606245 27763 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:35.606900 27763 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:35.606945 26369 leader_election.cc:401] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260924 14:29:35.608211 27787 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4.
I20260924 14:29:35.610450 27194 raft_consensus.cc:1226] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:35.611773 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:35.612033 27787 consensus_queue.cc:1046] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:35.612686 27792 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:35.613113 27792 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:35.613955 27792 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:35.610499 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:35.554544 27720 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:35.606099 26224 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:35.615814 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:35.615739 27191 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:35.620589 27192 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:35.616688 27793 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:35.621042 27186 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:29:35.623611 27789 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:35.606098 27438 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 14:29:35.625778 27807 log.cc:924] Time spent T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260924 14:29:35.616739 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:35.627219 27799 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260924 14:29:35.626835 27804 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.258s	user 0.000s	sys 0.004s
I20260924 14:29:35.587932 27781 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:35.615412 27792 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:35.634492 26369 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:35.631771 27787 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:35.639463 27777 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:35.640652 26249 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:35.641832 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:35.644052 26258 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:35.639566 26251 raft_consensus.cc:3032] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:35.645383 26251 raft_consensus.cc:729] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:35.650942 26251 consensus_queue.cc:260] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:35.652124 26251 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:35.652225 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:35.653030 26253 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:35.654523 27793 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:35.655920 27801 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260924 14:29:35.658216 27719 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:35.658806 27736 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:35.659308 27719 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:35.662595 26253 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:35.663255 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.663947 26406 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:35.664026 27523 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:35.665527 26406 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260924 14:29:35.667480 26406 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:29:35.667665 27483 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:35.669046 27483 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:29:35.669282 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:35.670200 27346 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:35.671609 26406 raft_consensus.cc:1203] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[4.2-4.2]   Dedup: 4.2->[]
I20260924 14:29:35.672127 26406 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260924 14:29:35.672956 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:35.673712 26402 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:35.674015 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:35.674414 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:35.676414 27765 raft_consensus.cc:3032] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:35.676786 27765 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:35.677743 27765 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:35.678357 27192 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
W20260924 14:29:35.660548 27764 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.842s	user 0.012s	sys 0.000s
I20260924 14:29:35.678653 27793 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:35.690068 27711 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:35.690598 27793 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:35.691089 27793 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:35.693200 27733 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:35.693763 26399 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:35.680137 26405 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:35.675134 27182 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:35.682888 27811 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:35.683871 27346 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:35.685886 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:35.686794 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:35.683892 27784 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260924 14:29:35.708623 27720 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:29:35.709800 27186 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:35.689069 26404 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3a94b735387741fca7f83059b64c0917"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:54910
I20260924 14:29:35.680912 26403 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:35.698055 26250 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:35.698899 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:35.705262 27811 consensus_queue.cc:1225] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260924 14:29:35.700826 27764 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:35.705565 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:35.702925 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:35.711344 26404 raft_consensus.cc:595] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:35.700784 27765 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:29:35.722496 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 359061us
I20260924 14:29:35.723623 26404 raft_consensus.cc:603] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260924 14:29:35.725538 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 483772us
I20260924 14:29:35.726853 26402 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:35.660224 27736 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260924 14:29:35.737941 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:35.744561 26295 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:35.747035 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:35.729949 26251 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:35.735133 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:35.715179 27191 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:35.726771 27372 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:35.725852 27787 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:35.735908 27181 raft_consensus.cc:2396] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260924 14:29:35.769433 27565 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260924 14:29:35.769773 27784 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:35.762465 27728 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:35.786680 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:35.788055 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:35.760617 27735 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260924 14:29:35.736879 27483 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:35.739490 27732 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:35.753866 26252 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:35.784871 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:29:35.776067 27783 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
W20260924 14:29:35.789093 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 58427.3us
I20260924 14:29:35.745672 27780 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260924 14:29:35.811877 27194 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:35.769578 27185 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:35.813042 27726 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:35.810058 27707 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:35.813781 27730 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:35.814332 27814 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:35.814404 27731 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:35.778012 27787 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:35.814283 27717 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:35.880553 27717 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:35.881338 27601 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:35.883832 27764 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:35.825440 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:35.786351 27570 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260924 14:29:35.883194 27717 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:35.870761 27777 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:35.821732 27740 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260924 14:29:35.875809 26400 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:35.815614 27784 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:35.822135 27483 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:35.860248 27733 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:35.878532 27814 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:35.879136 27718 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:35.922318 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:35.879436 27728 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260924 14:29:35.929457 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:35.931140 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:35.932900 27827 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260924 14:29:35.936726 26331 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:35.936956 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:35.916139 27777 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:35.927794 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:35.861248 27818 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260924 14:29:35.915339 27711 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:35.943010 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:35.947023 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:35.918936 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:35.934420 27819 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:35.950816 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 87169.1us
I20260924 14:29:35.965431 26402 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:35.934077 27814 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:35.969813 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:35.934820 27795 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:35.941617 27777 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:35.980338 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:35.981264 26332 raft_consensus.cc:2379] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:35.982867 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:35.950511 27764 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:35.950935 27782 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:35.942148 27649 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260924 14:29:35.967437 27780 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:35.943624 27718 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:35.984670 26324 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:35.939203 27733 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:35.965662 27711 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.017923 26404 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:36.012197 27192 raft_consensus.cc:2396] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260924 14:29:35.967114 27819 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:36.041777 27819 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.019964 27609 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:35.990536 27764 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.032459 26331 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:36.050973 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:36.049819 26404 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:36.048466 26326 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260924 14:29:36.056396 27833 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260924 14:29:36.036149 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:36.063333 26253 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:35.990718 27782 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.094152 26331 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:36.094817 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:36.098038 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:35.981832 27787 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.102192 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:36.090648 27783 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:36.111613 27346 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.051496 27609 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:36.118541 27740 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:36.082144 27793 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.122403 26323 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:36.016091 27784 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.099169 27181 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:36.122557 27729 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:36.251052 27438 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:36.259575 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 315439us
I20260924 14:29:36.266346 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:36.108091 27789 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.156s
W20260924 14:29:36.264160 27826 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260924 14:29:36.257757 27836 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:36.254307 27483 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.337082 26295 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:36.268594 27502 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:36.266988 27787 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:36.285912 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.352303 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.268385 27654 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.085s
I20260924 14:29:36.266417 27740 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:36.256677 27649 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260924 14:29:36.357542 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:36.364704 27839 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260924 14:29:36.366547 27825 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260924 14:29:36.361919 26249 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:36.367514 27733 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:36.368255 27793 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.368624 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:36.360080 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 75453.4us
I20260924 14:29:36.258581 27346 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.376699 26332 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:36.381624 27818 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:36.382417 27818 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.348347 27791 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:36.353626 27718 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.282867 27438 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.373955 27483 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
W20260924 14:29:36.389245 27835 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.388s	user 0.001s	sys 0.000s
I20260924 14:29:36.370700 27845 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:36.389748 27438 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.363310 27711 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.375952 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.417248 27483 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:36.422995 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.423182 27483 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.425719 27483 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:36.426450 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.391146 27731 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:36.427702 27837 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:36.403604 27836 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:36.389999 27845 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.485752 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:36.414155 27764 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.390666 27438 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260924 14:29:36.437860 27731 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:36.445683 27372 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.393296 27791 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.439216 27711 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:36.494114 27778 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.017s	sys 0.008s
I20260924 14:29:36.504974 27126 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:36.512145 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 64049.8us
W20260924 14:29:36.501168 27765 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.775s	user 0.012s	sys 0.000s
I20260924 14:29:36.497316 27782 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.504685 27731 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:36.450805 27843 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260924 14:29:36.483103 27777 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.523051 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:36.513247 27778 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.018s	sys 0.008s
I20260924 14:29:36.528262 27580 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:36.509090 27817 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.509975 27372 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.530961 27777 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:36.532212 27346 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.493772 27740 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.535617 26325 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:36.516193 27483 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:36.539880 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.584498 26328 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:36.514181 27711 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.541513 27845 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.542835 27791 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.542923 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:36.545770 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:36.548713 27523 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.549499 27819 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:36.550033 27814 consensus_meta.cc:287] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.002s
I20260924 14:29:36.538805 27718 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.552814 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:36.535157 27818 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.598033 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:36.592442 26401 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:36.572224 26331 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260924 14:29:36.600780 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:36.601485 26258 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:36.596625 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.598960 27819 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.594573 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:36.609475 26403 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:36.609761 27819 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:36.610440 27819 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:36.588713 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:36.569710 27817 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.621657 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:36.622596 26401 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:36.629829 26328 raft_consensus.cc:1203] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Deduplicated request from leader. Original: 1.1->[4.2-4.2]   Dedup: 4.2->[]
I20260924 14:29:36.630178 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:36.600313 27791 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.630596 26328 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260924 14:29:36.631044 26298 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:36.596858 27523 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.613777 27860 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:36.627795 27818 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:36.600975 27814 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.597442 27845 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260924 14:29:36.600160 27718 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.642165 27819 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:36.655085 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:36.652984 27818 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.658577 27814 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.659735 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:36.666879 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.638468 27372 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:36.659314 27860 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:36.636369 27791 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.648918 27346 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260924 14:29:36.674360 27864 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260924 14:29:36.672646 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:36.686273 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:36.687233 26257 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:36.688671 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.677681 27845 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:36.695248 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:36.696825 26328 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:36.683111 27346 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:36.725164 26369 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:36.683645 26326 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:36.730048 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:36.696132 27818 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:36.701319 27707 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:36.731422 27483 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:36.735868 27857 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:36.728469 27784 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:36.731523 27845 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:36.736809 27783 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.737059 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 239) took 1056 ms. Trace:
I20260924 14:29:36.734252 27765 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:36.702754 27346 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:36.743247 27483 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.744065 26324 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:36.742194 27707 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:36.744316 27788 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.703822 27711 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.723750 27852 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:36.749001 27765 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:36.746582 27857 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.758318 27707 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:36.766525 27765 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:36.772583 27718 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:36.770454 27523 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:36.747218 27188 rpcz_store.cc:276] 0924 14:29:35.680772 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.708026 (+ 27254us) service_pool.cc:227] Handling call
0924 14:29:36.736164 (+1028138us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eebce 000000000034aea9 00007f09be21aa30 00007f09be2e07b4 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:36.736362 (+   198us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:36.736568 (+   206us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:36.736599 (+    31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:36.736647 (+    48us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:36.736687 (+    40us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:36.736825 (+   138us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:36.736991 (+   166us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8060288}
I20260924 14:29:36.761183 27817 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.681951 27860 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.748067 27784 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.766335 27845 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:36.773603 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 246) took 1035 ms. Trace:
I20260924 14:29:36.767668 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:36.767257 27852 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.781307 27808 consensus_queue.cc:1046] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260924 14:29:36.788647 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.790181 27711 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.788218 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:36.799602 27502 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:36.824714 27192 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:36.760828 27788 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.828949 27788 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.829691 27765 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.816839 26326 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:36.831524 27523 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.832049 27523 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.834123 27523 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.749619 27783 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.841009 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:36.842312 27783 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.842351 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:36.843393 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 131) took 1478 ms. Trace:
I20260924 14:29:36.843751 27192 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:36.844340 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:36.844571 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:36.845288 26403 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:29:36.836659 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:36.842362 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:36.846541 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:36.847363 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:36.844345 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.848702 26331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:36.849291 26331 raft_consensus.cc:595] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:36.851611 27817 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:36.851274 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:36.852488 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:36.805233 27791 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.821545 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.864055 27483 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:36.864816 26329 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:36.854234 27711 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.869033 27523 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:36.870391 27523 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:36.870795 27523 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:36.790120 27191 rpcz_store.cc:276] 0924 14:29:35.737794 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.789967 (+ 52173us) service_pool.cc:227] Handling call
0924 14:29:36.542083 (+752116us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:36.542266 (+   183us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:36.542452 (+   186us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:29:36.603075 (+ 60623us) log.cc:842] Serialized 14 byte log entry
0924 14:29:36.767228 (+164153us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:36.770479 (+  3251us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:36.772414 (+  1935us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:36.772505 (+    91us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:36.773515 (+  1010us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":84272,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":2108,"raft.queue_time_us":3201,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":9647488,"thread_start_us":182428,"threads_started":2,"wal-append.queue_time_us":177480}
I20260924 14:29:36.843873 26252 rpcz_store.cc:276] 0924 14:29:35.364991 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.367804 (+  2813us) service_pool.cc:227] Handling call
0924 14:29:35.448611 (+ 80807us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:35.827685 (+379074us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:36.843132 (+1015447us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54000c1cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:36.843304 (+   172us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":639592704}
I20260924 14:29:36.859174 27784 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.862191 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 142) took 1139 ms. Trace:
I20260924 14:29:36.874091 26258 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:36.794099 27845 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.847239 26403 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:36.877252 26250 rpcz_store.cc:276] 0924 14:29:35.722983 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.723177 (+   194us) service_pool.cc:227] Handling call
0924 14:29:35.736746 (+ 13569us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:35.748719 (+ 11973us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:35.748751 (+    32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:35.764176 (+ 15425us) log.cc:842] Serialized 26 byte log entry
0924 14:29:35.884329 (+120153us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:35.884372 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:35.884451 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:36.861184 (+976733us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b24003c3d88. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b627ecf4 00007f09b627cdfb 00007f09b630b559 00007f09b630b1e5 00007f09b630a740 00007f09b8ffd578 00007f09b8ffdcd8 00007f09b8fd56e8 00007f09b8fd0aea 00007f09b8fcff36
0924 14:29:36.861904 (+   720us) raft_consensus.cc:1671] finished
0924 14:29:36.861925 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:36.862133 (+   208us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":4885,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":9754,"spinlock_wait_cycles":51294080,"thread_start_us":72639,"threads_started":1,"wal-append.queue_time_us":169959}
I20260924 14:29:36.886309 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.897006 27857 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:36.881130 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:36.898135 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.855746 27808 raft_consensus.cc:3032] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:36.886504 27860 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.873078 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.904137 26326 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:36.877569 27502 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:36.908867 26250 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:36.883268 27781 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:36.905853 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.913626 27781 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:36.911681 27502 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:36.914755 27502 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:36.887048 27791 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:36.904345 27523 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.912735 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:36.891613 27711 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.906512 27808 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:36.928532 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:36.929013 27523 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:36.922999 26329 raft_consensus.cc:2379] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:36.937299 27808 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:36.918473 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:36.920713 27791 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:36.921975 26324 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:36.955410 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:36.952050 27791 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:36.968869 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:36.968863 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:36.973671 27740 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:36.978202 26324 raft_consensus.cc:595] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:29:36.966424 26326 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:36.923840 27731 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:36.949306 27781 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:36.959879 27711 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:36.987313 27792 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:37.010710 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:37.012039 27787 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:36.988433 27760 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:37.014159 26401 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:37.015426 27852 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:37.026397 26224 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:37.028250 27781 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:37.028718 27781 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:37.028846 26327 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:29:37.029233 27781 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:37.028710 27850 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:37.028515 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:37.032020 27731 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:37.032083 27850 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:37.023169 27765 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:37.049196 26327 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:37.023694 27711 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:37.023931 27792 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.044078 27609 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:37.058506 27845 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:37.058246 26403 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:37.069202 27829 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260924 14:29:37.076748 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:37.103132 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:37.080137 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 142) took 1391 ms. Trace:
I20260924 14:29:37.084745 27817 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:37.071698 27346 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:37.086221 27718 raft_consensus.cc:2741] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260924 14:29:37.112730 26327 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260924 14:29:37.112156 27609 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:37.131259 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:37.098273 27707 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:37.123239 27185 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:37.129437 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:37.135067 27609 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:37.152347 27609 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:37.142539 27878 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.084s	user 0.001s	sys 0.001s
I20260924 14:29:37.125644 27760 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260924 14:29:37.140965 27787 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:37.156647 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 147) took 1071 ms. Trace:
I20260924 14:29:37.165918 26404 rpcz_store.cc:276] 0924 14:29:36.084961 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:36.085286 (+   325us) service_pool.cc:227] Handling call
0924 14:29:36.854805 (+769519us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:36.854992 (+   187us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:36.856190 (+  1198us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:36.856239 (+    49us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:36.856716 (+   477us) log.cc:842] Serialized 26 byte log entry
0924 14:29:37.041416 (+184700us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:37.041452 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:37.041522 (+    70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:37.154400 (+112878us) raft_consensus.cc:1671] finished
0924 14:29:37.154425 (+    25us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:37.156577 (+  2152us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":12716,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"spinlock_wait_cycles":6096128,"thread_start_us":184362,"threads_started":1,"wal-append.queue_time_us":219719}
I20260924 14:29:37.171727 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:37.174799 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:37.187791 27787 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:37.186332 27711 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:37.190979 27346 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:37.191690 27787 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.195910 27707 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:37.196635 27836 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:37.194388 27192 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:37.197088 27836 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:37.198124 27836 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:37.196880 27787 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:37.200245 27711 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:37.183598 27792 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:37.202008 27711 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.208648 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:37.209419 26250 raft_consensus.cc:2379] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:37.210243 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:37.210862 26250 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:37.212797 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:37.213124 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:37.213752 26250 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:37.216691 27483 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:37.218102 27483 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:37.218627 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:37.220135 27483 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:37.220222 27787 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:37.220992 27787 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.221887 27740 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:37.222877 27191 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:37.223299 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:37.225211 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:37.225747 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:37.203749 27836 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:37.230032 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:37.239655 27795 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:37.241269 27711 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:37.241664 27732 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:37.241964 27808 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:37.230547 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:37.236163 27846 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.883s	user 0.005s	sys 0.000s
I20260924 14:29:37.237906 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:37.239092 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:37.247964 26323 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:37.223920 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 695) took 2183 ms. Trace:
I20260924 14:29:37.237366 27502 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:37.257475 26256 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:37.106843 26398 rpcz_store.cc:276] 0924 14:29:35.677672 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.678117 (+   445us) service_pool.cc:227] Handling call
0924 14:29:35.678494 (+   377us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:35.678861 (+   367us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:35.678892 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:36.928145 (+1249253us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b2c0013ed68. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:29:36.928876 (+   731us) log.cc:842] Serialized 26 byte log entry
0924 14:29:36.957659 (+ 28783us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:36.958113 (+   454us) log.cc:842] Serialized 14 byte log entry
0924 14:29:37.011538 (+ 53425us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:37.027685 (+ 16147us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:37.038450 (+ 10765us) raft_consensus.cc:1671] finished
0924 14:29:37.046356 (+  7906us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:37.068630 (+ 22274us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":38072,"prepare.queue_time_us":9998,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":48942,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":6847,"spinlock_wait_cycles":74126208,"thread_start_us":28109,"threads_started":1,"wal-append.queue_time_us":28729}
I20260924 14:29:37.265285 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 736) took 1326 ms. Trace:
I20260924 14:29:37.249185 26400 rpcz_store.cc:276] 0924 14:29:35.041287 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.049326 (+  8039us) service_pool.cc:227] Handling call
0924 14:29:35.068076 (+ 18750us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:35.919067 (+850991us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:37.223635 (+1304568us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b54001e41d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:37.223842 (+   207us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1830702976}
I20260924 14:29:37.267598 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:37.269577 27887 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260924 14:29:37.271801 27787 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:37.272905 26399 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:37.276214 27502 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:37.265792 26255 rpcz_store.cc:276] 0924 14:29:35.938607 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.944457 (+  5850us) service_pool.cc:227] Handling call
0924 14:29:35.989313 (+ 44856us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:36.024408 (+ 35095us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:36.035877 (+ 11469us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:36.055867 (+ 19990us) log.cc:842] Serialized 26 byte log entry
0924 14:29:36.118438 (+ 62571us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:36.124718 (+  6280us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:37.152360 (+1027642us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b54000c26d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:37.152419 (+    59us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:37.235854 (+ 83435us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b24003c3d88. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b627ecf4 00007f09b627cdfb 00007f09b630b559 00007f09b630b1e5 00007f09b630a82d 00007f09b8fb946e 00007f09b8fd0c0c 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551
0924 14:29:37.249649 (+ 13795us) raft_consensus.cc:1671] finished
0924 14:29:37.258464 (+  8815us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:37.265217 (+  6753us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":410,"raft.queue_time_us":14071,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":6798,"spinlock_wait_cycles":12364544,"thread_start_us":44328,"threads_started":1,"wal-append.queue_time_us":210341}
I20260924 14:29:37.278088 27125 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:29:37.282327 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47708.7us
I20260924 14:29:37.290060 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:37.302304 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 243) took 1620 ms. Trace:
I20260924 14:29:37.310349 26256 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:37.302817 27186 rpcz_store.cc:276] 0924 14:29:35.682174 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:36.032895 (+350721us) service_pool.cc:227] Handling call
0924 14:29:37.109186 (+1076291us) spinlock_profiling.cc:243] Waited 3.2 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:37.109503 (+   317us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:37.109917 (+   414us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:37.109956 (+    39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:37.114597 (+  4641us) log.cc:842] Serialized 26 byte log entry
0924 14:29:37.142892 (+ 28295us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:37.150118 (+  7226us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:37.151093 (+   975us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:37.277764 (+126671us) raft_consensus.cc:1671] finished
0924 14:29:37.280512 (+  2748us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:37.302198 (+ 21686us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":827,"raft.run_wall_time_us":4193,"spinlock_wait_cycles":21716224,"thread_start_us":21206,"threads_started":1,"wal-append.queue_time_us":51568}
W20260924 14:29:37.314349 27785 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260924 14:29:37.315621 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 720) took 2396 ms. Trace:
I20260924 14:29:37.318449 27791 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.290898 27502 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:37.288443 27793 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260924 14:29:37.288890 27711 consensus_queue.cc:1046] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260924 14:29:37.325160 27816 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:37.370743 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:37.305033 27818 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:37.364558 27876 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260924 14:29:37.351168 27891 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:37.388779 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:37.388626 27781 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.389511 26326 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:37.370550 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:37.391752 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:37.393066 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 115058us
I20260924 14:29:37.394651 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:37.371557 27837 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:37.392554 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:37.409286 26255 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:37.411142 27779 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:37.392566 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 188) took 1029 ms. Trace:
I20260924 14:29:37.411748 27779 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:37.412470 27189 rpcz_store.cc:276] 0924 14:29:36.363206 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.370333 (+1007127us) service_pool.cc:227] Handling call
0924 14:29:37.392505 (+ 22172us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:37.413630 27743 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:37.413931 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 143) took 1736 ms. Trace:
I20260924 14:29:37.415565 26324 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:37.414613 26397 rpcz_store.cc:276] 0924 14:29:35.677951 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.679194 (+  1243us) service_pool.cc:227] Handling call
0924 14:29:35.679582 (+   388us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:35.679955 (+   373us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:35.679980 (+    25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:37.172280 (+1492300us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b2c0013ed68. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:29:37.185133 (+ 12853us) log.cc:842] Serialized 26 byte log entry
0924 14:29:37.358948 (+173815us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:37.367729 (+  8781us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:37.374758 (+  7029us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:37.413117 (+ 38359us) raft_consensus.cc:1671] finished
0924 14:29:37.413137 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:37.413865 (+   728us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":306,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"spinlock_wait_cycles":77187328,"thread_start_us":138218,"threads_started":1,"wal-append.queue_time_us":203006}
I20260924 14:29:37.382467 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:37.393143 27852 raft_consensus.cc:3032] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:37.427863 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:37.429559 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:37.421758 27822 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:37.432449 27781 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:37.432410 27822 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:37.432886 27779 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:37.385916 27891 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:37.434624 26297 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:37.407552 27791 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:37.383949 27818 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:37.379839 27816 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:37.316447 26256 rpcz_store.cc:276] 0924 14:29:34.915675 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:34.915823 (+   148us) service_pool.cc:227] Handling call
0924 14:29:34.916162 (+   339us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:35.413550 (+497388us) spinlock_profiling.cc:243] Waited 85.4 ms on lock 0x7b5400645120. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92de730 00007f09b9350300 00007f09b9351371 00007f09b934dedf 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc
0924 14:29:35.995253 (+581703us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b58002252b0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92c9f63 00007f09b932daba 00007f09b93453af 00007f09b9346b4e 00007f09b9350c89 00007f09b9351556 00007f09b934dedf 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684
0924 14:29:37.247920 (+1252667us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b58002255b0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92c9f63 00007f09b932daba 00007f09b93453af 00007f09b9346b4e 00007f09b9350c89 00007f09b9351556 00007f09b934dedf 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684
0924 14:29:37.311709 (+ 63789us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:37.312266 (+   557us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1858431616}
I20260924 14:29:37.414117 27743 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:37.434024 27780 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:37.435711 27787 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:29:37.468052 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 74030.5us
I20260924 14:29:37.424018 27852 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:37.465809 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:37.453439 27601 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:37.453562 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:37.468133 27793 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:37.466665 27892 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:37.478456 26256 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:37.451507 27891 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:37.478921 26255 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:37.471757 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:37.476544 27852 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:37.511955 27188 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:37.512593 27852 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:37.482653 26323 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:37.513103 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:37.435974 27837 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.501726 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:37.501379 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:37.515981 27190 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 4
I20260924 14:29:37.516924 26252 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:55136
I20260924 14:29:37.517647 26252 raft_consensus.cc:595] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:37.518080 26252 raft_consensus.cc:603] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:29:37.502774 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:37.470654 27893 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260924 14:29:37.519797 27189 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:37.521121 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:29:37.521903 27893 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:37.522483 27893 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.483255 27787 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:37.523818 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:29:37.524628 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:29:37.480259 27781 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:37.525832 27781 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.527532 27818 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:37.527865 27781 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:37.528069 27818 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:37.528586 27818 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:37.523825 27787 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:37.530454 27818 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:37.509264 27707 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260924 14:29:37.476471 26324 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:37.471688 27817 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:37.525018 27791 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:37.531378 27787 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:37.489612 27822 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:37.504128 26326 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:37.538771 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:37.513077 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:37.487300 27844 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.138s	user 0.005s	sys 0.000s
I20260924 14:29:37.472411 27346 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:37.485633 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:37.490163 27779 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:37.564692 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 139) took 1869 ms. Trace:
I20260924 14:29:37.566093 26327 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:37.544340 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:37.544094 27791 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:37.545372 27743 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:37.507565 27892 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.547724 26324 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260924 14:29:37.550047 27502 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:37.550205 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:37.478678 27793 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:37.613276 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 63157.5us
I20260924 14:29:37.612107 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:37.536242 26399 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:37.616415 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:37.476012 27780 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.623600 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:37.542853 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:37.589486 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50841.1us
I20260924 14:29:37.565165 26253 rpcz_store.cc:276] 0924 14:29:35.695321 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.696942 (+  1621us) service_pool.cc:227] Handling call
0924 14:29:35.706333 (+  9391us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:36.089520 (+383187us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:37.562361 (+1472841us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54004fa454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:37.564630 (+  2269us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":820047104}
I20260924 14:29:37.625898 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:37.592204 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:37.543260 27817 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:37.611725 27847 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:37.591408 27182 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:37.746884 26256 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:37.616750 27732 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:37.741464 26327 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:37.793678 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:37.796707 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:37.620611 27711 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:37.618391 27523 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.177s
I20260924 14:29:37.743654 27899 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:37.591851 27779 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:37.586676 27346 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:37.752701 27707 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:37.800058 27844 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260924 14:29:37.895596 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:37.896703 26256 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:37.626453 27795 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:37.741115 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:37.800340 27186 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:37.880849 27847 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260924 14:29:37.835080 27787 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:37.796036 26398 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
W20260924 14:29:37.536612 27765 consensus_queue.cc:1159] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e09fbebe29a49d291d23f9d395fe335" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:29:37.752485 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 126) took 1186 ms. Trace:
I20260924 14:29:37.933950 27783 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:37.739902 27781 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.178s
I20260924 14:29:37.943336 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 735) took 2006 ms. Trace:
W20260924 14:29:37.919983 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 365766us
I20260924 14:29:37.952970 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:37.888157 27733 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:37.954871 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:37.954044 27787 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:37.970916 27821 raft_consensus.cc:979] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: : Instructing follower 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:37.983345 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:37.948307 27893 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:37.917467 27707 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:37.892990 27897 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260924 14:29:38.002357 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:37.985751 26254 rpcz_store.cc:276] 0924 14:29:35.938259 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.939051 (+   792us) service_pool.cc:227] Handling call
0924 14:29:35.952882 (+ 13831us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:35.966670 (+ 13788us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:35.966713 (+    43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:35.976243 (+  9530us) log.cc:842] Serialized 26 byte log entry
0924 14:29:36.103299 (+127056us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:36.245987 (+142688us) log.cc:842] Serialized 14 byte log entry
0924 14:29:36.488684 (+242697us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:36.488797 (+   113us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:37.486528 (+997731us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:29:37.942932 (+456404us) raft_consensus.cc:1671] finished
0924 14:29:37.943083 (+   151us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:37.943258 (+   175us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":157819,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":21475,"raft.run_cpu_time_us":482,"raft.run_wall_time_us":14886,"spinlock_wait_cycles":13933824,"thread_start_us":344203,"threads_started":3,"wal-append.queue_time_us":255826}
I20260924 14:29:37.968312 27795 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:38.014178 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:38.019244 27842 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.666s	user 0.007s	sys 0.000s
I20260924 14:29:38.020538 27892 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:38.021621 27893 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:37.893638 27899 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:37.988888 27783 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:38.000583 27787 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:38.021049 27847 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:38.013113 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:38.073189 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:38.044034 27892 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.076107 27910 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:38.075814 27892 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:38.041338 27793 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:38.078084 27892 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:38.038874 27780 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:38.078608 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 737) took 2139 ms. Trace:
I20260924 14:29:37.964059 27181 rpcz_store.cc:276] 0924 14:29:36.554527 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:36.564577 (+ 10050us) service_pool.cc:227] Handling call
0924 14:29:37.485393 (+920816us) spinlock_profiling.cc:243] Waited 8.46 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:37.740880 (+255487us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":19458432}
I20260924 14:29:38.046676 27899 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:38.027978 27711 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:38.040251 27893 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:38.048373 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 248) took 2057 ms. Trace:
W20260924 14:29:38.049340 27900 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260924 14:29:38.013734 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.052932 27783 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:38.056154 27896 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260924 14:29:37.999477 27791 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:38.056820 27817 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:38.022420 27819 raft_consensus.cc:1067] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Signalling peer 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:38.059079 27570 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:38.041899 27905 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.062031 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.065574 27850 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:38.067693 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:38.068122 27189 consensus_meta.cc:287] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.001s
I20260924 14:29:38.068691 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 133) took 1248 ms. Trace:
I20260924 14:29:38.122159 27185 rpcz_store.cc:276] 0924 14:29:36.819979 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:36.835694 (+ 15715us) service_pool.cc:227] Handling call
0924 14:29:36.857916 (+ 22222us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:37.150307 (+292391us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.037949 (+887642us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b540000cb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:38.068596 (+ 30647us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":595954048}
I20260924 14:29:38.123821 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:38.125342 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.126748 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:37.998608 27733 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:38.128096 27911 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:38.129127 27733 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:38.128800 27911 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:38.070861 27891 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.534s	user 0.003s	sys 0.006s
I20260924 14:29:38.129992 27891 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:38.079075 26251 rpcz_store.cc:276] 0924 14:29:35.938919 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.966674 (+ 27755us) service_pool.cc:227] Handling call
0924 14:29:35.967018 (+   344us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:35.976679 (+  9661us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:35.976776 (+    97us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:35.980790 (+  4014us) log.cc:842] Serialized 26 byte log entry
0924 14:29:36.091890 (+111100us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:36.091933 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:36.092004 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:36.833006 (+741002us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b4800000fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:29:38.018945 (+1185939us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b24003c3e18. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b627ecf4 00007f09b627cdfb 00007f09b630b559 00007f09b630b1e5 00007f09b630a82d 00007f09b8fb946e 00007f09b8fd0c0c 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551
0924 14:29:38.078389 (+ 59444us) raft_consensus.cc:1671] finished
0924 14:29:38.078414 (+    25us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:38.078548 (+   134us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":4034,"raft.run_cpu_time_us":6768,"raft.run_wall_time_us":858115,"spinlock_wait_cycles":19893760,"thread_start_us":80171,"threads_started":1,"wal-append.queue_time_us":362745}
I20260924 14:29:38.131636 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:38.132119 27188 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:38.096705 27711 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:38.132530 27891 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:38.132759 27711 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:38.040580 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:38.097162 27194 rpcz_store.cc:276] 0924 14:29:35.990736 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.990925 (+   189us) service_pool.cc:227] Handling call
0924 14:29:35.991239 (+   314us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:35.994664 (+  3425us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:35.994697 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:35.995173 (+   476us) log.cc:842] Serialized 26 byte log entry
0924 14:29:37.363243 (+1368070us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b18001b1660. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:29:37.603595 (+240352us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:37.603648 (+    53us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:37.603742 (+    94us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:38.035631 (+431889us) raft_consensus.cc:1671] finished
0924 14:29:38.035653 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:38.048289 (+ 12636us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":8277,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":7393,"spinlock_wait_cycles":2315520,"thread_start_us":240072,"threads_started":1,"wal-append.queue_time_us":1917182}
I20260924 14:29:38.134318 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.136590 27186 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:38.118449 26405 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:38.137516 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.138648 27189 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:38.139643 27905 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:38.140077 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:38.141507 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.142105 27905 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:38.142721 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.135394 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.144196 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.095108 27847 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:38.108171 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 182652us
I20260924 14:29:38.129925 27787 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:38.170522 27787 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:38.171139 27787 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:38.171473 26399 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:38.171823 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:38.174002 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:38.165915 26405 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:38.176537 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:38.139124 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.184376 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.115695 27791 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.210122 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:38.131332 27911 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:38.247416 26332 raft_consensus.cc:2354] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4.
I20260924 14:29:38.253039 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:38.166556 27779 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:38.204391 27896 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260924 14:29:38.210121 27190 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.694s	user 0.012s	sys 0.000s
I20260924 14:29:38.171157 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 138) took 1023 ms. Trace:
I20260924 14:29:38.166183 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:38.262223 26332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 81) took 1815 ms (client timeout 2099 ms). Trace:
I20260924 14:29:38.281196 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:38.210391 27346 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:38.171602 27787 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:38.279247 26221 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 1.509s (SENT) [suppressed 11 similar messages]
I20260924 14:29:38.312789 27346 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:38.285544 26332 rpcz_store.cc:269] 0924 14:29:36.438177 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:36.438428 (+   251us) service_pool.cc:227] Handling call
0924 14:29:38.184167 (+1745739us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:38.253889 (+ 69722us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3600128}
I20260924 14:29:38.313481 27346 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:38.318814 27346 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:38.318970 27707 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:38.280835 27191 rpcz_store.cc:276] 0924 14:29:37.147534 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.148285 (+   751us) service_pool.cc:227] Handling call
0924 14:29:37.172815 (+ 24530us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:37.223339 (+ 50524us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.170902 (+947563us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b54003269d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:38.171087 (+   185us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":111943552}
I20260924 14:29:38.322108 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.323525 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.323762 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 546) took 2286 ms. Trace:
I20260924 14:29:38.324482 27189 raft_consensus.cc:2396] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260924 14:29:38.325416 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:38.260103 27791 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:38.325981 27189 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:38.178519 27847 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:38.327527 27791 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:38.283663 27190 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:38.328833 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.329203 27847 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:38.332119 27893 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:38.332559 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.340054 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.181165 27795 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:38.324146 26406 rpcz_store.cc:276] 0924 14:29:36.037267 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:36.037412 (+   145us) service_pool.cc:227] Handling call
0924 14:29:36.846362 (+808950us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:37.558583 (+712221us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63eeb7c 00007f09b63ee73b 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:37.558783 (+   200us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:37.588556 (+ 29773us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:37.588585 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:37.622236 (+ 33651us) log.cc:842] Serialized 26 byte log entry
0924 14:29:38.027163 (+404927us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:38.027239 (+    76us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.027331 (+    92us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:38.323518 (+296187us) raft_consensus.cc:1671] finished
0924 14:29:38.323539 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:38.323688 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7130,"raft.queue_time_us":31704,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":14193,"spinlock_wait_cycles":23834752,"thread_start_us":227242,"threads_started":1,"wal-append.queue_time_us":522433}
I20260924 14:29:38.345454 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:38.341883 27182 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.516s	user 0.007s	sys 0.004s
I20260924 14:29:38.422541 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:38.433890 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:38.446663 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 450010us
W20260924 14:29:38.505400 27124 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.006s	sys 0.022s
W20260924 14:29:38.505980 27124 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.006s	sys 0.022s
I20260924 14:29:38.422359 27731 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:38.520793 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 80871.4us
W20260924 14:29:38.546759 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.002s	sys 0.022s
W20260924 14:29:38.555913 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.002s	sys 0.022s
I20260924 14:29:38.535276 26298 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:38.508260 26403 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.007s	sys 0.014s
I20260924 14:29:38.422669 27182 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:38.573503 27125 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:38.508004 27852 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.616401 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:38.616955 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:38.571934 26403 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.007s	sys 0.015s
I20260924 14:29:38.644131 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:38.646196 27123 leader_election.cc:398] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4.
I20260924 14:29:38.647024 27123 leader_election.cc:401] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:29:38.561753 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.652146 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.654985 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:38.574208 27899 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.615144 27733 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.637205 27813 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
W20260924 14:29:38.668241 26257 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.000s	sys 0.034s
I20260924 14:29:38.563246 27731 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:38.649608 27711 consensus_meta.cc:287] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.002s
I20260924 14:29:38.685604 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.642045 27847 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:38.619575 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 190) took 1114 ms. Trace:
W20260924 14:29:38.714298 26257 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.000s	sys 0.040s
I20260924 14:29:38.719412 27897 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4.
W20260924 14:29:38.711577 26255 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.008s	sys 0.032s
W20260924 14:29:38.718703 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 439474us
I20260924 14:29:38.648437 27893 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:38.664506 27814 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
W20260924 14:29:38.685211 27902 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.017s	sys 0.019s
I20260924 14:29:38.653455 27779 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:38.709461 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 274) took 1213 ms. Trace:
W20260924 14:29:38.748469 26255 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.008s	sys 0.032s
W20260924 14:29:38.752754 27902 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.018s	sys 0.019s
I20260924 14:29:38.724455 27813 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:38.755611 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 159) took 1187 ms. Trace:
I20260924 14:29:38.785648 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.724334 27740 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:38.689633 27899 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:38.791679 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:38.644708 27852 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:38.799655 26249 raft_consensus.cc:1226] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260924 14:29:38.800592 27914 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:38.791386 27791 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:38.720216 27182 rpcz_store.cc:276] 0924 14:29:37.505375 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.512788 (+  7413us) service_pool.cc:227] Handling call
0924 14:29:38.612768 (+1099980us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:38.783459 27787 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:38.802299 26258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 402) took 2027 ms (client timeout 2048 ms). Trace:
W20260924 14:29:38.832119 26258 rpcz_store.cc:269] 0924 14:29:36.769817 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:36.852249 (+ 82432us) service_pool.cc:227] Handling call
0924 14:29:38.796792 (+1944543us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b5400716454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:38.796955 (+   163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":48156032}
I20260924 14:29:38.833644 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:38.766961 27926 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:38.803922 27919 log.cc:924] Time spent T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260924 14:29:38.804193 27836 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:38.837152 27836 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:38.767975 27185 rpcz_store.cc:276] 0924 14:29:37.496232 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.436914 (+940682us) service_pool.cc:227] Handling call
0924 14:29:38.562737 (+125823us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:38.619799 (+ 57062us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:38.632424 (+ 12625us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:38.644061 (+ 11637us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:38.648967 (+  4906us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.659630 (+ 10663us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:38.692110 (+ 32480us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:29:38.726133 27783 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.000s	sys 0.021s
W20260924 14:29:38.839725 27783 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.000s	sys 0.022s
I20260924 14:29:38.718993 27711 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:38.840092 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.841418 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.841967 27711 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:38.841786 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.842581 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.842875 27181 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:38.762096 27814 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:38.843637 26251 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:38.844434 27814 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:38.839361 27836 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:38.845077 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:38.846330 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 272) took 1350 ms. Trace:
I20260924 14:29:38.849716 27852 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:38.850453 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:38.851697 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:38.852387 26258 raft_consensus.cc:2354] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4.
I20260924 14:29:38.853169 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:38.856312 27914 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:38.857702 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.858053 27837 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:38.858994 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.859656 27185 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:38.860515 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.862358 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 193) took 1348 ms. Trace:
I20260924 14:29:38.862926 27837 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:38.863961 27837 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:38.866430 27837 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:38.867641 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:38.868492 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.869019 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:38.869295 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 196) took 1335 ms. Trace:
I20260924 14:29:38.869678 27194 rpcz_store.cc:276] 0924 14:29:37.533717 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.145080 (+611363us) service_pool.cc:227] Handling call
0924 14:29:38.868329 (+723249us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:38.869236 (+   907us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":48052096}
I20260924 14:29:38.870186 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:38.873946 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 163) took 1272 ms. Trace:
W20260924 14:29:38.823326 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 421584us
I20260924 14:29:38.874607 26404 rpcz_store.cc:276] 0924 14:29:37.601508 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.615530 (+ 14022us) service_pool.cc:227] Handling call
0924 14:29:37.743477 (+127947us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:37.802219 (+ 58742us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:37.802266 (+    47us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:37.802323 (+    57us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:37.802374 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.867391 (+1065017us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b54002756d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:38.873671 (+  6280us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:38.873855 (+   184us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":134449024}
I20260924 14:29:38.871254 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 195) took 1337 ms. Trace:
I20260924 14:29:38.879148 27194 rpcz_store.cc:276] 0924 14:29:37.533385 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.870475 (+1337090us) service_pool.cc:227] Handling call
0924 14:29:38.870762 (+   287us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:38.870951 (+   189us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:38.870979 (+    28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:38.871020 (+    41us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:38.871051 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.871104 (+    53us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:38.871205 (+   101us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:38.880858 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:38.817932 26397 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.006s	sys 0.008s
I20260924 14:29:38.723996 27847 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:38.716703 27733 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:38.808969 27696 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.029s	sys 0.032s
I20260924 14:29:38.828086 27913 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.827917 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.846743 27191 rpcz_store.cc:276] 0924 14:29:37.495699 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.321403 (+825704us) service_pool.cc:227] Handling call
0924 14:29:38.844884 (+523481us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:38.846259 (+  1375us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":35159424}
I20260924 14:29:38.871270 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:38.828812 27791 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:38.901686 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:38.901003 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:38.873049 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 267) took 1379 ms. Trace:
I20260924 14:29:38.904423 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.904547 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.905246 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.905478 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:38.906474 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 251571us
W20260924 14:29:38.942538 26371 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.012s	sys 0.039s
W20260924 14:29:38.942901 26371 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.012s	sys 0.039s
I20260924 14:29:38.911157 27194 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:38.912052 26258 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:38.914062 26325 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:38.831839 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:38.822587 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:38.906466 27696 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.030s	sys 0.032s
I20260924 14:29:38.898244 27837 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.988016 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 767) took 1449 ms. Trace:
I20260924 14:29:38.905475 27899 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:38.996485 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:38.916260 27570 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:38.997295 26258 rpcz_store.cc:276] 0924 14:29:37.538270 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.987053 (+1448783us) service_pool.cc:227] Handling call
0924 14:29:38.987514 (+   461us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:38.987684 (+   170us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:38.987705 (+    21us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:38.987746 (+    41us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:38.987771 (+    25us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.987827 (+    56us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:38.987956 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:29:38.991870 27933 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260924 14:29:38.917052 27812 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.917690 27783 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.916644 27836 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.009691 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 768) took 1126 ms. Trace:
I20260924 14:29:39.010867 26258 rpcz_store.cc:276] 0924 14:29:37.882820 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.006912 (+1124092us) service_pool.cc:227] Handling call
0924 14:29:39.007357 (+   445us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.009201 (+  1844us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.009256 (+    55us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.009322 (+    66us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:39.009382 (+    60us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.009465 (+    83us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.009625 (+   160us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:39.013545 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 409) took 1121 ms. Trace:
I20260924 14:29:39.013969 26258 rpcz_store.cc:276] 0924 14:29:37.892181 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.012293 (+1120112us) service_pool.cc:227] Handling call
0924 14:29:39.012712 (+   419us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.012921 (+   209us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.012957 (+    36us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.013013 (+    56us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:39.013055 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.013124 (+    69us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.013317 (+   193us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:38.941020 27852 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.015484 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 158) took 1532 ms. Trace:
I20260924 14:29:38.862802 27185 rpcz_store.cc:276] 0924 14:29:37.514261 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.861414 (+1347153us) service_pool.cc:227] Handling call
0924 14:29:38.861854 (+   440us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:38.862031 (+   177us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:38.862057 (+    26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:38.862098 (+    41us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:38.862131 (+    33us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.862188 (+    57us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:38.862307 (+   119us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:39.016155 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.021914 26250 rpcz_store.cc:276] 0924 14:29:37.483173 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.996642 (+513469us) service_pool.cc:227] Handling call
0924 14:29:38.026284 (+ 29642us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:38.026536 (+   252us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:29:38.524019 (+497483us) log.cc:842] Serialized 14 byte log entry
0924 14:29:39.015126 (+491107us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.015180 (+    54us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:39.015217 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.015278 (+    61us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.015397 (+   119us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":116850,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":3022,"thread_start_us":934318,"threads_started":2}
I20260924 14:29:38.989038 26330 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:39.005087 27779 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:39.007700 27189 consensus_meta.cc:287] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.681s	user 0.014s	sys 0.000s
W20260924 14:29:39.020879 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 485646us
I20260924 14:29:38.946266 27740 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.826225 27813 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:39.022828 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 769) took 1038 ms. Trace:
I20260924 14:29:38.913583 27188 rpcz_store.cc:276] 0924 14:29:37.494004 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.512708 (+ 18704us) service_pool.cc:227] Handling call
0924 14:29:38.872836 (+1360128us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b540079f9d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:38.872980 (+   144us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1418182784}
I20260924 14:29:39.004047 27925 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:38.913975 27913 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:38.999311 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 55113us
I20260924 14:29:39.058138 26258 rpcz_store.cc:276] 0924 14:29:37.984603 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.022008 (+1037405us) service_pool.cc:227] Handling call
0924 14:29:39.022350 (+   342us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.022517 (+   167us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.022541 (+    24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.022578 (+    37us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:39.022604 (+    26us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.022650 (+    46us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.022762 (+   112us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:29:39.029385 26252 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.009s	sys 0.010s
I20260924 14:29:39.067869 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 410) took 1082 ms. Trace:
I20260924 14:29:39.036626 27899 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:39.003919 27837 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.081655 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 162) took 1083 ms. Trace:
I20260924 14:29:39.041877 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 766) took 1504 ms. Trace:
I20260924 14:29:39.041002 27191 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:39.086050 27787 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:39.087317 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 207) took 1538 ms. Trace:
I20260924 14:29:39.087708 27191 rpcz_store.cc:276] 0924 14:29:37.548602 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.941125 (+1392523us) service_pool.cc:227] Handling call
0924 14:29:39.004961 (+ 63836us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.086998 (+ 82037us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.087236 (+   238us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:39.088508 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.080715 26250 rpcz_store.cc:276] 0924 14:29:37.985270 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.026146 (+1040876us) service_pool.cc:227] Handling call
0924 14:29:39.067219 (+ 41073us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.067465 (+   246us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.067509 (+    44us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.067562 (+    53us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:39.067604 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.067669 (+    65us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.067801 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:39.090160 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:39.086431 26257 rpcz_store.cc:276] 0924 14:29:37.537970 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.135909 (+597939us) service_pool.cc:227] Handling call
0924 14:29:38.136212 (+   303us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:38.136519 (+   307us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:38.136546 (+    27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:38.136987 (+   441us) log.cc:842] Serialized 26 byte log entry
0924 14:29:38.722741 (+585754us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:38.723116 (+   375us) log.cc:842] Serialized 14 byte log entry
0924 14:29:38.724497 (+  1381us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.724588 (+    91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:39.041654 (+317066us) raft_consensus.cc:1671] finished
0924 14:29:39.041672 (+    18us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.041812 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10715,"prepare.queue_time_us":153,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":2074,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":16182,"thread_start_us":585438,"threads_started":1,"wal-append.queue_time_us":695612}
I20260924 14:29:39.056303 27731 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:39.093389 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 771) took 1034 ms. Trace:
I20260924 14:29:39.019279 27812 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.018539 27836 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.111199 26257 rpcz_store.cc:276] 0924 14:29:38.058622 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.092395 (+1033773us) service_pool.cc:227] Handling call
0924 14:29:39.092733 (+   338us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.092995 (+   262us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.093024 (+    29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.093063 (+    39us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:39.093087 (+    24us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.093137 (+    50us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.093332 (+   195us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:39.054946 27189 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
W20260924 14:29:39.082420 26252 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.009s	sys 0.014s
I20260924 14:29:39.064694 27918 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.086169 26258 rpcz_store.cc:276] 0924 14:29:37.997904 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.079793 (+1081889us) service_pool.cc:227] Handling call
0924 14:29:39.080154 (+   361us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.080343 (+   189us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.080371 (+    28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.081380 (+  1009us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:39.081420 (+    40us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.081484 (+    64us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.081599 (+   115us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:39.017999 27783 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:39.034063 27852 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.114176 27913 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:39.128500 26295 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.011s	sys 0.020s
I20260924 14:29:39.057307 27779 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:39.135071 26295 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.011s	sys 0.020s
I20260924 14:29:39.064487 27740 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:39.138836 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 680013us
I20260924 14:29:39.114455 27182 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:50086
I20260924 14:29:39.082141 27925 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:39.140784 27733 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.141680 27836 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.141991 27779 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.116706 27899 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:39.145123 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:39.134853 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 772) took 1076 ms. Trace:
I20260924 14:29:39.087317 27787 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.157835 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 157) took 1675 ms. Trace:
W20260924 14:29:39.204185 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 59175.3us
I20260924 14:29:39.130077 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 268) took 1635 ms. Trace:
I20260924 14:29:39.205171 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.204429 26256 rpcz_store.cc:276] 0924 14:29:37.482502 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.915299 (+432797us) service_pool.cc:227] Handling call
0924 14:29:37.919041 (+  3742us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:37.919274 (+   233us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:29:38.346281 (+427007us) log.cc:842] Serialized 14 byte log entry
0924 14:29:39.157494 (+811213us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.157547 (+    53us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:39.157586 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.157649 (+    63us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.157780 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16978,"prepare.queue_time_us":223960,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":37006,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":18361,"thread_start_us":1205957,"threads_started":3,"wal-append.queue_time_us":791367}
I20260924 14:29:39.129494 27711 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:39.203716 26257 rpcz_store.cc:276] 0924 14:29:38.058911 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.118537 (+1059626us) service_pool.cc:227] Handling call
0924 14:29:39.134395 (+ 15858us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.134564 (+   169us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.134587 (+    23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.134628 (+    41us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:39.134657 (+    29us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.134702 (+    45us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.134802 (+   100us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:39.129580 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.212721 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:39.143455 27913 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.166795 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.144210 27914 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.222713 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:39.219121 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.224612 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.214304 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 155) took 1733 ms. Trace:
I20260924 14:29:39.233875 27125 leader_election.cc:398] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4.
I20260924 14:29:39.234444 27125 leader_election.cc:401] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260924 14:29:39.237890 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.238930 26325 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:39.209300 27926 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:39.214241 27836 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.211627 27779 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.212956 27791 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:39.243954 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 153845us
I20260924 14:29:39.133104 27918 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.228098 26253 rpcz_store.cc:276] 0924 14:29:37.481000 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.791086 (+310086us) service_pool.cc:227] Handling call
0924 14:29:37.833423 (+ 42337us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:37.882640 (+ 49217us) raft_consensus.cc:1520] Early marking committed up to index 2
0924 14:29:38.364430 (+481790us) log.cc:842] Serialized 14 byte log entry
0924 14:29:39.212081 (+847651us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.212156 (+    75us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:39.212217 (+    61us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.214096 (+  1879us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.214240 (+   144us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":34360,"prepare.queue_time_us":260398,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":29312,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":1273517,"threads_started":3,"wal-append.queue_time_us":773427}
I20260924 14:29:39.213703 27711 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:39.216065 27952 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.225070 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:39.304657 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.247323 26327 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:39.226794 27718 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.207769 27189 rpcz_store.cc:276] 0924 14:29:37.494314 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.325169 (+830855us) service_pool.cc:227] Handling call
0924 14:29:39.127584 (+802415us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:39.224663 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.241346 27733 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:39.233929 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 765) took 1736 ms. Trace:
I20260924 14:29:39.211303 27899 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.313299 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 174) took 1128 ms. Trace:
I20260924 14:29:39.296474 27914 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:39.327477 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:38.809701 26403 rpcz_store.cc:276] 0924 14:29:37.554987 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.562027 (+  7040us) service_pool.cc:227] Handling call
0924 14:29:37.741919 (+179892us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:37.799859 (+ 57940us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:29:37.923075 (+123216us) log.cc:842] Serialized 14 byte log entry
0924 14:29:38.663928 (+740853us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:38.667602 (+  3674us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:38.708678 (+ 41076us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.719483 (+ 10805us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:38.741818 (+ 22335us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":39660,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":149489,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"spinlock_wait_cycles":20739840,"thread_start_us":703332,"threads_started":1,"wal-append.queue_time_us":791071}
I20260924 14:29:39.330588 26406 rpcz_store.cc:276] 0924 14:29:38.184454 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.425137 (+240683us) service_pool.cc:227] Handling call
0924 14:29:38.425466 (+   329us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:38.425680 (+   214us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:29:38.467322 (+ 41642us) log.cc:842] Serialized 14 byte log entry
0924 14:29:39.312965 (+845643us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.313029 (+    64us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:39.313078 (+    49us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.313138 (+    60us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.313245 (+   107us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2581,"prepare.queue_time_us":186262,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":57,"thread_start_us":793317,"threads_started":2,"wal-append.queue_time_us":685746}
I20260924 14:29:39.329982 27346 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:39.317612 27814 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:39.236454 27913 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.364389 27733 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:39.248226 27836 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:39.332747 27927 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.005s	sys 0.024s
I20260924 14:29:39.365693 27814 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:39.327287 27718 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.428570 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:39.432857 27852 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.433073 27925 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.436098 27816 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.435818 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:39.308815 27779 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:39.390762 27927 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.005s	sys 0.024s
I20260924 14:29:39.460505 26253 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:39.321192 27952 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.357230 27914 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:39.362315 27346 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:39.470095 27852 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:39.475163 27954 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260924 14:29:39.475872 27946 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260924 14:29:39.405602 27837 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:39.479837 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:39.448360 27814 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:39.448608 27960 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:39.447587 27783 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.498129 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:39.499174 26253 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:39.471994 27740 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:39.490319 27940 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.323s	user 0.002s	sys 0.000s
I20260924 14:29:39.478719 27954 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:29:39.508751 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 296134us
W20260924 14:29:39.488386 27959 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260924 14:29:39.482463 27961 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.482725 27927 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:39.516218 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 291840us
I20260924 14:29:39.517189 27125 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:39.490700 27837 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:39.492560 27791 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:29:39.521656 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.523562 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.525600 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:39.496454 27812 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.528309 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 774) took 1357 ms. Trace:
I20260924 14:29:39.531791 26257 rpcz_store.cc:276] 0924 14:29:38.171311 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.207321 (+1036010us) service_pool.cc:227] Handling call
0924 14:29:39.221319 (+ 13998us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.221539 (+   220us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:29:39.221841 (+   302us) log.cc:842] Serialized 14 byte log entry
0924 14:29:39.528037 (+306196us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.528090 (+    53us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:39.528128 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.528174 (+    46us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.528266 (+    92us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":3598,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":4188,"thread_start_us":296202,"threads_started":1}
I20260924 14:29:39.500792 27957 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:39.534621 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 295266us
I20260924 14:29:39.530390 26253 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260924 14:29:39.536362 27125 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.511273 27960 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:29:39.513985 27787 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:39.514508 27954 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:39.553856 27711 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.554131 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:39.557034 27962 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260924 14:29:39.558790 27943 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260924 14:29:39.561754 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 770) took 1537 ms. Trace:
I20260924 14:29:39.562213 26250 rpcz_store.cc:276] 0924 14:29:38.024876 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.089962 (+1065086us) service_pool.cc:227] Handling call
0924 14:29:39.090238 (+   276us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.144916 (+ 54678us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.144943 (+    27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.144996 (+    53us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:39.145022 (+    26us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.561501 (+416479us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b54004fa954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:39.561543 (+    42us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.561685 (+   142us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":125329152}
I20260924 14:29:39.565562 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:39.566756 27186 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 198) took 2021 ms (client timeout 2230 ms). Trace:
W20260924 14:29:39.567266 27186 rpcz_store.cc:269] 0924 14:29:37.545307 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.319837 (+774530us) service_pool.cc:227] Handling call
0924 14:29:39.563821 (+1243984us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:39.566674 (+  2853us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":38356352}
I20260924 14:29:39.572317 27913 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.527799 27951 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:39.574136 26329 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:39.527360 27926 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:39.578343 27346 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:39.579660 27948 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260924 14:29:39.581307 27918 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.602849 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 573) took 2455 ms. Trace:
I20260924 14:29:39.565038 27917 raft_consensus.cc:979] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:29:39.567037 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.603399 26401 rpcz_store.cc:276] 0924 14:29:37.147534 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.538503 (+2390969us) service_pool.cc:227] Handling call
0924 14:29:39.602309 (+ 63806us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.602479 (+   170us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.602503 (+    24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.602542 (+    39us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:39.602569 (+    27us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.602619 (+    50us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.602768 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:39.542585 27812 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.541522 27502 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:39.544529 27965 raft_consensus.cc:3032] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:39.548786 27791 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.524678 27927 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.601034 27951 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:39.529884 27837 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:39.608563 27787 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:39.647516 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 342935us
I20260924 14:29:39.518972 27961 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.637342 27965 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:39.675758 27850 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:39.645121 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:39.692255 27185 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:39.694252 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:39.694661 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.695230 27185 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:39.696494 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:39.697490 27185 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:39.698469 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:39.701150 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 155752us
I20260924 14:29:39.329649 26255 rpcz_store.cc:276] 0924 14:29:37.497438 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.135141 (+637703us) service_pool.cc:227] Handling call
0924 14:29:38.135459 (+   318us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:38.135646 (+   187us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:29:38.805504 (+669858us) log.cc:842] Serialized 14 byte log entry
0924 14:29:39.169290 (+363786us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.205319 (+ 36029us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:39.209756 (+  4437us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.215914 (+  6158us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.224319 (+  8405us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":7976,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":48,"raft.queue_time_us":166423,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":8737,"thread_start_us":992843,"threads_started":2,"wal-append.queue_time_us":491674}
I20260924 14:29:39.547528 27957 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:39.729174 27957 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:39.729256 27814 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.667420 27787 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.730660 27194 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:39.709460 27813 consensus_queue.cc:1046] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:39.710040 27743 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.732523 27743 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.732851 27813 raft_consensus.cc:3032] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:39.733462 27813 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:39.734164 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 154) took 2254 ms. Trace:
I20260924 14:29:39.734440 27813 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:39.710969 27967 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260924 14:29:39.708235 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.712278 27961 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.712899 27837 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:39.671643 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 167) took 1572 ms. Trace:
W20260924 14:29:39.738837 27939 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.939s	user 0.002s	sys 0.000s
I20260924 14:29:39.645642 27791 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:39.713960 27904 raft_consensus.cc:1067] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:29:39.700829 27836 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.715384 27922 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:39.683094 27718 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.716058 27952 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:39.717686 27965 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.699136 27185 raft_consensus.cc:2362] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4. Re-sending same reply.
I20260924 14:29:39.720357 27945 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:29:39.724584 27928 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.927s	user 0.004s	sys 0.000s
I20260924 14:29:39.726662 27975 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.710227 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.734608 27743 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.731320 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 277) took 1215 ms. Trace:
I20260924 14:29:39.754424 27975 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.736469 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:39.756038 26254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 164) took 1743 ms (client timeout 2018 ms). Trace:
W20260924 14:29:39.756407 26254 rpcz_store.cc:269] 0924 14:29:38.015577 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.017610 (+  2033us) service_pool.cc:227] Handling call
0924 14:29:39.736288 (+1718678us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:39.755975 (+ 19687us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8535296}
I20260924 14:29:39.756879 27975 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.737067 27899 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.755023 27194 rpcz_store.cc:276] 0924 14:29:38.515520 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.987520 (+472000us) service_pool.cc:227] Handling call
0924 14:29:39.729993 (+742473us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:39.730223 (+   230us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.731068 (+   845us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.731268 (+   200us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13185792}
I20260924 14:29:39.757826 27733 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4.
W20260924 14:29:39.759523 27893 consensus_meta.cc:287] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.007s	user 0.002s	sys 0.014s
I20260924 14:29:39.758646 27814 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.760003 27893 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:39.760669 27188 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:39.761786 27182 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:29:39.762212 27182 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:39.638685 27812 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.764307 27812 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:39.764784 27847 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:39.759019 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:39.765950 27194 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:39.766625 27819 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:39.767637 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.744199 27791 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.773916 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 160) took 2289 ms. Trace:
I20260924 14:29:39.774410 26251 rpcz_store.cc:276] 0924 14:29:37.484663 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.133436 (+648773us) service_pool.cc:227] Handling call
0924 14:29:38.133690 (+   254us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:38.844039 (+710349us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.773683 (+929644us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54004c9d54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:39.773861 (+   178us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1632273152}
I20260924 14:29:39.743834 27978 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260924 14:29:39.744016 27922 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:39.703832 27914 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.745750 27927 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:39.795424 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 107315us
I20260924 14:29:39.745893 27954 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.746768 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:39.690728 27779 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.739147 26398 rpcz_store.cc:276] 0924 14:29:38.099603 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.336414 (+236811us) service_pool.cc:227] Handling call
0924 14:29:38.342166 (+  5752us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:38.420633 (+ 78467us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:29:39.134718 (+714085us) log.cc:842] Serialized 14 byte log entry
0924 14:29:39.635040 (+500322us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.639796 (+  4756us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:39.640756 (+   960us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.661323 (+ 20567us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.666371 (+  5048us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":37984,"prepare.queue_time_us":232869,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":37706,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":20586,"thread_start_us":959552,"threads_started":2,"wal-append.queue_time_us":578125}
I20260924 14:29:39.751701 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 147) took 2548 ms. Trace:
I20260924 14:29:39.734584 26252 rpcz_store.cc:276] 0924 14:29:37.479554 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.518949 (+ 39395us) service_pool.cc:227] Handling call
0924 14:29:37.519330 (+   381us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:37.804573 (+285243us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:29:38.281491 (+476918us) log.cc:842] Serialized 14 byte log entry
0924 14:29:39.137979 (+856488us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.140342 (+  2363us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:39.140403 (+    61us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.733929 (+593526us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b540002bf54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:39.733982 (+    53us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.734105 (+   123us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13221,"prepare.queue_time_us":307660,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":113712,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":64119,"spinlock_wait_cycles":534167808,"thread_start_us":1038007,"threads_started":2,"wal-append.queue_time_us":1020760}
I20260924 14:29:39.753515 27787 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:39.776117 26324 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:39.769537 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.772899 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 166) took 1323 ms. Trace:
I20260924 14:29:39.813046 26256 rpcz_store.cc:276] 0924 14:29:38.449139 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.244520 (+795381us) service_pool.cc:227] Handling call
0924 14:29:39.311854 (+ 67334us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:39.318654 (+  6800us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.318698 (+    44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:39.319229 (+   531us) log.cc:842] Serialized 26 byte log entry
0924 14:29:39.683716 (+364487us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:39.707503 (+ 23787us) log.cc:842] Serialized 14 byte log entry
0924 14:29:39.708334 (+   831us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.708548 (+   214us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:39.772651 (+ 64103us) raft_consensus.cc:1671] finished
0924 14:29:39.772677 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.772832 (+   155us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6157,"prepare.queue_time_us":23029,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":44632,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":9240,"thread_start_us":320837,"threads_started":1,"wal-append.queue_time_us":440874}
W20260924 14:29:39.772904 27930 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.964s	user 0.001s	sys 0.001s
I20260924 14:29:39.768029 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.768746 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:39.789865 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 249953us
I20260924 14:29:39.791500 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260924 14:29:39.768976 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 232706us
I20260924 14:29:39.771855 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:39.798714 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 104138us
I20260924 14:29:39.829527 26369 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:39.800230 26295 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:39.770643 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.804378 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 773) took 1666 ms. Trace:
I20260924 14:29:39.830822 26258 rpcz_store.cc:276] 0924 14:29:38.138190 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.125397 (+987207us) service_pool.cc:227] Handling call
0924 14:29:39.125805 (+   408us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.126061 (+   256us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:29:39.138167 (+ 12106us) log.cc:842] Serialized 14 byte log entry
0924 14:29:39.804027 (+665860us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.804080 (+    53us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:39.804127 (+    47us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.804203 (+    76us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.804320 (+   117us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":72613,"prepare.queue_time_us":16238,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":104874,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":15310,"thread_start_us":592293,"threads_started":2,"wal-append.queue_time_us":398845}
I20260924 14:29:39.773746 27819 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.839815 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 152) took 3393 ms. Trace:
I20260924 14:29:39.808419 27186 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:39.802368 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.843113 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.803887 26249 rpcz_store.cc:276] 0924 14:29:37.202833 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.204349 (+  1516us) service_pool.cc:227] Handling call
0924 14:29:38.799273 (+1594924us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:38.826520 (+ 27247us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.751471 (+924951us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b5400716450. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:39.751634 (+   163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3637547392}
I20260924 14:29:39.811308 27954 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:39.801994 27927 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:39.821007 26398 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:39.822790 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.824841 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.827672 27126 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.833343 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:39.834614 27125 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.835592 27893 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.835675 26400 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:39.836028 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.849399 26399 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:39.839464 27609 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:39.839095 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:39.859314 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:39.808957 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:39.863356 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.864209 26251 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:29:39.864991 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:39.839542 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 443440us
I20260924 14:29:39.768474 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:39.840231 26402 rpcz_store.cc:276] 0924 14:29:36.446387 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:36.848710 (+402323us) service_pool.cc:227] Handling call
0924 14:29:36.848998 (+   288us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:29:36.891214 (+ 42216us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:36.891259 (+    45us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:29:36.891965 (+   706us) log.cc:842] Serialized 52 byte log entry
0924 14:29:38.076406 (+1184441us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b1800210940. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:29:38.640879 (+564473us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:38.640922 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:38.641052 (+   130us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:39.765804 (+1124752us) spinlock_profiling.cc:243] Waited 6.35 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:29:39.839592 (+ 73788us) raft_consensus.cc:1671] finished
0924 14:29:39.839618 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.839758 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":24525,"raft.queue_time_us":50088,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5172,"spinlock_wait_cycles":18623744,"thread_start_us":436323,"threads_started":1,"wal-append.queue_time_us":1850006}
I20260924 14:29:39.837553 27922 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.842334 27847 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:39.858209 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:39.867591 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:39.868273 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:39.864498 27978 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:39.873018 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.874090 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.874277 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.874835 26405 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:39.871129 26221 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:39.875689 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.876834 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.858254 27483 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:39.869088 27609 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:39.946842 27609 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:39.878026 27992 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:39.951548 27992 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:39.954078 27992 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:39.878077 27990 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:39.905690 27986 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:39.878093 27787 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:39.959822 27787 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.857733 27893 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.963385 27893 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:39.877612 27983 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:39.964614 27819 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:39.908015 27126 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.878504 27818 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260924 14:29:39.872566 27978 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:39.965643 27978 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:39.965976 27818 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:39.879732 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.869292 27847 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:39.874154 27954 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:39.857692 27927 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:39.911264 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.863432 26252 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:39.881402 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:39.885219 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:39.928423 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 281) took 1281 ms. Trace:
I20260924 14:29:39.929924 27523 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:39.893167 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:39.996028 26404 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:39.893388 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:39.893621 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.002233 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.966404 27818 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:40.003758 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.893667 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:40.004859 27927 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:40.004181 27818 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:40.005774 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:40.006181 27818 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:40.006625 27787 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:39.991976 27893 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:39.894236 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 258) took 2439 ms. Trace:
I20260924 14:29:39.896152 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.895717 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.897662 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:39.975790 27992 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:40.011399 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 113816us
I20260924 14:29:40.012424 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.896785 26328 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:39.900380 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.903797 26403 raft_consensus.cc:2379] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:29:39.905385 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47327us
I20260924 14:29:39.894546 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:39.898458 27125 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:39.894120 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:39.941350 27813 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:39.893262 27779 consensus_queue.cc:1046] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260924 14:29:39.964315 27983 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:39.878499 27914 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260924 14:29:39.976397 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 68494.8us
I20260924 14:29:39.993162 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:39.994022 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 108931us
I20260924 14:29:40.030220 26325 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:40.030692 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.010242 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 147) took 1394 ms. Trace:
I20260924 14:29:40.034708 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.035431 26399 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:39.991225 27954 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:39.991361 27847 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:39.995276 27191 rpcz_store.cc:276] 0924 14:29:38.646540 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.088572 (+442032us) service_pool.cc:227] Handling call
0924 14:29:39.088986 (+   414us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.142257 (+ 53271us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.142291 (+    34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.142335 (+    44us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:39.142367 (+    32us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.928137 (+785770us) spinlock_profiling.cc:243] Waited 53 ms on lock 0x7b5400074c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:39.928194 (+    57us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.928351 (+   157us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":121947520}
I20260924 14:29:39.999181 27523 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:40.007068 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:39.895790 27182 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.015774 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:39.995764 27819 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:40.023190 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:40.066628 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 43529.4us
I20260924 14:29:40.067519 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:40.026787 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 128435us
I20260924 14:29:39.996670 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:40.068318 27194 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:40.069368 27126 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:40.033919 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.033030 27891 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:40.052011 26327 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:40.056519 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.009313 27192 rpcz_store.cc:276] 0924 14:29:37.455435 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.457638 (+  2203us) service_pool.cc:227] Handling call
0924 14:29:39.846486 (+2388848us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:39.859552 (+ 13066us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.877767 (+ 18215us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:39.893857 (+ 16090us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:39.893917 (+    60us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:39.893953 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.894014 (+    61us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:39.894139 (+   125us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3237120}
W20260924 14:29:40.047598 27795 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.244s	user 0.004s	sys 0.049s
I20260924 14:29:40.070634 26405 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:40.078114 26399 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:40.014691 27990 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:40.073127 26325 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:40.031208 27956 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:40.028467 27186 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:40.093607 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 212) took 1331 ms. Trace:
I20260924 14:29:40.094164 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.094650 27925 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:40.039005 27954 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:40.095958 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.097584 27740 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:40.098508 27740 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:29:40.099684 27973 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:40.100392 27973 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:40.100695 27973 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:40.101058 27973 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:40.101799 27951 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:40.102355 27951 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.094038 27189 rpcz_store.cc:276] 0924 14:29:38.762671 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.434659 (+671988us) service_pool.cc:227] Handling call
0924 14:29:39.435109 (+   450us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.435316 (+   207us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:29:39.860675 (+425359us) log.cc:842] Serialized 14 byte log entry
0924 14:29:40.093170 (+232495us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:40.093229 (+    59us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:40.093279 (+    50us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:40.093363 (+    84us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:40.093543 (+   180us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":42912,"prepare.queue_time_us":35880,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":4195,"raft.queue_time_us":47164,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":572855,"threads_started":2}
I20260924 14:29:40.084532 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.055331 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 275) took 2550 ms. Trace:
I20260924 14:29:40.099140 26080 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 4 to 6, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
W20260924 14:29:40.079936 27795 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.276s	user 0.004s	sys 0.049s
I20260924 14:29:40.089021 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.089210 27913 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:40.134831 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:40.143994 26326 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:40.145834 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.091771 27609 raft_consensus.cc:2741] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260924 14:29:40.092193 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.035095 27188 rpcz_store.cc:276] 0924 14:29:38.615724 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.091144 (+475420us) service_pool.cc:227] Handling call
0924 14:29:39.091635 (+   491us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.761048 (+669413us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:39.994423 (+233375us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54003101d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:40.009658 (+ 15235us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1538136448}
I20260924 14:29:40.147979 27779 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.148375 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:40.148556 27779 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.093429 27182 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:40.151078 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:50086 (request call id 145) took 1576 ms. Trace:
I20260924 14:29:40.151654 27182 rpcz_store.cc:276] 0924 14:29:38.614931 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.076432 (+461501us) service_pool.cc:227] Handling call
0924 14:29:40.092068 (+1015636us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54003101d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b9345da7 00007f09be291ee0 00007f09b7983784 00007f09b7983712 00007f09b7983641 00007f09b79835bc 00007f09b7982872 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:40.150963 (+ 58895us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1274284288}
I20260924 14:29:40.152537 27502 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:40.153213 27502 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:40.154423 27502 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:40.102694 27951 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:40.104955 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:40.037416 27927 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:40.037405 27984 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:40.018775 27992 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.114756 27989 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:40.087992 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:40.117219 26080 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 1 to 2, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:40.121095 26224 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.072793 26249 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:55136
I20260924 14:29:40.135555 27954 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.136523 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:40.114143 27190 rpcz_store.cc:276] 0924 14:29:37.504739 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:38.438634 (+933895us) service_pool.cc:227] Handling call
0924 14:29:39.824153 (+1385519us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:39.824373 (+   220us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.826400 (+  2027us) raft_consensus.cc:1520] Early marking committed up to index 2
0924 14:29:39.847965 (+ 21565us) log.cc:842] Serialized 14 byte log entry
0924 14:29:40.025276 (+177311us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:40.034599 (+  9323us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:40.036935 (+  2336us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:40.037031 (+    96us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:40.050827 (+ 13796us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44596,"prepare.queue_time_us":12554,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":5483,"raft.queue_time_us":11176,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6502,"spinlock_wait_cycles":62385536,"thread_start_us":161821,"threads_started":1,"wal-append.queue_time_us":215452}
I20260924 14:29:40.167241 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.140312 27523 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:40.146387 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.094501 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:40.081173 27922 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:40.159714 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.073669 27914 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:40.072770 27791 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:29:40.162091 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 69960.6us
I20260924 14:29:40.212425 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.214167 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:40.162849 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.215710 27765 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:40.204994 27372 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:40.216233 27765 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.216722 27765 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:40.216380 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 53675.2us
I20260924 14:29:40.219132 27711 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:40.219914 27711 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:40.084282 27819 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:40.228016 27819 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:40.174618 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 69744.8us
I20260924 14:29:40.178617 26402 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:46098
I20260924 14:29:40.230813 26402 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:29:40.231696 26402 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:40.178551 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:40.107683 27949 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.091s
I20260924 14:29:40.159816 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:40.188380 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 67383.7us
I20260924 14:29:40.199695 27189 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:40.151298 27779 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:40.171285 27913 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:40.191054 26080 catalog_manager.cc:5495] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 1 to 3, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:29:40.262023 27913 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.262538 27913 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:40.262730 26080 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 4, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 4 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:40.265108 26080 catalog_manager.cc:5495] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 1 to 3, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:29:40.186127 27850 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:40.197886 27991 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:40.224288 26253 raft_consensus.cc:1226] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260924 14:29:40.267721 27991 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.267570 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:40.268285 27991 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:40.223661 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.204164 27502 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:40.216539 27372 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:40.232193 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.233661 26399 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:40.090050 27965 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:40.278856 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46757.3us
I20260924 14:29:40.241986 27194 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:40.245172 27916 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:40.247165 27995 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260924 14:29:40.248250 27954 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:40.248530 27983 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:40.230703 27483 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:40.232436 26249 raft_consensus.cc:595] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:40.209772 27922 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:40.257201 27978 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.258386 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:40.260044 27992 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:40.260433 27819 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:40.260037 26224 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:40.221046 27711 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:40.270498 26080 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 1 to 3, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 3 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
W20260924 14:29:40.272676 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 176679us
W20260924 14:29:40.273384 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50088.3us
I20260924 14:29:40.293218 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.293238 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:40.274003 26402 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:40.267787 27850 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:40.289266 26249 raft_consensus.cc:603] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:29:40.285890 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:40.279069 27791 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:40.281373 27523 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:40.303369 27523 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.303858 27523 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:40.304859 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.283980 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 160) took 2728 ms. Trace:
I20260924 14:29:40.307677 26400 rpcz_store.cc:276] 0924 14:29:37.555380 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:37.561847 (+  6467us) service_pool.cc:227] Handling call
0924 14:29:39.749273 (+2187426us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:39.749599 (+   326us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:39.858181 (+108582us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:40.283715 (+425534us) spinlock_profiling.cc:243] Waited 76.1 ms on lock 0x7b5400477054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:40.283917 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":182859392}
I20260924 14:29:40.298539 27483 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:40.309160 27765 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:40.310408 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.311726 26329 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:29:40.301005 27372 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:40.301397 27954 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:40.308859 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:40.309592 27483 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.297572 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:40.313621 26402 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:40.300033 27949 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:40.293350 26080 catalog_manager.cc:5495] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:29:40.314847 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:40.281322 27779 consensus_queue.cc:1046] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260924 14:29:40.328603 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:40.329723 26330 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:40.326592 26080 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 4, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 4 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:40.284610 27696 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260924 14:29:40.323999 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.335860 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.327224 27194 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:40.292016 26326 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:40.339794 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:40.297472 27791 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.315938 27956 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:40.394320 27194 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
W20260924 14:29:40.396951 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57266.6us
I20260924 14:29:40.321305 27954 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.288435 27916 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:40.396211 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.333379 27483 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:40.331770 27990 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:40.404287 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.428139 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:40.341291 27968 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 14:29:40.442626 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 46523.9us
I20260924 14:29:40.341578 27992 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:40.445585 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.450071 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.446693 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:40.451109 26325 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:40.451598 26325 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:40.452677 26325 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:40.341449 27765 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:40.460667 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.334808 27733 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.099s
I20260924 14:29:40.335566 27982 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.098s
I20260924 14:29:40.404150 27812 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:40.487788 27812 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:40.488371 27812 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:40.490161 27991 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:40.461977 27819 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:40.408500 27954 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:40.399057 27787 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:40.400660 27897 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:40.426702 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.462271 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.484341 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.485054 27765 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:40.376377 27791 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:40.431936 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.458909 27986 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:40.476508 27990 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:40.465641 27893 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:40.400470 27956 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.499972 27732 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:40.505790 27927 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:40.536916 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 87229.2us
I20260924 14:29:40.408871 27850 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.493374 27951 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:40.542070 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 95233.2us
I20260924 14:29:40.514640 27819 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:40.512794 27847 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:40.554852 27732 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:40.567782 27711 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:40.568310 27926 raft_consensus.cc:3032] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:40.564021 26400 raft_consensus.cc:2396] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260924 14:29:40.573175 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.573980 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:40.575021 27991 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:40.575387 27951 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:40.576499 27916 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:40.578020 27779 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.578832 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.553201 27893 raft_consensus.cc:686] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:40.548997 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:40.580035 27953 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:40.580856 26406 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:40.580741 27913 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:40.514521 27787 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:40.584823 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:40.585203 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.553826 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:40.666180 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 112424us
I20260924 14:29:40.667111 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:40.582346 27813 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.001s
I20260924 14:29:40.520548 27897 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:40.583534 27978 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:40.591514 27922 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:40.622543 27926 raft_consensus.cc:729] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:40.549855 27956 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:40.668617 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.586519 27523 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.678894 27483 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.730978 27523 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:40.587071 27927 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:40.731197 27483 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:40.732988 27927 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:40.732784 27893 consensus_queue.cc:237] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:40.676507 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 91390.4us
I20260924 14:29:40.738133 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:40.725288 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 43692.6us
W20260924 14:29:40.727166 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 58580.3us
I20260924 14:29:40.741190 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.728591 27123 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.743896 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:40.725840 27787 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:40.679144 26397 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:40.750782 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:40.668340 27765 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:40.770207 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.809795 27194 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:40.681257 27991 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:40.661082 27779 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.723780 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.684137 27570 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:40.724411 27913 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:40.847996 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:40.848763 27189 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:40.848942 27779 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:40.849720 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.850375 26250 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:40.850621 27897 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:40.851667 27570 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.851311 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:40.854460 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:40.834678 26258 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:40.856029 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.772049 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.772590 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.857048 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.826711 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.724433 27847 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:40.858494 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 238515us
I20260924 14:29:40.739809 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.860136 26326 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:29:40.771235 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.861697 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:40.862365 26251 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:40.862914 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.863301 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.865712 26251 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:40.830883 27194 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:40.667670 27953 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:40.679630 27819 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.837245 27791 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:40.839865 26400 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:40.840456 27926 consensus_queue.cc:260] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:40.807879 27733 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:40.845103 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.881280 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.881917 26255 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:40.882958 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.884238 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.889468 27818 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:40.890116 27818 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:40.893127 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 597) took 1050 ms. Trace:
W20260924 14:29:40.893752 27973 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.792s	user 0.007s	sys 0.004s
I20260924 14:29:40.893697 26406 rpcz_store.cc:276] 0924 14:29:39.842676 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.842842 (+   166us) service_pool.cc:227] Handling call
0924 14:29:40.892860 (+1050018us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54001e46d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:40.893012 (+   152us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1694951808}
I20260924 14:29:40.894348 27973 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.895608 27186 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:40.685163 28008 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:40.868142 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.901484 26253 raft_consensus.cc:2379] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:40.902453 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.748278 28009 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:40.871165 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:40.811231 27990 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:40.820907 27914 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:40.891255 27818 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.882977 27953 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:40.911753 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:40.915308 26250 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:40.825003 27813 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:40.916795 27813 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:40.917461 27791 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:40.919035 27813 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:40.829140 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:40.828847 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:40.920622 26329 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:40.899741 26401 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260924 14:29:40.922619 27818 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:40.904568 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:40.924261 26250 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:40.924391 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.909853 27570 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:40.925787 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.926532 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.874451 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.903698 26332 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:40.927842 27779 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:40.869931 26403 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:40.915069 27954 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.928634 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:40.930263 27954 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:40.930464 27814 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:40.930958 27814 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:40.911062 27812 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:40.932097 27814 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:40.932161 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.932693 26251 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:40.869782 27960 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:40.933542 27812 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:40.933725 27960 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:40.934320 27960 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:40.913910 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.885413 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.939253 26251 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:40.933763 26256 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:40.941177 26401 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:40.930223 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:40.942123 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 70810.3us
W20260924 14:29:40.913457 27502 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260924 14:29:40.914083 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.953639 26332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:40.948792 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.955598 27954 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:40.956188 27954 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:40.957105 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:40.957116 27954 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.948570 27812 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:40.961311 27812 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:40.961755 27812 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:40.962023 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.963590 27813 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:40.963637 27916 raft_consensus.cc:2741] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260924 14:29:40.965018 27916 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:40.966686 27916 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:29:40.968107 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:40.968930 27194 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:40.970677 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:40.971580 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.984022 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.978066 27904 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:40.985836 26399 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:40.986353 27904 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:40.987679 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:40.987285 27904 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:40.986021 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:40.999711 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:40.980697 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:40.983739 27992 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:41.005625 26256 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260924 14:29:41.004626 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:41.010444 26256 pending_rounds.cc:76] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 1
I20260924 14:29:40.999783 26255 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:41.010783 26256 pending_rounds.cc:97] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:41.014132 27779 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:41.016265 26256 raft_consensus.cc:2866] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:41.017432 27732 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:41.022902 26405 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:41.027426 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.028370 27194 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:41.036775 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:41.041818 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:41.029345 28009 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260924 14:29:41.046628 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:41.046965 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:41.048148 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:41.049268 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.049383 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:41.050334 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:41.050472 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.051642 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:41.051895 27989 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:41.052289 27989 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:41.052623 27989 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.053056 28008 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:41.053582 27992 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:41.046628 26256 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:41.054023 27992 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:41.054507 27992 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.055176 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:41.059760 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.060921 27850 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:41.063457 27973 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:41.066805 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:41.067772 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.009495 27570 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:41.072641 28017 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:41.062211 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.080013 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.081645 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.074046 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.089576 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.037029 27904 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:41.091924 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:41.099858 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:41.084192 27850 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:41.083145 28008 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:41.091289 27914 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:41.122473 26398 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:41.094331 27819 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:41.102584 27732 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:41.096601 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.134231 27819 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:41.142202 27126 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:29:41.144076 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47708.5us
I20260924 14:29:41.088008 27960 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:41.146994 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.149717 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.145134 27819 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:41.149729 27570 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:41.152171 27570 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:41.151037 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.154886 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.154857 27194 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:41.156994 27787 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:41.138299 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.104286 27956 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:41.151141 27954 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:41.160199 28008 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:41.156512 27927 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:41.153137 27847 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:41.179399 26253 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:41.165719 27523 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260924 14:29:41.172515 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 180) took 1140 ms. Trace:
I20260924 14:29:41.182894 26404 rpcz_store.cc:276] 0924 14:29:40.032075 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:40.032819 (+   744us) service_pool.cc:227] Handling call
0924 14:29:40.050696 (+ 17877us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:40.085636 (+ 34940us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:40.085682 (+    46us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:40.085741 (+    59us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:40.085784 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:41.172221 (+1086437us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b5400450454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:41.172284 (+    63us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:41.172437 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":79746432}
I20260924 14:29:41.186630 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.188541 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.194425 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.195338 27188 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:41.196523 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:41.197132 27188 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:41.198356 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.177835 27191 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:41.199102 27189 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:41.199330 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 159) took 1285 ms. Trace:
I20260924 14:29:41.199779 27191 rpcz_store.cc:276] 0924 14:29:39.914112 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:40.071768 (+157656us) service_pool.cc:227] Handling call
0924 14:29:41.138073 (+1066305us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63eeb7c 00007f09b63ee73b 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:41.199257 (+ 61184us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":14524800}
I20260924 14:29:41.200914 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.201679 27192 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:41.202463 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.202670 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.171691 27570 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:41.189322 27194 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
W20260924 14:29:41.224859 26327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 112) took 1455 ms (client timeout 1687 ms). Trace:
I20260924 14:29:41.225302 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.226327 27194 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:41.227393 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.207995 26325 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:41.229132 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:41.230083 27194 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:41.203497 27927 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:41.225467 26327 rpcz_store.cc:269] 0924 14:29:39.769863 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.771544 (+  1681us) service_pool.cc:227] Handling call
0924 14:29:41.224636 (+1453092us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b5400237654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:41.224792 (+   156us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":484322688}
I20260924 14:29:41.173352 27954 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:41.246472 27194 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:41.249010 26402 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:29:41.177999 28008 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:41.201581 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:41.255410 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 94844.2us
I20260924 14:29:41.256464 28008 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:41.209453 27716 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:41.227922 27779 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:41.261881 26404 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.257359 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.204365 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.205228 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:41.272294 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.273604 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:41.274773 27188 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:41.200291 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.277192 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.278285 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:41.268306 27194 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:41.279258 27189 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:41.264318 27897 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:41.282370 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:41.282478 27897 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:41.283119 26257 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 2: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:41.264300 27716 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:41.284154 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 594) took 1460 ms. Trace:
I20260924 14:29:41.284010 27716 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:41.284631 27897 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:41.284590 26397 rpcz_store.cc:276] 0924 14:29:39.824184 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.824300 (+   116us) service_pool.cc:227] Handling call
0924 14:29:41.283963 (+1459663us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b5400476154. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:41.284094 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1806417792}
I20260924 14:29:41.285254 27716 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.287386 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:41.255115 27954 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.203415 27192 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:41.204815 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.284314 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.267047 26325 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:41.270298 27779 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:41.291584 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.279174 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.303587 26404 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:41.301736 27192 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:41.305856 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.262952 27813 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260924 14:29:41.310141 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.310169 27192 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:41.313948 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:41.315135 26258 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:41.300845 27182 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:41.317718 27182 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260924 14:29:41.299399 27847 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:41.322878 27192 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:41.336670 26221 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.338505 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.340399 27192 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:41.340672 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.341820 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.308813 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:41.343408 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.308650 27191 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:29:41.327859 28040 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:41.309657 28036 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:29:41.342659 27914 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:41.367725 27914 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:41.368140 27914 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:41.369176 27732 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:41.371596 27989 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:41.373682 27989 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:41.376547 27927 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:41.383574 27956 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:41.383996 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 334687us
I20260924 14:29:41.389217 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.343000 28017 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:41.347259 27913 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:41.313187 27960 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:41.347896 27990 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:41.300776 27814 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:41.303385 27992 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:41.358012 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:41.423650 26404 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:41.358913 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:41.361210 27185 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:41.313287 27893 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260924 14:29:41.403638 27570 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:41.352742 27765 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:41.423997 26369 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:41.405087 27591 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:41.415170 27191 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:41.434648 27126 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:41.429179 26399 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:41.349404 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.428274 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:41.432021 26332 raft_consensus.cc:595] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:29:41.432690 26252 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:41.436560 26332 raft_consensus.cc:603] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:29:41.410799 27913 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:41.415269 28042 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:41.441102 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 91803.1us
I20260924 14:29:41.439246 26327 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:41.446651 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:41.415802 27968 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:41.447368 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.416218 27990 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:41.448810 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.454905 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.456043 27992 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:41.456995 27185 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:41.458802 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:41.459606 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 55124.5us
I20260924 14:29:41.459725 27989 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:41.460548 27814 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:41.461697 28009 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:41.445469 28021 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:41.437959 27765 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:29:41.462356 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 53392.7us
I20260924 14:29:41.443315 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:41.472178 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.472285 28048 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:41.445308 27913 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.490007 27914 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:41.477298 28039 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.160s	user 0.002s	sys 0.000s
W20260924 14:29:41.478554 27186 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 162) took 1404 ms (client timeout 1777 ms). Trace:
I20260924 14:29:41.483434 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.516656 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.517715 27346 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:41.518204 26295 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:41.518478 27346 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260924 14:29:41.485153 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62338.9us
I20260924 14:29:41.485742 27185 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.445983 27733 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:41.491014 27765 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:41.443142 27954 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:41.488197 27989 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:41.465301 27990 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:41.465170 27791 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:41.476697 28049 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:29:41.470189 27926 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:41.519893 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:41.485996 26323 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:41.445825 27960 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:41.485502 28021 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:41.499131 28051 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260924 14:29:41.593072 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 73270.3us
I20260924 14:29:41.597213 27897 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:29:41.597708 27897 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:41.598670 27954 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:41.599331 27591 consensus_queue.cc:1046] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:41.599970 27787 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:41.600196 28008 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:41.602349 28008 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:41.602408 27822 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:41.602888 27822 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:29:41.519882 27186 rpcz_store.cc:269] 0924 14:29:40.073994 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:40.094197 (+ 20203us) service_pool.cc:227] Handling call
0924 14:29:41.478312 (+1384115us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54004533d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:41.478463 (+   151us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1594023680}
I20260924 14:29:41.576164 27904 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
I20260924 14:29:41.604547 27191 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:41.499526 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:29:41.613729 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 114375us
I20260924 14:29:41.551082 27346 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:41.615599 27346 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:41.615924 27346 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:41.616715 27346 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:41.617206 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.617805 27346 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:41.466090 27968 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:41.525978 27986 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:41.631755 27986 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:41.632194 27986 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:41.632017 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:41.632902 26324 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:41.465898 28042 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:41.603673 27822 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:41.636626 27822 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:41.637610 28042 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:41.642073 27914 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:41.620020 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:41.583649 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 149103us
I20260924 14:29:41.621074 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.580611 28011 consensus_queue.cc:1046] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260924 14:29:41.598743 27897 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:41.597296 27716 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:41.623142 27989 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:41.623027 27902 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:41.624909 27960 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:41.659113 26250 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:41.625188 27191 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:41.628484 28006 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:41.592888 28021 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:41.666698 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:41.657755 28035 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260924 14:29:41.650517 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:29:41.654729 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 57674.2us
I20260924 14:29:41.664939 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:41.647652 27914 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:41.663787 28042 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260924 14:29:41.680475 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 60540.7us
I20260924 14:29:41.700956 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:41.643509 27812 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.681s	user 0.011s	sys 0.003s
I20260924 14:29:41.630218 26329 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:41.686946 27192 raft_consensus.cc:2396] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:29:41.591653 28009 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:29:41.680176 28041 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260924 14:29:41.590360 27893 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:41.696025 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:41.643595 27986 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:41.575333 27916 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260924 14:29:41.662906 27716 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:41.645761 27968 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:41.726528 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:41.646929 27953 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260924 14:29:41.710598 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.686081 27969 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:41.713599 26405 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:41.699417 27914 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:41.708495 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:41.704226 28042 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:41.729538 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:41.697822 28006 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:41.740969 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.706391 28049 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:41.699239 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 216) took 2203 ms. Trace:
I20260924 14:29:41.723547 27954 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260924 14:29:41.727023 27893 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:41.758913 26398 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:41.761030 26404 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:41.726495 28055 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:41.804654 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 63840.8us
I20260924 14:29:41.805632 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.766188 27795 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:41.769706 28048 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:41.708618 27812 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:41.767587 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 59779us
W20260924 14:29:41.775563 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 74700.3us
I20260924 14:29:41.697904 27591 consensus_queue.cc:1046] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260924 14:29:41.701722 27731 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260924 14:29:41.768549 26323 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:41.784214 27914 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.771821 27979 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:41.819633 27185 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:41.842527 27791 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:41.843008 27791 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:41.842849 28006 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:41.843912 27791 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:41.844921 27791 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.852246 27716 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:41.853982 27765 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:41.856196 26254 raft_consensus.cc:2379] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:41.857288 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:41.859177 28008 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:41.859613 27182 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:41.859843 28008 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:41.831334 28011 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260924 14:29:41.860980 28008 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:41.824571 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:41.821645 27893 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:41.826946 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:41.791317 28042 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:29:41.790428 27819 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.632s	user 0.012s	sys 0.000s
W20260924 14:29:41.793213 28045 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260924 14:29:41.835901 27812 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:41.836288 27986 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:41.836653 26401 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
W20260924 14:29:41.864167 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 49681.6us
I20260924 14:29:41.812837 27787 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:41.813849 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:41.877159 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:41.870752 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57388.9us
I20260924 14:29:41.813030 27732 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:41.882261 26253 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:55136
I20260924 14:29:41.882719 27732 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:41.882925 26253 raft_consensus.cc:595] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:41.883148 27732 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.883325 26253 raft_consensus.cc:603] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:29:41.794623 27181 rpcz_store.cc:276] 0924 14:29:39.494261 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.834988 (+340727us) service_pool.cc:227] Handling call
0924 14:29:39.835402 (+   414us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:40.724549 (+889147us) raft_consensus.cc:1520] Early marking committed up to index 1
0924 14:29:40.724584 (+    35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:40.724632 (+    48us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:40.724670 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:41.681631 (+956961us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b540000d054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:41.682438 (+   807us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:41.697094 (+ 14656us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":15788,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":20678,"spinlock_wait_cycles":1950798720}
I20260924 14:29:41.771521 28009 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:41.873685 27819 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:41.809870 27795 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:41.893561 27190 raft_consensus.cc:2396] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260924 14:29:41.894763 26405 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:41.874341 27765 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:41.798310 27779 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:41.874362 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:41.765630 27960 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:41.898877 27779 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:41.904134 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:41.868468 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 125461us
I20260924 14:29:41.792768 28049 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:41.832999 28040 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:41.881466 27372 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:41.906121 28049 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:41.916878 27926 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260924 14:29:41.917232 27787 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260924 14:29:41.917994 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:41.936254 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:41.813527 28055 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:41.894095 27795 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:41.943904 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:41.950934 27346 raft_consensus.cc:3032] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:41.951501 27346 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:41.952435 27346 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:41.954429 26332 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:41.928894 27822 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260924 14:29:41.933218 27731 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260924 14:29:41.961278 27346 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:41.963408 28052 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:41.965943 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:41.965955 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:41.967973 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:41.967904 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:41.968686 26254 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:41.970427 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:41.938035 27893 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260924 14:29:41.973148 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:41.939177 27917 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260924 14:29:41.954662 26323 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:41.957825 27905 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:42.019166 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:42.023631 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:42.024031 28062 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260924 14:29:41.981992 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:41.945780 27913 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:41.992313 27732 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:42.034531 26253 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:41.997180 26330 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:42.008975 27819 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:42.033903 27990 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:42.017493 28052 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:42.047147 27732 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:42.048349 27990 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:42.061434 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:42.029008 27905 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:42.066262 26254 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
W20260924 14:29:42.084697 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 101757us
I20260924 14:29:42.088478 27922 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:42.089136 27922 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:42.092427 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:42.093822 26401 raft_consensus.cc:2396] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260924 14:29:42.072227 27483 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:42.130321 27483 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:42.132148 27787 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:42.132654 27483 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:42.134531 28069 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260924 14:29:42.135221 27913 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:42.136343 26404 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
W20260924 14:29:42.137895 28065 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260924 14:29:42.144001 26297 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:42.145011 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:42.147624 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:42.148461 26327 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:42.150715 28055 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:42.147364 27813 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:42.154683 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 788) took 1311 ms. Trace:
I20260924 14:29:42.156886 26402 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:42.158836 27901 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:42.160970 27979 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:42.162402 27897 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:42.151044 27591 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:42.159374 27901 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:42.174687 26258 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:42.155139 26250 rpcz_store.cc:276] 0924 14:29:40.842856 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:40.927209 (+ 84353us) service_pool.cc:227] Handling call
0924 14:29:41.658881 (+731672us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:41.659724 (+   843us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:42.154471 (+494747us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b54002882d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:42.154619 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1681863424}
I20260924 14:29:42.174968 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:42.196518 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:42.207571 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 234510us
I20260924 14:29:42.223196 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:42.227492 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:42.229796 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:42.205627 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:42.235227 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:42.236975 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:42.237746 26405 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:42.239302 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:42.241933 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:42.242892 27190 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:42.243986 27819 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:42.231885 27787 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:42.244138 28008 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:42.253008 27989 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:42.275743 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:42.232147 28055 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260924 14:29:42.261243 27893 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:42.282954 26250 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:42.244426 27523 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260924 14:29:42.286295 28055 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:42.286876 28055 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:42.287084 28055 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:42.288046 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:42.287856 28055 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:42.288096 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:42.291674 26257 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:42.291934 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:42.292541 26405 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:42.293187 27732 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:42.294276 27779 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:29:42.270679 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:42.296185 27190 raft_consensus.cc:2396] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260924 14:29:42.118853 27922 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:42.294935 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:42.297276 27818 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:42.297616 27922 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:42.297835 27818 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:42.300685 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:42.302132 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:42.303054 26403 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:42.306941 27126 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:42.167006 27901 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:42.311939 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:42.318964 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 413256us
I20260924 14:29:42.322038 26295 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:42.323748 26405 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:42.324602 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:42.325521 27905 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:42.325670 27189 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:42.325769 27818 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:42.326661 27901 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:42.263408 27850 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260924 14:29:42.334725 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:42.335481 26250 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:42.327786 27926 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:42.336625 27926 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:42.336977 27926 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:42.298552 27732 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:42.328795 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:42.340600 26325 raft_consensus.cc:2396] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:29:42.299492 27893 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:42.310683 28052 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:42.306463 27483 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:42.312057 27989 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:42.355151 27189 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260924 14:29:42.391922 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 51266.6us
I20260924 14:29:42.399722 26329 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260924 14:29:42.402882 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 71608.3us
W20260924 14:29:42.405748 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 92008.2us
I20260924 14:29:42.402927 27812 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
W20260924 14:29:42.408215 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 86276.4us
I20260924 14:29:42.409132 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:42.409816 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:42.410295 26331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:42.410910 26331 raft_consensus.cc:595] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:42.413587 28008 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:42.423620 26404 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:42.407166 27591 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260924 14:29:42.408435 27986 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:42.445808 26404 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:54910
I20260924 14:29:42.446868 26404 raft_consensus.cc:595] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:42.447499 26404 raft_consensus.cc:603] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:29:42.450202 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:29:42.455488 27989 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:42.460093 27716 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:42.469635 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:42.470410 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61520.9us
W20260924 14:29:42.364653 27914 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.012s
I20260924 14:29:42.348734 27893 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:29:42.480762 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 83621.1us
I20260924 14:29:42.489799 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:42.511687 26254 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:42.512167 26257 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:42.515637 26256 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
W20260924 14:29:42.520610 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 213766us
I20260924 14:29:42.343043 27523 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.139s
I20260924 14:29:42.342638 28042 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:42.355855 27732 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.165s
W20260924 14:29:42.567698 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 48196.1us
I20260924 14:29:42.522820 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:42.391834 27483 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:42.477804 28061 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.089s
I20260924 14:29:42.568836 26250 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:29:42.481540 27905 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.088s
I20260924 14:29:42.609302 26401 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:42.527194 27913 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:42.528681 28068 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:42.492955 27914 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:42.566650 27899 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:42.625809 26397 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:42.625800 26332 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260924 14:29:42.643821 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 122925us
I20260924 14:29:42.602658 28079 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:42.622759 28006 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:42.626976 26398 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5. Ops: []
W20260924 14:29:42.661620 28021 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.964s	user 0.013s	sys 0.000s
I20260924 14:29:42.629871 27904 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260924 14:29:42.642621 27913 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:42.696930 27346 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:42.697809 27346 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:42.698745 27346 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:42.700191 27346 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:42.686064 28021 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:42.683768 27765 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:42.648295 28068 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:42.722709 28076 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260924 14:29:42.705296 28078 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.100s	user 0.001s	sys 0.001s
W20260924 14:29:42.713991 28067 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260924 14:29:42.662763 28079 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:42.665956 28006 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:42.711282 28021 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:42.735507 28064 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.357s	user 0.000s	sys 0.002s
I20260924 14:29:42.733681 28042 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:42.744267 27978 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:42.885517 27978 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:42.799590 26253 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:42.740173 27913 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:42.801427 26258 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260924 14:29:42.803421 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 192668us
I20260924 14:29:42.890450 27570 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:42.890893 27914 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:42.892184 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:42.893378 26398 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:42.809435 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:29:42.894547 27779 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:42.895363 27779 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:42.899008 27978 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:42.901015 27978 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:42.814693 26324 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:42.902868 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 471) took 1199 ms. Trace:
I20260924 14:29:42.903301 26324 rpcz_store.cc:276] 0924 14:29:41.703161 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:41.706237 (+  3076us) service_pool.cc:227] Handling call
0924 14:29:42.813568 (+1107331us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:42.814125 (+   557us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:42.902511 (+ 88386us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:42.902754 (+   243us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":44419840}
I20260924 14:29:42.829636 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 627) took 1117 ms. Trace:
I20260924 14:29:42.905794 26397 rpcz_store.cc:276] 0924 14:29:41.712121 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:41.717488 (+  5367us) service_pool.cc:227] Handling call
0924 14:29:42.625432 (+907944us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:42.625633 (+   201us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:42.661716 (+ 36083us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:42.829231 (+167515us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9560960}
I20260924 14:29:42.844797 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 241) took 1138 ms. Trace:
I20260924 14:29:42.854689 27190 pending_rounds.cc:76] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 1
I20260924 14:29:42.856163 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:42.859112 26327 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.013s	sys 0.020s
I20260924 14:29:42.864218 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:42.873782 27483 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:42.712029 27765 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:42.743078 27972 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:42.763393 28068 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:42.933727 27972 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:29:42.762055 28071 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260924 14:29:42.936193 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 418709us
W20260924 14:29:42.764444 28081 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260924 14:29:42.763597 28084 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260924 14:29:42.914119 26327 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.013s	sys 0.020s
I20260924 14:29:42.913292 27570 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:42.944490 27979 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:42.945174 27979 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:42.908092 27182 rpcz_store.cc:276] 0924 14:29:41.705742 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:41.717697 (+ 11955us) service_pool.cc:227] Handling call
0924 14:29:41.851910 (+134213us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:41.860057 (+  8147us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:42.844246 (+984189us) spinlock_profiling.cc:243] Waited 83.6 ms on lock 0x7b54002cae50. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:42.844552 (+   306us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":192375552}
I20260924 14:29:42.953191 27779 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:42.766487 28079 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:42.962397 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 126813us
I20260924 14:29:42.966984 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:42.969815 27795 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:42.937611 28009 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:42.971150 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:42.971908 28009 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:42.908217 27190 pending_rounds.cc:97] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:29:42.947017 27570 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:42.935813 27972 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:43.000815 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.004058 27902 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:42.945196 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.004695 27765 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.019593 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:42.935241 28006 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:42.933130 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:29:43.048024 28088 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:43.050051 28055 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:42.990839 27190 raft_consensus.cc:2866] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:29:43.059108 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 79630.1us
I20260924 14:29:43.051571 27779 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:43.068151 27779 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:43.068836 27813 consensus_meta.cc:287] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.917s	user 0.006s	sys 0.015s
I20260924 14:29:43.069377 27813 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 14:29:43.069604 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 103209us
I20260924 14:29:43.050083 27570 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.076632 28009 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:43.078063 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 133136us
I20260924 14:29:43.078508 27913 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:43.051138 27126 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.084867 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.087663 27913 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260924 14:29:43.085058 27926 consensus_meta.cc:287] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.016s
I20260924 14:29:43.091109 27926 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:43.092800 28049 consensus_meta.cc:287] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.154s	user 0.017s	sys 0.002s
I20260924 14:29:43.053395 27502 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:42.947023 27979 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.078262 27902 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.099627 27185 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
W20260924 14:29:43.119510 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 43487.1us
I20260924 14:29:43.119997 26397 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:43.099830 27813 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.093417 28049 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.114248 26079 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 3, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:29:43.116374 27972 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:43.115893 27502 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.127034 28009 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:43.130782 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 123128us
I20260924 14:29:43.109489 27779 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.129303 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.111347 27926 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.169677 28090 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:43.171938 26252 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 1. Current term is 2. Ops: []
W20260924 14:29:43.173046 27897 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.010s	user 0.017s	sys 0.009s
I20260924 14:29:43.173674 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.179761 27989 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.180779 27989 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.129190 27914 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.182364 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.182204 27779 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:43.203419 27904 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.203586 27791 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260924 14:29:43.204793 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 153932us
I20260924 14:29:43.207803 28079 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:43.209008 28079 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:43.209405 28079 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:43.209800 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:43.203137 27779 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.211597 27914 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.212347 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.213305 26254 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:43.213685 27779 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:43.214984 27502 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:43.131929 27913 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260924 14:29:43.213685 27914 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.216809 28009 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:43.217448 27502 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.217995 27502 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:43.218680 27901 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.891s	user 0.014s	sys 0.006s
I20260924 14:29:43.219026 28009 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:43.219619 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.207121 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:43.131057 27570 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.210597 28079 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.228307 26258 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:43.165530 28088 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.201253 26079 catalog_manager.cc:5495] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:43.244938 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.222426 27502 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.186993 27813 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.260281 27926 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:43.263296 27822 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.204237 27904 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.267403 26401 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:43.257382 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.243134 26397 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:43.273056 27926 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.306670 27926 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:43.309559 26405 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
W20260924 14:29:43.312177 28100 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260924 14:29:43.299525 27904 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.300366 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.277637 27972 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:43.301682 26080 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 3) took 7741 ms. Trace:
I20260924 14:29:43.303023 26079 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 3, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:29:43.291725 27822 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.304561 27765 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:43.304637 27893 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.010s
I20260924 14:29:43.305312 27570 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.305895 27812 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.305341 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.315032 26224 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:43.324275 27972 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.326484 26400 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:43.318617 28049 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:43.327850 26079 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:43.328723 27979 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.382453 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.383649 26250 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:43.385627 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.287189 28079 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.324331 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:43.312282 28098 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260924 14:29:43.346518 26079 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 3, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 3 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:43.350238 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:43.388767 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 73828.9us
I20260924 14:29:43.352567 26326 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:43.352967 27822 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:43.357908 27609 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.391767 27609 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.320354 26080 rpcz_store.cc:276] 0924 14:29:35.566990 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:35.568024 (+  1034us) service_pool.cc:227] Handling call
0924 14:29:43.301457 (+7733433us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:29:40.667296 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:29:40.832314 (+165018us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:29:40.842225 (+  9911us) write_op.cc:435] Acquired schema lock
0924 14:29:40.842253 (+    28us) tablet.cc:660] Decoding operations
0924 14:29:40.843480 (+  1227us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:29:40.843552 (+    72us) write_op.cc:641] Partition lock acquired for write op
0924 14:29:40.843590 (+    38us) tablet.cc:683] Acquiring locks for 8 operations
0924 14:29:40.844450 (+   860us) tablet.cc:699] Row locks acquired
0924 14:29:40.844481 (+    31us) write_op.cc:260] PREPARE: finished
0924 14:29:40.844712 (+   231us) write_op.cc:270] Start()
0924 14:29:40.844905 (+   193us) write_op.cc:276] Timestamp: P: 1790260180844624 usec, L: 0
0924 14:29:40.844933 (+    28us) op_driver.cc:331] REPLICATION: starting
0924 14:29:40.937037 (+ 92104us) log.cc:842] Serialized 3281 byte log entry
0924 14:29:41.268996 (+331959us) op_driver.cc:444] REPLICATION: finished
0924 14:29:41.487558 (+218562us) write_op.cc:301] APPLY: starting
0924 14:29:41.491725 (+  4167us) tablet.cc:1365] starting BulkCheckPresence
0924 14:29:41.509129 (+ 17404us) tablet.cc:1368] finished BulkCheckPresence
0924 14:29:41.509270 (+   141us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:29:41.636632 (+127362us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:29:41.636841 (+   209us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0924 14:29:41.636897 (+    56us) write_op.cc:312] APPLY: finished
0924 14:29:41.637207 (+   310us) log.cc:842] Serialized 64 byte log entry
0924 14:29:41.637449 (+   242us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:41.637574 (+   125us) write_op.cc:454] Released schema lock
0924 14:29:41.637631 (+    57us) write_op.cc:341] FINISH: Updating metrics
0924 14:29:41.637937 (+   306us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:41.637969 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41633,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":149242,"apply.run_cpu_time_us":4333,"apply.run_wall_time_us":157039,"num_ops":8,"prepare.queue_time_us":203284,"prepare.run_cpu_time_us":18335,"prepare.run_wall_time_us":426351,"raft.queue_time_us":106273,"raft.run_cpu_time_us":14541,"raft.run_wall_time_us":155758,"replication_time_us":417064,"spinlock_wait_cycles":31488,"thread_start_us":321680,"threads_started":4,"wal-append.queue_time_us":184660,"wal-append.run_cpu_time_us":21359,"wal-append.run_wall_time_us":1512854}]]}
I20260924 14:29:43.358198 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.359671 27980 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.360482 27813 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.363070 26324 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:43.364614 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.365325 27812 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.366623 27989 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:43.367969 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.369402 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.378945 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:43.380705 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.343147 27902 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.389271 27182 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:43.388832 27913 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:43.389882 26405 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.351295 27893 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.397082 26404 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:29:43.355695 27972 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:43.396627 27922 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.098s	user 0.010s	sys 0.019s
I20260924 14:29:43.409636 26079 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:43.406632 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.409417 27185 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:43.410435 26258 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:43.413910 26252 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.415643 26250 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
W20260924 14:29:43.416538 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44193.6us
I20260924 14:29:43.420930 27972 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:43.420445 27609 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.426918 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.428992 27893 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.430826 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 248) took 1192 ms. Trace:
I20260924 14:29:43.411054 27913 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.435560 28040 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.436208 28040 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:43.432308 27181 rpcz_store.cc:276] 0924 14:29:42.237969 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:42.244309 (+  6340us) service_pool.cc:227] Handling call
0924 14:29:42.840202 (+595893us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:42.840910 (+   708us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:29:42.842002 (+  1092us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:42.842055 (+    53us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:29:42.924061 (+ 82006us) log.cc:842] Serialized 52 byte log entry
0924 14:29:43.184305 (+260244us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:43.405252 (+220947us) log.cc:842] Serialized 14 byte log entry
0924 14:29:43.429437 (+ 24185us) log.cc:842] Serialized 14 byte log entry
0924 14:29:43.430249 (+   812us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:43.430356 (+   107us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:43.430507 (+   151us) raft_consensus.cc:1671] finished
0924 14:29:43.430520 (+    13us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:43.430670 (+   150us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":91,"prepare.queue_time_us":36113,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":7685,"raft.queue_time_us":55275,"raft.run_cpu_time_us":633,"raft.run_wall_time_us":4043,"spinlock_wait_cycles":5977472,"thread_start_us":239982,"threads_started":2,"wal-append.queue_time_us":368460}
I20260924 14:29:43.433118 27913 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:43.444666 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.445322 28088 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:43.446102 27989 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:43.446211 26405 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:43.453263 26079 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
W20260924 14:29:43.457198 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 69019.7us
W20260924 14:29:43.456979 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 87787.7us
I20260924 14:29:43.460245 26079 catalog_manager.cc:5495] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:43.465498 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.465031 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.486721 26398 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:43.487790 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.488770 26402 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:43.490653 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.491773 26254 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:43.496529 27978 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260924 14:29:43.498035 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.498633 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.499080 26328 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:43.500375 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.447088 27980 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.491232 27812 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:43.506307 26332 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:43.506296 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.508164 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:43.508379 26331 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:43.498132 28070 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.509169 28070 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.510512 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.510753 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:43.512226 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.512506 26328 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:29:43.492377 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.516096 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:43.516932 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:29:43.517419 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51252.7us
I20260924 14:29:43.518996 28088 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:29:43.519301 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.519524 28088 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.519651 27916 consensus_queue.cc:1046] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:43.520025 28088 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:29:43.520090 26398 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:43.520979 27893 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:43.521214 27372 raft_consensus.cc:3032] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:43.521332 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.521745 27372 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:43.522081 26405 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:43.498186 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.524564 26295 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:43.524843 27185 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:43.527407 27731 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:43.527794 27731 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.528124 27731 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.528622 27126 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:43.529584 27972 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:43.529778 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.530175 27972 raft_consensus.cc:686] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:43.530557 27191 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:43.531241 27972 consensus_queue.cc:237] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:43.533301 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.533999 27194 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:43.515314 26329 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:43.509912 27980 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:43.511219 28070 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.539651 26298 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:43.517586 26079 catalog_manager.cc:5495] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 3, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 3 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:43.537526 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:43.537578 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:43.542481 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.543212 26404 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:43.522717 27372 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.544479 27813 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:43.545135 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:43.519208 27986 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:43.546437 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.547984 27990 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:43.555475 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.555320 28070 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.555043 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.550521 26252 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:43.560359 27185 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:43.558842 26079 catalog_manager.cc:5495] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 4, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:43.555393 27570 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:43.568462 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:43.538949 28056 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260924 14:29:43.561595 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:43.571362 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.557482 27986 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.558476 27904 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:43.574256 26295 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:43.577569 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:43.568684 27570 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.561259 26400 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:43.573939 27980 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:43.579201 27372 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:43.572878 27609 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260924 14:29:43.577148 27986 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.586541 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:43.561687 27989 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260924 14:29:43.594522 27609 raft_consensus.cc:3032] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:43.594951 27609 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:43.595913 27181 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:29:43.597023 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.596114 27609 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.592620 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.576711 27904 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.598948 27904 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:43.599056 27609 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:43.599287 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:43.581489 27795 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:43.600749 27795 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.601118 27795 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.563027 27990 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.605443 26328 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:43.607264 28040 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.593986 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.585120 27570 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.591230 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:43.593201 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.563287 27812 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260924 14:29:43.608742 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.594043 27573 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:43.588624 27980 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:29:43.592801 28055 consensus_meta.cc:287] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.007s
I20260924 14:29:43.612852 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.609081 26079 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 5, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 5 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:43.573993 28070 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.615024 28009 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.617681 27779 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:43.618005 27822 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:43.619628 26256 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:43.624321 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.626150 28049 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.626682 28049 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.624339 28009 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.615253 27502 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.623673 27822 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:43.629689 28009 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:43.607918 27990 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.615737 27813 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:43.630493 27822 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.612735 28090 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:43.616825 27980 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:43.637630 26079 catalog_manager.cc:5495] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:29:43.631471 28040 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:43.612912 27979 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260924 14:29:43.647236 28040 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.654592 28040 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:43.637678 27813 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.623320 27926 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:43.671798 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.647660 28009 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.668435 26404 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:29:43.641295 28055 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:43.663743 26079 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 5, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 5 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:29:43.647428 28070 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.675349 27913 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.677846 28009 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.692946 28009 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:43.674921 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:43.694536 27926 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.675957 27731 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.695829 28055 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.675246 27732 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260924 14:29:43.696640 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 127975us
I20260924 14:29:43.697216 27731 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.697656 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:43.698324 26400 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:43.679425 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.700142 27192 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:43.702587 28049 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:43.703641 26295 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:43.702085 27913 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.705268 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:43.706030 27192 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.706144 27978 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:43.706600 27978 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.706955 27978 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:43.696969 27732 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.707568 27732 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.682945 27779 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:43.665653 27813 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:43.687114 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 252) took 1031 ms. Trace:
I20260924 14:29:43.703182 28049 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:43.716168 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.717242 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:43.709154 28088 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:43.719193 27570 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.720041 26255 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:43.743703 28073 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:43.743827 28088 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.744145 28073 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:43.744522 27186 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:43.744468 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:43.745249 26255 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:43.745117 28073 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.721792 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.746488 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.746970 26327 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:43.723975 26256 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:43.748514 27822 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.748731 26256 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:43.732079 27190 rpcz_store.cc:276] 0924 14:29:42.655521 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:42.657009 (+  1488us) service_pool.cc:227] Handling call
0924 14:29:42.676484 (+ 19475us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:43.068314 (+391830us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:43.068383 (+    69us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:43.068903 (+   520us) log.cc:842] Serialized 26 byte log entry
0924 14:29:43.168693 (+ 99790us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:43.169108 (+   415us) log.cc:842] Serialized 14 byte log entry
0924 14:29:43.238529 (+ 69421us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:43.306115 (+ 67586us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540000d054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:43.306174 (+    59us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:43.686720 (+380546us) raft_consensus.cc:1671] finished
0924 14:29:43.686842 (+   122us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:43.687052 (+   210us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":63362,"prepare.queue_time_us":292,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":76,"raft.queue_time_us":65448,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":92,"spinlock_wait_cycles":407889152,"thread_start_us":96534,"threads_started":1,"wal-append.queue_time_us":433771}
I20260924 14:29:43.755642 27570 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.756565 27189 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:43.759469 26254 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:29:43.761379 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.762017 26253 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:43.763000 26257 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:43.763104 27901 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:43.725224 27795 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.764874 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.765637 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:43.765895 27922 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:43.766170 26329 raft_consensus.cc:2354] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5.
I20260924 14:29:43.748543 26252 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260924 14:29:43.766390 27922 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.708381 27502 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:43.694618 28091 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:29:43.768350 28091 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:43.730505 27893 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:43.769469 28091 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.773797 28055 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.760641 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:43.774729 26402 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:29:43.779520 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.765919 27795 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.780262 26254 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:43.781231 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.782037 26256 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:43.782912 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:43.789266 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:43.789435 27980 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.790169 27190 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 5
I20260924 14:29:43.792213 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:43.793411 28055 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:43.794202 28055 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:43.741696 28049 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.742151 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:43.742980 27185 raft_consensus.cc:2354] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:29:43.746423 28073 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:43.747354 26249 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:43.730538 27731 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:43.764247 27901 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:43.724861 27986 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.796016 27765 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:43.796813 27123 leader_election.cc:398] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5.
I20260924 14:29:43.845381 27123 leader_election.cc:401] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:29:43.799233 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:43.801211 27978 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:43.802345 27813 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.804231 27922 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.806434 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.694511 27904 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:43.810674 27980 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:43.810892 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:43.811303 26400 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:29:43.808408 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:43.811606 28040 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.814249 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:43.740586 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:43.822253 26328 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:29:43.860953 27893 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260924 14:29:43.823117 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 85141.2us
W20260924 14:29:43.861093 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 120829us
I20260924 14:29:43.861385 27893 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:43.861824 27893 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:43.846769 27901 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:43.827427 26402 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:29:43.846012 27192 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:43.843945 27731 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:43.847759 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 51141.3us
I20260924 14:29:43.873840 26295 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:43.854589 26404 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
W20260924 14:29:43.859146 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50820.7us
I20260924 14:29:43.869971 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:43.875558 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:43.851969 26255 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:29:43.868638 27189 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:43.873984 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:43.876870 26405 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:43.733111 26079 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 4, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 4 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:43.827359 27732 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:43.882314 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:43.883308 26255 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 8
I20260924 14:29:43.897284 26403 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:43.897014 26079 catalog_manager.cc:5495] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:43.910061 26079 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 5, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:29:43.911098 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:43.893651 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:43.897961 26323 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:43.888379 26330 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260924 14:29:43.922993 26406 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:43.918607 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:43.930967 26221 leader_election.cc:398] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:29:43.939688 26221 leader_election.cc:401] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:29:43.880657 27979 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:43.943315 26255 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:29:43.886194 27902 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:43.891731 27570 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:29:43.891060 28009 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:43.955003 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.927381 27502 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:29:43.958045 26079 catalog_manager.cc:5495] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 5, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:29:43.898365 27914 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260924 14:29:43.968385 26405 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:43.973189 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.003686 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:43.998914 26332 raft_consensus.cc:2396] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260924 14:29:44.013123 27192 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:43.985039 27189 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:44.001220 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:44.010164 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:43.982796 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:43.972486 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.952473 27902 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:43.956310 27573 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:29:43.963721 26397 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:44.016934 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:44.021626 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 134752us
I20260924 14:29:43.953222 27913 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:29:44.023720 27182 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 5: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:44.017658 26079 catalog_manager.cc:5495] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 4, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:29:44.024700 26403 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.023135 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:44.023300 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:44.031512 27185 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:44.023257 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.029838 26328 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:43.969416 27570 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:43.985872 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:43.976142 27904 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:44.039532 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:43.990584 27731 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:43.984467 27502 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:44.009846 27978 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:44.041790 26327 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:43.993470 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:44.035995 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:43.976373 27813 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:43.980706 28040 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:43.970438 27979 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:43.991539 27732 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:43.969080 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.036451 26405 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 6: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:44.038319 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:44.055554 26249 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:43.973472 27986 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:43.979925 27190 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:44.021447 27902 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:43.979743 28055 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:44.045125 28042 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:44.045318 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:44.044112 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:44.046418 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.047186 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.022624 28049 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:44.029630 27914 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:44.046579 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:44.039718 27573 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260924 14:29:44.041049 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:44.073195 26329 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:44.054711 27185 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.057379 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.062150 26331 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:44.047188 27913 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:43.982874 27901 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.077652 26295 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.062770 27186 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:44.084015 26401 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:44.062117 27990 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:44.086700 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.087527 27822 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:44.087786 26297 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:44.053520 27978 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260924 14:29:44.092396 26224 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:44.073809 28049 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:44.068010 27779 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5.
I20260924 14:29:44.058568 27850 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260924 14:29:44.078560 27914 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.095921 27822 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:44.110589 27914 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:44.085343 27986 consensus_queue.cc:1159] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c22052275034c369b7f7dca086fbd1e" responder_term: 4 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:29:44.109752 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.099977 27978 raft_consensus.cc:3032] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:44.129480 26329 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
W20260924 14:29:44.156466 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 64172.6us
I20260924 14:29:44.153667 26325 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:44.070521 28090 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.077s
I20260924 14:29:44.171352 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:44.153134 28073 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:44.095857 28125 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:44.112155 27978 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:44.152446 27813 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:29:44.163199 27914 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:44.164440 28055 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
W20260924 14:29:44.181316 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 94725.2us
I20260924 14:29:44.147615 27980 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260924 14:29:44.158968 27731 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260924 14:29:44.163162 27822 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.093626 27913 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.098603 27893 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.195303 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:44.100504 27990 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.174005 28073 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:44.155309 28049 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.156397 27502 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:44.210065 28125 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:44.210592 28125 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:44.211041 28125 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:44.212216 28055 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:44.212716 27978 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.213405 27990 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:44.213230 28055 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.214257 27978 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:44.207013 28073 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.167769 27979 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.225134 26403 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:44.243206 28006 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:44.257515 27182 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:44.257988 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.262462 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.265004 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:44.265972 26402 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:44.264446 27989 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260924 14:29:44.264438 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 113565us
I20260924 14:29:44.283963 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:44.103675 27570 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:44.285367 26298 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.285524 28006 consensus_queue.cc:1046] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:44.286506 28006 raft_consensus.cc:3032] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:44.286814 28006 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:44.287643 27901 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.285421 27573 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:44.288545 27573 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:44.288182 28006 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.289050 27573 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:44.290160 27922 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:44.289907 27573 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.290810 27609 raft_consensus.cc:3032] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:44.290668 27922 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.291183 27609 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:44.292068 27609 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.293442 27609 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:44.267098 28009 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:44.314227 27185 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:44.289620 28006 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.316162 27192 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260924 14:29:44.316332 26295 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.335283 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.356436 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.331655 27731 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:44.378988 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:44.345786 27972 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:44.387924 27765 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:44.390005 28049 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:44.391752 28125 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.358789 28042 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:44.381709 28009 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:44.393270 27731 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:44.399700 27765 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:29:44.422703 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 66408.3us
I20260924 14:29:44.424391 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.434864 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:44.435912 28088 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:44.436582 28088 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:44.439435 26404 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:44.448212 27893 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:44.448614 28137 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:44.427857 27765 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:44.442934 27787 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260924 14:29:44.449826 27922 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:44.452280 28125 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:44.446283 28130 raft_consensus.cc:2741] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260924 14:29:44.446499 27913 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:44.461593 26328 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:29:44.406664 27972 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:44.462154 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:44.463066 26323 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:44.463140 27902 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.463963 27972 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:44.465453 27902 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:44.466179 26327 raft_consensus.cc:2354] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6.
I20260924 14:29:44.468633 28088 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:44.468753 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:44.469825 28073 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:44.475858 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 51710.3us
I20260924 14:29:44.476722 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:44.462239 27731 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.475010 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.495244 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:44.496028 26402 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:44.496282 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:44.412079 28042 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.501595 26403 raft_consensus.cc:2354] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:44.502350 26221 leader_election.cc:398] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6.
I20260924 14:29:44.502775 26221 leader_election.cc:401] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:29:44.518615 26224 leader_election.cc:398] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:44.519055 26224 leader_election.cc:401] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260924 14:29:44.522866 26402 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:29:44.526489 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.528550 27185 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: [3.3-4.4]
I20260924 14:29:44.528751 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:44.528848 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:44.529562 26258 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:44.530830 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:44.530922 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:44.533293 27989 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:44.533406 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.533671 27812 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:44.533887 27609 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:44.534165 27812 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:44.536638 26403 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:29:44.537953 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:44.539635 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:44.539259 27812 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:44.540262 26255 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.534500 27609 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:44.504566 27972 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:44.505388 28009 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:44.551421 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:44.551965 28009 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:44.543041 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.545459 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:44.546542 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.555025 26325 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:29:44.547098 26332 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260924 14:29:44.547673 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:44.548384 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:44.549074 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:44.549363 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:44.551015 28128 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.545576 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.561333 26329 raft_consensus.cc:2396] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260924 14:29:44.556293 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:44.552958 28009 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.559113 27185 raft_consensus.cc:2396] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260924 14:29:44.562481 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:44.529469 27186 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:44.564047 28126 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 6
I20260924 14:29:44.563747 27609 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.566308 26327 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:44.503790 28088 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260924 14:29:44.567924 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.568580 26331 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:44.541728 27779 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260924 14:29:44.557108 26400 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:44.572700 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.573238 27523 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:44.574738 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.575647 27523 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:44.575815 27902 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:44.576020 27523 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:44.576310 27902 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:44.576723 27902 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.577013 27523 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.578222 27523 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.533891 27989 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.581399 27989 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:44.582134 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:44.582795 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:44.583482 26256 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:44.583658 28073 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:44.553382 28042 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:44.585578 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:44.589692 27893 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.002s
I20260924 14:29:44.592149 27502 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:44.591712 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.594197 27787 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:44.594662 27787 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:44.595026 27787 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.608263 26258 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:44.606455 26255 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:44.611263 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:44.612953 27126 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:44.618917 26257 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260924 14:29:44.621125 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.623370 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:44.621837 26325 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:44.628711 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:44.629644 26256 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:44.628811 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:44.631319 26258 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:44.624315 26250 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:44.634686 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.637591 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.639571 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.642256 28056 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:44.642693 28056 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:44.643059 28056 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:44.643589 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:44.649422 28128 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:44.650674 28056 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.654487 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:44.655170 26325 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:44.658355 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:44.671066 26325 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:44.686040 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.634824 27893 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:44.690884 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:44.701529 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.703244 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:44.635533 27978 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:44.661355 28056 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:44.716130 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:44.721283 26297 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.650674 27765 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.694875 27914 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:44.728014 27914 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:44.709898 28055 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:44.711709 28125 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:44.721860 27980 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:44.667029 27913 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260924 14:29:44.725724 27372 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:44.728487 27914 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.755836 27765 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:44.723333 28135 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:44.756836 28135 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.759124 28135 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:44.744051 28144 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:44.744649 26252 raft_consensus.cc:2354] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5.
I20260924 14:29:44.745074 28127 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:44.747675 27913 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:44.710572 27902 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.735965 28125 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:44.794307 28125 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:44.726325 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.706925 28042 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:44.798202 28042 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:44.799171 28042 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.763284 27372 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:44.768086 26254 raft_consensus.cc:2396] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260924 14:29:44.771763 27125 leader_election.cc:398] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5.
I20260924 14:29:44.804085 27125 leader_election.cc:401] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260924 14:29:44.776453 26401 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:29:44.788043 27502 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:44.801541 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 75313.3us
I20260924 14:29:44.806028 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.807590 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:44.719360 27978 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:44.735209 28141 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:44.809953 27372 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.778117 28127 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:44.811769 28006 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:44.814114 27902 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:44.830775 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:44.818321 28137 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:44.818612 28144 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:44.787557 28128 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260924 14:29:44.822467 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:44.826669 27190 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:44.827099 27926 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:44.834270 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:44.835623 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:44.836694 27609 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:44.837929 27190 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:44.793744 27913 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:44.831063 28127 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:29:44.822100 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50474.8us
I20260924 14:29:44.839771 27609 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:44.842020 28128 consensus_queue.cc:1225] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260924 14:29:44.827944 27502 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.844511 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.844791 27502 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:44.845252 26323 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:29:44.845196 27913 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.846716 27913 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:44.848242 26403 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:44.848775 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:44.849432 26323 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:44.850452 28073 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:44.851266 26398 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:44.852361 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.853127 28055 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:44.856014 27570 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:44.856488 27570 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:44.856804 27570 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:44.860795 26252 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:44.861181 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:44.861768 26257 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:44.831507 26327 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:44.832582 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:44.867317 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.870446 26253 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:44.888588 26255 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:44.839051 28125 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.878182 28128 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:44.896164 28128 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:44.896582 28128 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:44.896484 27502 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:44.926182 26257 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:44.932646 26297 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:44.881361 28073 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260924 14:29:44.949330 27893 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:44.950219 27893 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:44.920495 27813 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:44.930254 26249 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:44.930343 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:44.920393 27795 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260924 14:29:44.893639 27787 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:44.976298 26332 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:29:44.969034 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:29:44.973568 27893 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:44.974639 26402 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:29:44.992329 26405 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:44.956276 27979 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:45.006361 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:45.012022 26401 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:44.990721 26252 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:45.035145 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:45.035771 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:44.971688 27813 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:45.036039 26329 raft_consensus.cc:2354] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6.
I20260924 14:29:45.021633 26327 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:29:45.038594 27185 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:45.041301 26221 leader_election.cc:398] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6.
I20260924 14:29:45.041642 26221 leader_election.cc:401] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260924 14:29:45.045152 28146 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260924 14:29:45.046519 28152 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260924 14:29:45.072098 28125 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:45.017318 28009 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260924 14:29:45.024950 28079 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260924 14:29:45.078372 28151 log.cc:924] Time spent T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Append to log took a long time: real 0.317s	user 0.004s	sys 0.000s
I20260924 14:29:45.028213 27978 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:45.033115 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:45.083104 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:45.106591 27192 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:45.036471 27979 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:45.128453 26405 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260924 14:29:45.132601 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 134337us
W20260924 14:29:45.136399 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 84453.4us
I20260924 14:29:45.141181 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:45.127962 27731 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:45.050570 28128 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.142910 27787 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:45.178787 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 187036us
I20260924 14:29:45.104411 28070 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260924 14:29:45.138815 26323 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:45.081774 27914 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:45.185576 26250 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:29:45.187366 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:45.188299 26253 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:45.189428 26254 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:29:45.144624 27978 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260924 14:29:45.193591 27123 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:45.173991 27731 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.185655 27922 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:45.203018 27979 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:45.209302 27731 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:45.208094 26329 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:45.210791 26329 raft_consensus.cc:595] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:45.211192 26329 raft_consensus.cc:603] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:29:45.212357 28128 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:45.212507 27893 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:45.212992 27893 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:45.213333 27893 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:45.204524 27926 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5.
I20260924 14:29:45.214476 26323 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:45.215272 26323 raft_consensus.cc:595] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:45.208112 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:45.221331 27185 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:45.222965 27182 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:29:45.224763 26400 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:45.229053 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:45.230079 26329 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:45.237344 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:45.252115 28168 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:45.252780 28168 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.255046 28168 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:45.256182 28157 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260924 14:29:45.257937 26295 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:45.259034 27978 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:45.259407 27978 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:45.259804 27978 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:29:45.260470 28165 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260924 14:29:45.263149 28164 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260924 14:29:45.264262 28169 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:45.264741 28169 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:45.260110 28127 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:45.242987 28126 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.678s	user 0.010s	sys 0.007s
I20260924 14:29:45.265060 28169 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:45.275971 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:45.282133 26329 raft_consensus.cc:2379] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:45.286391 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:45.289358 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:45.220134 27922 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.290246 26398 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
W20260924 14:29:45.297881 28154 log.cc:924] Time spent T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.265s	user 0.004s	sys 0.003s
I20260924 14:29:45.303086 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:45.303602 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:45.304586 26252 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:45.290769 27813 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:45.306694 27914 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:45.293859 28144 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:45.267172 26323 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:45.317239 26323 raft_consensus.cc:595] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:45.317426 27914 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:45.317699 26323 raft_consensus.cc:603] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:29:45.318099 27914 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:45.318885 26257 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260924 14:29:45.319322 26257 pending_rounds.cc:76] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 3
I20260924 14:29:45.319692 26257 pending_rounds.cc:97] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260924 14:29:45.320015 26257 raft_consensus.cc:2866] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:45.319027 27914 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.316174 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:45.323655 27523 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:45.319852 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:45.274765 28168 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260924 14:29:45.329293 26325 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:45.327934 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:45.335765 27185 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:45.337030 26329 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:45.315172 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:45.332437 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:45.288116 28126 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6.
I20260924 14:29:45.278352 28091 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260924 14:29:45.380807 26251 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:29:45.360026 27185 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:29:45.309110 28144 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.320016 28127 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:45.390192 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:45.399806 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47369.9us
W20260924 14:29:45.400034 28163 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260924 14:29:45.408577 26325 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:29:45.355465 26405 raft_consensus.cc:2354] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5.
I20260924 14:29:45.418359 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:45.407825 27189 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:45.425293 26255 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:45.428301 26224 leader_election.cc:398] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5.
I20260924 14:29:45.428726 26224 leader_election.cc:401] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260924 14:29:45.429477 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:45.430143 26253 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:45.344686 27570 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.402387 26249 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
W20260924 14:29:45.433697 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 146442us
I20260924 14:29:45.407707 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:45.462867 26325 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:45.463587 26325 raft_consensus.cc:595] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:45.478644 26253 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:45.482223 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:45.443610 28049 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6.
I20260924 14:29:45.461946 27181 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:45.463471 28009 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:45.482198 28090 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:45.487707 28137 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5.
I20260924 14:29:45.473256 28006 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260924 14:29:45.476625 27609 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260924 14:29:45.482810 27978 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.503919 28009 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:29:45.543311 28178 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260924 14:29:45.543275 28009 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.517730 28176 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:45.517264 28159 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:45.548651 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:45.550423 26253 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:45.569195 27570 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:45.570873 27978 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:45.572404 28174 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260924 14:29:45.580428 28144 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:45.520884 27893 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.583781 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 834) took 1063 ms. Trace:
I20260924 14:29:45.584569 26252 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:45.585577 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:45.586480 26258 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 2: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:45.510852 28090 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.584352 26251 rpcz_store.cc:276] 0924 14:29:44.520589 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:44.521253 (+   664us) service_pool.cc:227] Handling call
0924 14:29:45.583516 (+1062263us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b5400782dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:45.583680 (+   164us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1952549760}
W20260924 14:29:45.616577 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 183990us
I20260924 14:29:45.640380 27893 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:45.655184 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 223679us
I20260924 14:29:45.634284 26253 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
W20260924 14:29:45.697459 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 262006us
I20260924 14:29:45.685899 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:45.700973 26255 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260924 14:29:45.701385 26255 pending_rounds.cc:76] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 2
I20260924 14:29:45.701642 26255 pending_rounds.cc:97] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260924 14:29:45.701926 26255 raft_consensus.cc:2866] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:29:45.704255 27125 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:45.708181 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:45.708902 27192 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:45.711120 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:45.557022 28176 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.720856 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:45.721704 26329 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:45.703593 27893 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:45.707746 28042 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:45.711463 27570 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:45.744776 26405 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:45.743290 27192 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
W20260924 14:29:45.754851 28192 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260924 14:29:45.756129 27978 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:45.737920 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:45.756665 27978 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:45.765758 26249 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:29:45.735433 27893 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:45.806191 28169 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.507s	user 0.013s	sys 0.000s
I20260924 14:29:45.851887 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:45.852813 26253 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:45.814082 27978 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:45.858450 27980 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:45.853754 27191 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:45.861744 27893 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:45.866752 28176 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:45.847872 28169 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:45.869663 28169 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:45.870087 28040 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:45.870648 28040 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:45.872805 28040 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:45.898088 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:45.901675 28042 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:29:45.908156 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 197089us
I20260924 14:29:45.910194 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:45.911091 26329 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:45.911284 26295 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:45.899128 26257 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:45.916376 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:45.917119 27192 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:45.892132 28090 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:45.933974 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:45.940013 27902 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:45.940800 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:45.942370 28196 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:45.943020 28196 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.946535 28185 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:45.947130 28185 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:45.947799 28185 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:45.948833 26401 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:29:45.948009 28042 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:45.942921 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 702) took 1357 ms. Trace:
I20260924 14:29:45.956143 26400 rpcz_store.cc:276] 0924 14:29:44.585727 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:44.589808 (+  4081us) service_pool.cc:227] Handling call
0924 14:29:45.942537 (+1352729us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400123ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:45.942739 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":269442816}
W20260924 14:29:45.961661 28191 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.391s	user 0.003s	sys 0.000s
I20260924 14:29:45.968961 27922 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:45.973644 28155 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.901s	user 0.003s	sys 0.000s
I20260924 14:29:45.976078 27902 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:45.977166 27902 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:45.979287 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:45.979794 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:45.980472 26331 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:45.981324 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:45.904369 27181 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:45.981904 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:45.982488 26254 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:45.984077 28159 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:45.984903 27189 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:45.988113 26253 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:45.989432 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:45.990171 26251 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:45.903795 28189 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:46.002985 26254 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:46.005869 26329 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:46.007933 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:46.014259 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 261601us
I20260924 14:29:46.012257 26253 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:46.017941 26329 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:46.025601 28009 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:46.027745 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 373) took 1186 ms. Trace:
I20260924 14:29:46.027917 27570 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:46.028908 27570 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:46.030557 26329 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:46.029363 26255 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:46.031790 26329 raft_consensus.cc:595] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:29:46.009305 27922 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:46.036779 28167 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
W20260924 14:29:46.037565 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 137496us
I20260924 14:29:46.042740 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:46.043913 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:46.044636 27182 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
W20260924 14:29:46.046238 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 64659.4us
I20260924 14:29:46.016901 27904 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260924 14:29:46.064021 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:46.050740 26332 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:46.016098 28189 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:46.068988 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:46.052122 27913 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:46.098117 26327 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:46.098325 27182 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:46.073021 28176 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:46.042326 27922 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:46.107333 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 66536.4us
W20260924 14:29:46.104624 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53031.2us
W20260924 14:29:46.108434 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47181.5us
W20260924 14:29:46.075966 28125 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.007s	sys 0.025s
W20260924 14:29:46.112231 28125 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.007s	sys 0.026s
I20260924 14:29:46.116122 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:46.075237 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:46.117856 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:46.098598 28196 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:46.068526 28101 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260924 14:29:46.137288 26332 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:46.152431 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:46.154803 27123 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:46.107903 28176 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:46.184089 28176 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:46.184948 28194 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:46.185618 28194 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:46.188422 28185 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:46.028281 27186 rpcz_store.cc:276] 0924 14:29:44.841718 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:44.842302 (+   584us) service_pool.cc:227] Handling call
0924 14:29:44.842668 (+   366us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:29:44.894236 (+ 51568us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:44.902885 (+  8649us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:29:45.420196 (+517311us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b2c001c06e8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:29:45.492282 (+ 72086us) log.cc:842] Serialized 52 byte log entry
0924 14:29:45.548388 (+ 56106us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:45.713601 (+165213us) log.cc:842] Serialized 14 byte log entry
0924 14:29:45.960842 (+247241us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:29:46.027185 (+ 66343us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b180016c480. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09b6261990 00007f09b6423864 00007f09b6423155 00007f09b93463d0 00007f09b9358576 00007f09b93690d7 00007f09b9369058
0924 14:29:46.027384 (+   199us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:46.027494 (+   110us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:46.027545 (+    51us) raft_consensus.cc:1671] finished
0924 14:29:46.027556 (+    11us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:46.027691 (+   135us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":58652,"prepare.queue_time_us":155881,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":375327,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":200,"spinlock_wait_cycles":84082688,"thread_start_us":126824,"threads_started":2,"wal-append.queue_time_us":71083}
I20260924 14:29:46.190522 28217 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:46.191182 28217 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:46.193264 28185 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:46.184108 28186 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:46.196070 28186 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:46.197009 28186 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:46.199054 28009 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:46.165331 28168 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:46.201761 28168 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:46.202173 28168 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:46.203778 28217 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:46.205159 28194 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:46.207970 27922 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:46.132299 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:46.208966 26324 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:46.119400 28127 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260924 14:29:46.166457 27779 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:46.211490 27779 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:46.211831 27779 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:46.123956 28220 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:46.212296 28220 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:46.212630 28220 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:46.141690 26397 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:29:46.216508 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46867.3us
I20260924 14:29:46.182695 27190 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:46.232380 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:46.233237 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 51274.6us
I20260924 14:29:46.123956 28180 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.204s
I20260924 14:29:46.237932 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:46.219177 28130 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260924 14:29:46.240276 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:46.241082 26403 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:46.240010 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:46.242057 26401 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:46.253446 26331 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:46.256764 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:46.258898 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:46.262229 26255 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:46.263638 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:46.272203 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 103654us
I20260924 14:29:46.254302 28189 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:46.238626 28184 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260924 14:29:46.283391 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:46.285102 27182 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:46.283581 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:46.294508 28126 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:46.294684 27181 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:29:46.275628 27978 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:46.296737 26328 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:46.296868 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:46.297888 27978 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:46.287709 28194 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:46.302193 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:46.302846 26331 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:46.304440 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:46.305163 27185 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:46.307859 28144 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260924 14:29:46.283388 28189 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:46.287563 27902 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:46.317976 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:46.256278 28212 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260924 14:29:46.303210 28042 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:46.332167 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:46.336319 27523 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:46.337361 28214 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.486s	user 0.002s	sys 0.000s
I20260924 14:29:46.336988 27523 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:46.338120 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:46.318735 28166 raft_consensus.cc:979] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: : Instructing follower 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:46.300261 28194 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:46.357316 26331 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:29:46.327849 27902 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:46.359481 27372 raft_consensus.cc:2741] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260924 14:29:46.360248 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:46.324287 28189 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:46.341950 28042 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:46.359481 28166 raft_consensus.cc:1067] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Signalling peer 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:46.362759 27186 raft_consensus.cc:2354] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6.
I20260924 14:29:46.367549 28189 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:46.379258 27185 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:29:46.373423 26328 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:46.364885 27902 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:46.386773 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 67472.3us
I20260924 14:29:46.361845 28127 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:46.405571 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:46.409519 28127 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:46.411769 28042 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:46.409958 28219 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:29:46.386155 27523 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:46.401553 27893 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:46.402899 28217 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:46.428421 28219 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:46.441483 27893 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:46.448729 28233 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260924 14:29:46.459074 27779 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:46.464658 28176 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:46.479907 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:46.466311 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106137us
I20260924 14:29:46.475706 27893 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:46.482061 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:46.428893 26397 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:46.443421 28127 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:46.491909 27185 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:46.500715 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:46.501675 28194 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:46.505131 27779 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:46.509117 28168 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:46.445920 28217 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:46.511348 28219 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:46.517513 28176 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:46.518620 27182 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:46.507040 28220 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:46.539860 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:46.540009 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:46.542626 26251 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:46.546705 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 142880us
W20260924 14:29:46.548373 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 230483us
I20260924 14:29:46.526944 28126 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:46.551970 27126 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:46.555250 26295 leader_election.cc:398] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6.
I20260924 14:29:46.555706 26295 leader_election.cc:401] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260924 14:29:46.543668 28130 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:29:46.553478 27185 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:50480
I20260924 14:29:46.560249 27185 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:29:46.560629 27185 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:46.561249 28217 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:46.569361 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:46.570271 27188 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:46.576355 28220 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:46.576246 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:46.575248 27902 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:46.579752 28168 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:46.580065 28188 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6.
I20260924 14:29:46.556947 28126 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:46.581218 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:46.582020 26397 raft_consensus.cc:2354] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6.
I20260924 14:29:46.580080 27902 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:46.556761 28186 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:46.582777 28126 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:46.583148 28186 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:46.583606 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:46.584044 28186 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:46.585670 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:46.588173 27902 raft_consensus.cc:979] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: : Instructing follower 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:46.588862 28188 raft_consensus.cc:1067] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Signalling peer 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:46.592237 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:46.598577 26256 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:46.599502 27194 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:50480
I20260924 14:29:46.600275 27194 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:29:46.600615 27194 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:46.589926 27185 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:46.603680 26253 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260924 14:29:46.603292 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 219087us
I20260924 14:29:46.606837 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:46.615970 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:46.616671 26405 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:46.622036 26253 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:29:46.614293 27185 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:46.608829 28189 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:46.625452 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:46.626219 28168 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:29:46.627370 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:46.628069 27181 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:46.630359 26298 leader_election.cc:398] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6.
W20260924 14:29:46.632676 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 78484us
I20260924 14:29:46.631227 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:46.634464 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:46.662487 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:46.680167 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47291.3us
I20260924 14:29:46.683117 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:46.683149 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:46.684202 26331 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:46.593293 28202 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260924 14:29:46.687047 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:46.693181 26251 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:46.697221 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:46.694880 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:46.702157 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:46.702433 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:46.702987 26254 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:46.704169 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:46.706282 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 75389.8us
I20260924 14:29:46.700572 26405 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:46.625484 28189 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:46.711848 26255 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:46.712891 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:46.711350 27126 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:46.717562 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:46.718384 26398 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:46.722270 26257 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:46.610476 27570 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.094s
W20260924 14:29:46.726208 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 177637us
I20260924 14:29:46.716842 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:46.720268 26254 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:29:46.725644 27186 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:46.735762 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:46.716360 27972 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:46.766924 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:46.776975 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 71646.9us
I20260924 14:29:46.783895 26369 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:46.762030 28198 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.620s	user 0.002s	sys 0.000s
I20260924 14:29:46.751624 28091 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:46.853662 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 275) took 1370 ms. Trace:
I20260924 14:29:46.761548 27523 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:46.751458 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:46.849274 27972 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:46.872406 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:46.854094 26406 rpcz_store.cc:276] 0924 14:29:45.483704 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:45.483873 (+   169us) service_pool.cc:227] Handling call
0924 14:29:45.493493 (+  9620us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:45.504178 (+ 10685us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:45.504209 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:45.511983 (+  7774us) log.cc:842] Serialized 26 byte log entry
0924 14:29:45.730041 (+218058us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:45.730100 (+    59us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:45.730188 (+    88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:46.628847 (+898659us) spinlock_profiling.cc:243] Waited 8.19 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:29:46.853339 (+224492us) raft_consensus.cc:1671] finished
0924 14:29:46.853369 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:46.853595 (+   226us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":6955,"spinlock_wait_cycles":18831232,"thread_start_us":185135,"threads_started":1,"wal-append.queue_time_us":331264}
I20260924 14:29:46.883852 26330 raft_consensus.cc:2354] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6.
I20260924 14:29:46.849291 28172 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:46.890226 27787 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:29:46.921552 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 70788.1us
I20260924 14:29:46.927282 27123 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:46.938019 28091 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:47.082152 28091 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:46.939388 28250 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260924 14:29:47.084151 27188 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
W20260924 14:29:46.872215 28235 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260924 14:29:46.779353 28188 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.155s
W20260924 14:29:47.081700 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 154503us
I20260924 14:29:46.901190 28186 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:47.086606 28186 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:47.088958 28186 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:47.089286 27779 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:46.874655 27972 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:47.092808 26398 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:29:47.100178 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:47.040985 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 285) took 1330 ms. Trace:
I20260924 14:29:47.046180 28255 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:47.049849 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 307040us
I20260924 14:29:47.064575 27181 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
W20260924 14:29:47.114470 27194 consensus_meta.cc:287] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260924 14:29:47.115726 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:47.125682 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:47.123651 27189 rpcz_store.cc:276] 0924 14:29:45.710734 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:45.710873 (+   139us) service_pool.cc:227] Handling call
0924 14:29:45.725076 (+ 14203us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:45.985261 (+260185us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:47.040670 (+1055409us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b540000e954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:47.040906 (+   236us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":596733696}
I20260924 14:29:47.124635 27194 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:47.151782 27194 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:47.113338 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 396) took 1078 ms. Trace:
I20260924 14:29:47.141925 28042 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:47.143775 28225 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:47.154155 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:47.167083 26403 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
W20260924 14:29:47.174139 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 213178us
I20260924 14:29:47.182181 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:47.191565 27185 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:47.150041 27523 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260924 14:29:47.188506 28253 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:47.188788 27978 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:47.202521 27978 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:47.191556 28225 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:47.166334 27191 rpcz_store.cc:276] 0924 14:29:46.035194 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:46.043824 (+  8630us) service_pool.cc:227] Handling call
0924 14:29:47.112574 (+1068750us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:47.112807 (+   233us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:47.112995 (+   188us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:47.113021 (+    26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:47.113065 (+    44us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:47.113109 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:47.113165 (+    56us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:47.113279 (+   114us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5015424}
I20260924 14:29:47.194612 28220 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:47.205238 28225 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:47.206080 28220 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:47.208348 28220 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:47.201673 27795 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:47.143622 28255 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:47.214166 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 818) took 1175 ms. Trace:
I20260924 14:29:47.214910 28255 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:47.205091 27978 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:47.218107 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:47.196463 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:47.219298 28260 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:47.219946 28260 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:47.221001 28260 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:47.237102 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:47.238606 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:47.239521 26325 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:47.239902 26325 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:47.242084 27123 leader_election.cc:398] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6.
I20260924 14:29:47.242475 27123 leader_election.cc:401] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260924 14:29:47.246256 27190 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:47.248240 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:47.251896 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 73430.5us
I20260924 14:29:47.248582 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:47.252867 26325 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:47.225018 27904 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:47.224844 28255 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:47.256338 26325 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:47.256644 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:47.268366 28257 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260924 14:29:47.272850 27914 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:47.273624 28255 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:47.273674 27914 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:29:47.275662 28255 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:47.275772 26324 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:47.281466 26257 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:47.283164 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:47.285327 28270 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260924 14:29:47.287714 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:47.231336 27978 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260924 14:29:47.268965 28261 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6.
I20260924 14:29:47.214802 26400 rpcz_store.cc:276] 0924 14:29:46.038825 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:46.040775 (+  1950us) service_pool.cc:227] Handling call
0924 14:29:47.213218 (+1172443us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:47.213473 (+   255us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:47.213706 (+   233us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:47.213760 (+    54us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:47.213818 (+    58us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:47.213887 (+    69us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:47.213962 (+    75us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:47.214095 (+   133us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":10150144}
I20260924 14:29:47.334573 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:47.341152 26400 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:47.343957 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:47.344738 27192 raft_consensus.cc:2362] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6. Re-sending same reply.
I20260924 14:29:47.360051 26325 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:47.363636 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:47.368001 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 395) took 1326 ms. Trace:
I20260924 14:29:47.370201 28237 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:47.368551 27192 rpcz_store.cc:276] 0924 14:29:46.034899 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:46.041449 (+  6550us) service_pool.cc:227] Handling call
0924 14:29:47.343790 (+1302341us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:47.361388 (+ 17598us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":12991872}
I20260924 14:29:47.345438 27980 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260924 14:29:47.411046 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:47.421887 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:47.422178 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:47.371037 28237 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:47.448184 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:47.393294 28088 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.059s
I20260924 14:29:47.419966 27980 consensus_queue.cc:1225] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260924 14:29:47.483314 27795 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:47.417424 28194 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260924 14:29:47.506650 28237 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:47.520193 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:47.521142 26402 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:47.522651 28259 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:47.527748 26397 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:47.543938 26402 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:47.544705 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:47.542518 28202 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:47.525257 28259 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:47.544773 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:47.536224 28070 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
W20260924 14:29:47.555835 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 318915us
I20260924 14:29:47.594432 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:47.577245 28127 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:47.601958 28261 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:47.603340 26295 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:47.574175 28202 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:47.600561 27850 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:47.620879 27850 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:47.618629 28088 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260924 14:29:47.630442 28202 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:47.630718 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:47.612344 28127 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:47.636179 28127 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:47.637771 28228 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.337s	user 0.005s	sys 0.006s
I20260924 14:29:47.639204 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:47.641831 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:47.642367 27850 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:47.642484 28070 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:47.643657 28070 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:47.648001 28070 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:47.649245 28259 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:47.651568 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:47.656992 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32930 (request call id 8) took 1631 ms. Trace:
I20260924 14:29:47.657440 26329 rpcz_store.cc:276] 0924 14:29:46.025487 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:46.030441 (+  4954us) service_pool.cc:227] Handling call
0924 14:29:47.656656 (+1626215us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400404d20. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92e3dc0 00007f09b93489c2 00007f09be29281a 00007f09b7985d04 00007f09b7985c92 00007f09b7985bc1 00007f09b7985b3c 00007f09b7984df2 00007f09b777894c 00007f09b7777d86
0924 14:29:47.656908 (+   252us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":512277760}
I20260924 14:29:47.659296 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:29:47.663398 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:47.669657 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:47.671334 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:47.696017 28180 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:47.700384 28180 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:47.740080 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:47.754551 28180 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:47.760944 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:29:47.764262 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:47.764678 28242 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:47.793047 26331 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:47.793566 26331 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:47.761912 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:29:47.818131 28188 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:47.821029 28188 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:47.835088 26257 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.028s	sys 0.034s
I20260924 14:29:47.794330 26331 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:47.835700 26257 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.029s	sys 0.034s
I20260924 14:29:47.841254 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:47.852829 26331 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:47.842304 26401 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:29:47.878885 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:47.837059 28169 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.016s	sys 0.037s
I20260924 14:29:47.765393 28242 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:47.889652 28169 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.017s	sys 0.037s
I20260924 14:29:47.881194 28188 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:47.900884 26256 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:47.903318 26079 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 137) took 7952 ms. Trace:
I20260924 14:29:47.878249 28241 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:47.889122 28254 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260924 14:29:47.912267 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.015s	sys 0.045s
W20260924 14:29:47.928936 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.015s	sys 0.045s
I20260924 14:29:47.894071 28125 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:29:47.929229 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 562751us
I20260924 14:29:47.903851 26079 rpcz_store.cc:276] 0924 14:29:39.951248 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.951608 (+   360us) service_pool.cc:227] Handling call
0924 14:29:47.903245 (+7951637us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:29:44.436195 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:29:44.436868 (+   673us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:29:44.436924 (+    56us) write_op.cc:435] Acquired schema lock
0924 14:29:44.436944 (+    20us) tablet.cc:660] Decoding operations
0924 14:29:44.443039 (+  6095us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:29:44.443150 (+   111us) write_op.cc:641] Partition lock acquired for write op
0924 14:29:44.443208 (+    58us) tablet.cc:683] Acquiring locks for 18 operations
0924 14:29:44.550821 (+107613us) tablet.cc:699] Row locks acquired
0924 14:29:44.550860 (+    39us) write_op.cc:260] PREPARE: finished
0924 14:29:44.551016 (+   156us) write_op.cc:270] Start()
0924 14:29:44.551178 (+   162us) write_op.cc:276] Timestamp: P: 1790260184550960 usec, L: 0
0924 14:29:44.551227 (+    49us) op_driver.cc:331] REPLICATION: starting
0924 14:29:44.645337 (+ 94110us) log.cc:842] Serialized 7191 byte log entry
0924 14:29:44.761213 (+115876us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b180035df40. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:29:44.954249 (+193036us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:29:45.351292 (+397043us) op_driver.cc:444] REPLICATION: finished
0924 14:29:45.519785 (+168493us) write_op.cc:301] APPLY: starting
0924 14:29:45.540005 (+ 20220us) tablet.cc:1365] starting BulkCheckPresence
0924 14:29:45.544430 (+  4425us) tablet.cc:1368] finished BulkCheckPresence
0924 14:29:45.544470 (+    40us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:29:45.642089 (+ 97619us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:29:45.642473 (+   384us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0924 14:29:45.642541 (+    68us) write_op.cc:312] APPLY: finished
0924 14:29:45.643063 (+   522us) log.cc:842] Serialized 124 byte log entry
0924 14:29:45.643260 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:45.648765 (+  5505us) write_op.cc:454] Released schema lock
0924 14:29:45.702586 (+ 53821us) write_op.cc:341] FINISH: Updating metrics
0924 14:29:45.727560 (+ 24974us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:45.730175 (+  2615us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":154567,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":101293,"apply.run_cpu_time_us":6147,"apply.run_wall_time_us":242576,"num_ops":18,"prepare.queue_time_us":196318,"prepare.run_cpu_time_us":58510,"prepare.run_wall_time_us":291474,"raft.queue_time_us":215538,"raft.run_cpu_time_us":7547,"raft.run_wall_time_us":167429,"replication_time_us":788037,"spinlock_wait_cycles":60511872,"thread_start_us":425250,"threads_started":4,"wal-append.queue_time_us":89246,"wal-append.run_cpu_time_us":19037,"wal-append.run_wall_time_us":1883815}]]}
I20260924 14:29:47.944758 27126 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:47.961467 26398 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260924 14:29:47.965152 28234 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:47.965544 28242 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:47.965855 28234 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:47.975978 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:47.928680 28254 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:47.966889 28234 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:47.981449 28176 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:47.925976 28241 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:48.022040 28241 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:48.005302 28254 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:29:47.947639 28260 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.072s
I20260924 14:29:48.039305 28267 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:48.041927 26405 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:48.050506 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:48.058275 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 86105.2us
I20260924 14:29:48.063683 27904 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:48.040194 28267 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:48.106093 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44768.2us
I20260924 14:29:48.109817 28267 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:48.106631 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 58446.9us
I20260924 14:29:48.143087 26331 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
W20260924 14:29:48.161818 26331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 186) took 1475 ms (client timeout 1907 ms). Trace:
W20260924 14:29:48.162197 26331 rpcz_store.cc:269] 0924 14:29:46.686578 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:46.686736 (+   158us) service_pool.cc:227] Handling call
0924 14:29:47.792898 (+1106162us) spinlock_profiling.cc:243] Waited 59.6 ms on lock 0x7b540067d2a0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92de730 00007f09b9351eff 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86
0924 14:29:48.161720 (+368822us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":137028864}
W20260924 14:29:48.162967 27978 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260924 14:29:48.184209 28187 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:48.189340 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:48.196842 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.024s	sys 0.048s
W20260924 14:29:48.197613 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.024s	sys 0.048s
W20260924 14:29:48.198120 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 594108us
W20260924 14:29:48.205785 28219 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.019s	sys 0.056s
W20260924 14:29:48.206298 28219 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.019s	sys 0.057s
W20260924 14:29:48.210233 26404 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.018s	sys 0.072s
W20260924 14:29:48.211333 26404 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.019s	sys 0.072s
I20260924 14:29:48.213702 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 761) took 1026 ms. Trace:
W20260924 14:29:48.205838 26258 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.034s	sys 0.041s
W20260924 14:29:48.218456 26258 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.034s	sys 0.041s
I20260924 14:29:48.224455 26327 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:29:48.216982 26404 rpcz_store.cc:276] 0924 14:29:47.187212 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:47.187388 (+   176us) service_pool.cc:227] Handling call
0924 14:29:47.659874 (+472486us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:47.660053 (+   179us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:47.660244 (+   191us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:29:47.660539 (+   295us) log.cc:842] Serialized 14 byte log entry
0924 14:29:48.213219 (+552680us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:48.213287 (+    68us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:48.213354 (+    67us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:48.213497 (+   143us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:48.213642 (+   145us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1661,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"spinlock_wait_cycles":14899456,"thread_start_us":552276,"threads_started":1}
I20260924 14:29:48.222658 28127 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:29:48.242429 28201 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.028s	sys 0.028s
I20260924 14:29:48.272531 26327 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:48.279137 26258 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:48.281939 26327 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:48.289318 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:48.276845 28201 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.029s	sys 0.028s
I20260924 14:29:48.264031 28159 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:48.282256 28137 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:48.280030 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:48.301893 26369 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:48.304601 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:48.315630 28294 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260924 14:29:48.300069 28159 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:48.328187 27502 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:48.300024 26330 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:48.328660 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:48.318774 28211 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.011s	sys 0.061s
W20260924 14:29:48.336719 28211 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.011s	sys 0.063s
I20260924 14:29:48.348807 27502 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:48.309128 28219 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:29:48.358021 28299 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260924 14:29:48.317638 27972 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260924 14:29:48.309125 28137 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:48.371274 28137 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:48.375133 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:48.376121 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:48.302868 28297 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260924 14:29:48.324710 28088 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260924 14:29:48.400254 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:48.402770 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:48.404274 28219 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:48.383196 27850 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:48.388082 26078 catalog_manager.cc:5495] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:48.387723 28267 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
W20260924 14:29:48.424196 27124 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.044s	sys 0.065s
W20260924 14:29:48.434916 27124 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.044s	sys 0.065s
I20260924 14:29:48.436625 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:48.438370 26078 catalog_manager.cc:5495] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 5, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 5 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:48.450433 26331 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:48.451710 28159 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:48.453852 28291 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260924 14:29:48.455492 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 520) took 1101 ms. Trace:
I20260924 14:29:48.449430 28137 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:29:48.455915 28159 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:48.462565 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:48.430922 28254 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:48.482841 26329 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
W20260924 14:29:48.519639 26325 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.010s	sys 0.047s
W20260924 14:29:48.520139 26325 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.010s	sys 0.047s
I20260924 14:29:48.549171 28254 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:48.564248 26328 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:48.551301 28237 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:48.455874 26326 rpcz_store.cc:276] 0924 14:29:47.353737 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:47.356895 (+  3158us) service_pool.cc:227] Handling call
0924 14:29:47.391937 (+ 35042us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:47.460231 (+ 68294us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:47.471689 (+ 11458us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:47.510251 (+ 38562us) log.cc:842] Serialized 26 byte log entry
0924 14:29:47.888504 (+378253us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:47.941886 (+ 53382us) log.cc:842] Serialized 14 byte log entry
0924 14:29:47.981195 (+ 39309us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:47.981287 (+    92us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:47.990857 (+  9570us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b18003ce1a0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:29:48.455148 (+464291us) raft_consensus.cc:1671] finished
0924 14:29:48.455259 (+   111us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:48.455404 (+   145us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":66086,"prepare.queue_time_us":34256,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":13391,"raft.queue_time_us":45262,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":9895,"spinlock_wait_cycles":25122688,"thread_start_us":367008,"threads_started":1,"wal-append.queue_time_us":797486}
I20260924 14:29:48.587558 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 521) took 1194 ms. Trace:
I20260924 14:29:48.594336 26325 rpcz_store.cc:276] 0924 14:29:47.392696 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:47.392859 (+   163us) service_pool.cc:227] Handling call
0924 14:29:47.393154 (+   295us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:47.427879 (+ 34725us) raft_consensus.cc:1520] Early marking committed up to index 2
0924 14:29:47.929637 (+501758us) log.cc:842] Serialized 14 byte log entry
0924 14:29:48.549852 (+620215us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:48.549922 (+    70us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:48.549978 (+    56us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:48.586700 (+ 36722us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b1800108880. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9
0924 14:29:48.587280 (+   580us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b54005d8c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:48.587322 (+    42us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:48.587492 (+   170us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":30128,"prepare.queue_time_us":213110,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":71,"raft.queue_time_us":29322,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":6358,"spinlock_wait_cycles":81687680,"thread_start_us":1072903,"threads_started":2}
I20260924 14:29:48.577641 28263 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
W20260924 14:29:48.721558 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 122149us
W20260924 14:29:48.725597 26256 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.013s	sys 0.032s
W20260924 14:29:48.726675 26256 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.013s	sys 0.032s
I20260924 14:29:48.770483 27502 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:48.771150 27502 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:48.725211 28137 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:48.717823 28267 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:48.770045 28187 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:48.842499 26405 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.019s	sys 0.010s
W20260924 14:29:48.851678 26405 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.019s	sys 0.010s
I20260924 14:29:48.773535 28197 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:48.867811 28296 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:48.874971 27502 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:48.876257 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:48.789690 28137 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:48.846968 28240 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:48.848341 28263 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:48.889694 26257 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.019s	sys 0.021s
W20260924 14:29:48.890146 26257 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.019s	sys 0.021s
I20260924 14:29:48.892561 26328 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260924 14:29:48.894241 26370 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.023s	sys 0.017s
W20260924 14:29:48.894658 26370 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.023s	sys 0.017s
W20260924 14:29:48.887935 28091 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.011s	sys 0.026s
I20260924 14:29:48.803253 28267 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:48.896179 26371 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.003s	sys 0.037s
W20260924 14:29:48.917284 26371 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.003s	sys 0.037s
W20260924 14:29:48.917798 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 582379us
I20260924 14:29:48.859874 28187 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:48.930979 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:48.910679 28091 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.011s	sys 0.026s
I20260924 14:29:48.938903 27502 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:29:48.888777 28296 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:48.875152 28197 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:48.893617 28263 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:48.902657 28169 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
W20260924 14:29:48.955345 27124 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.010s	sys 0.012s
W20260924 14:29:48.955811 27124 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.010s	sys 0.012s
I20260924 14:29:48.893141 28240 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:48.962334 27904 raft_consensus.cc:979] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:29:48.957398 28091 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:29:48.973786 28169 raft_consensus.cc:3032] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:48.982271 28211 raft_consensus.cc:1067] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:29:48.996598 28297 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:48.997349 28297 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:49.000310 26327 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.013s	sys 0.012s
W20260924 14:29:49.000686 26327 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.013s	sys 0.012s
I20260924 14:29:48.994993 27502 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:49.002013 27502 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:48.984067 28091 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:29:49.002648 28302 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260924 14:29:48.991074 28169 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:49.023748 27502 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:49.026842 28137 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:49.035969 28263 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:49.037725 28240 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:49.038987 28297 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:49.016150 28306 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260924 14:29:49.022825 28303 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260924 14:29:49.079200 28091 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:49.098260 28091 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:29:49.074658 26323 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.019s	sys 0.031s
I20260924 14:29:49.078969 28187 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:49.124173 28197 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:49.101302 28169 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:49.127096 28169 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:49.076658 28267 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:49.100880 27902 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:49.133755 28296 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:49.186458 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 301) took 1203 ms. Trace:
I20260924 14:29:49.187054 26401 rpcz_store.cc:276] 0924 14:29:47.982735 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:47.982868 (+   133us) service_pool.cc:227] Handling call
0924 14:29:47.983218 (+   350us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:29:47.983617 (+   399us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:47.983647 (+    30us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:29:47.984237 (+   590us) log.cc:842] Serialized 52 byte log entry
0924 14:29:48.436661 (+452424us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:48.436717 (+    56us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:48.436802 (+    85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:49.186136 (+749334us) raft_consensus.cc:1671] finished
0924 14:29:49.186167 (+    31us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:49.186383 (+   216us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1214,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":452014,"threads_started":1,"wal-append.queue_time_us":856248}
I20260924 14:29:49.205300 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:49.207114 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 911) took 1075 ms. Trace:
I20260924 14:29:49.158880 27902 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:49.196887 26295 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 1.533s (SENT) [suppressed 14 similar messages]
I20260924 14:29:49.207989 26256 rpcz_store.cc:276] 0924 14:29:48.091426 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:48.095123 (+  3697us) service_pool.cc:227] Handling call
0924 14:29:48.097378 (+  2255us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:29:48.126788 (+ 29410us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:48.126832 (+    44us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:29:48.145516 (+ 18684us) log.cc:842] Serialized 52 byte log entry
0924 14:29:48.728140 (+582624us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:49.165843 (+437703us) log.cc:842] Serialized 14 byte log entry
0924 14:29:49.166787 (+   944us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:49.166907 (+   120us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:49.166948 (+    41us) raft_consensus.cc:1671] finished
0924 14:29:49.166959 (+    11us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:49.167152 (+   193us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28870,"prepare.queue_time_us":142435,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":29616,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":12335,"thread_start_us":941328,"threads_started":2,"wal-append.queue_time_us":862631}
W20260924 14:29:49.252533 28308 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260924 14:29:49.262403 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:49.266793 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:49.285473 26402 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:29:49.299938 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:49.300329 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:49.312041 28304 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.452s	user 0.005s	sys 0.000s
W20260924 14:29:49.316541 28320 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260924 14:29:49.319191 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.326287 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:49.301256 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:49.329778 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:49.331168 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:49.360412 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:49.272918 28169 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:49.272918 28296 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:49.456470 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 127264us
I20260924 14:29:49.327579 28197 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:49.489753 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.458653 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 310) took 1230 ms. Trace:
I20260924 14:29:49.281697 28316 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:49.466728 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 269853us
I20260924 14:29:49.324656 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:49.488010 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:49.432139 28317 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260924 14:29:49.488924 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:49.520725 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.501539 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:49.536741 27189 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:49.540184 28316 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:49.472174 28242 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:49.542706 28242 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:49.544255 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 202) took 1182 ms. Trace:
I20260924 14:29:49.544963 27902 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:49.545740 28316 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:49.489522 28169 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:49.547474 28242 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:49.548377 28169 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:49.464706 28187 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:49.551913 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:29:49.465452 28293 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:49.556442 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:49.544655 26327 rpcz_store.cc:276] 0924 14:29:48.362121 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:48.362275 (+   154us) service_pool.cc:227] Handling call
0924 14:29:48.362700 (+   425us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:48.368176 (+  5476us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:48.368207 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:48.441387 (+ 73180us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b2c001e1fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:29:48.447023 (+  5636us) log.cc:842] Serialized 26 byte log entry
0924 14:29:49.001017 (+553994us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:49.303294 (+302277us) log.cc:842] Serialized 14 byte log entry
0924 14:29:49.303772 (+   478us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:49.303838 (+    66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:49.543964 (+240126us) raft_consensus.cc:1671] finished
0924 14:29:49.544070 (+   106us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:49.544199 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5015,"prepare.queue_time_us":381702,"prepare.run_cpu_time_us":10770,"prepare.run_wall_time_us":67733,"raft.queue_time_us":5580,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":169,"spinlock_wait_cycles":17341312,"thread_start_us":837179,"threads_started":2,"wal-append.queue_time_us":746729}
I20260924 14:29:49.562251 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.516500 26405 rpcz_store.cc:276] 0924 14:29:48.228357 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:48.228495 (+   138us) service_pool.cc:227] Handling call
0924 14:29:48.228836 (+   341us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:48.234131 (+  5295us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:48.234174 (+    43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:48.240130 (+  5956us) log.cc:842] Serialized 26 byte log entry
0924 14:29:48.861478 (+621348us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:48.861537 (+    59us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:48.865228 (+  3691us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:49.458437 (+593209us) raft_consensus.cc:1671] finished
0924 14:29:49.458459 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:49.458595 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":5851,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":5736,"thread_start_us":565778,"threads_started":1,"wal-append.queue_time_us":798640}
I20260924 14:29:49.596087 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:49.472589 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:49.499577 28313 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.276s	user 0.000s	sys 0.005s
W20260924 14:29:49.602985 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 299333us
I20260924 14:29:49.605203 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:49.608821 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47651.9us
I20260924 14:29:49.464203 28334 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:49.612416 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:49.623030 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:49.625484 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:49.558904 28219 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:49.515251 28296 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:49.631385 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:49.617050 28328 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:49.635982 26405 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:46098
I20260924 14:29:49.659752 26405 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:29:49.660151 26405 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:49.538077 28197 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:49.644236 28240 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:49.631966 28242 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:49.645282 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:49.607933 28315 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:49.607527 28040 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:49.656255 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.656512 26327 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0756687d99e643c094531124c9438376"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:49.729686 26327 raft_consensus.cc:595] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:49.714680 26405 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:49.561676 28187 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:49.597667 28293 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:49.716431 28323 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.390s	user 0.000s	sys 0.001s
I20260924 14:29:49.663715 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.717329 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:49.704443 27188 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260924 14:29:49.689186 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.731539 27181 raft_consensus.cc:2396] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260924 14:29:49.731827 26405 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:49.742436 27190 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:49.715967 28166 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:49.661257 28219 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:49.747355 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 133964us
I20260924 14:29:49.736012 26255 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:29:49.742671 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.669524 28240 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:49.752799 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.752535 28166 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:49.752907 28336 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:29:49.754891 28187 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:49.756064 26327 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:49.767999 26327 raft_consensus.cc:595] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:49.756407 28197 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:49.761866 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.685742 28242 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:49.731426 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:49.665098 28328 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:49.790297 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 142168us
I20260924 14:29:49.785409 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:49.794705 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:49.830436 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:49.832088 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 100765us
I20260924 14:29:49.833073 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 205) took 1272 ms. Trace:
I20260924 14:29:49.833492 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:49.833611 26331 rpcz_store.cc:276] 0924 14:29:48.561052 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:48.561247 (+   195us) service_pool.cc:227] Handling call
0924 14:29:48.561718 (+   471us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:48.575303 (+ 13585us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:48.575332 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:48.577367 (+  2035us) log.cc:842] Serialized 26 byte log entry
0924 14:29:49.062496 (+485129us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:49.062537 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:49.062621 (+    84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:49.832799 (+770178us) raft_consensus.cc:1671] finished
0924 14:29:49.832830 (+    31us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:49.833001 (+   171us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":100,"raft.queue_time_us":265,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":15872,"thread_start_us":477653,"threads_started":1,"wal-append.queue_time_us":643475}
W20260924 14:29:49.831614 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 210146us
W20260924 14:29:49.841398 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50891.9us
I20260924 14:29:49.839208 28331 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:49.857506 26331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:49.858326 26331 raft_consensus.cc:595] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:49.857590 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 78846.1us
I20260924 14:29:49.742672 28040 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:49.861814 27126 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:49.741485 28334 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:49.868244 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 241442us
I20260924 14:29:49.831704 28325 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:49.888151 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:49.895376 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:49.896185 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:49.864742 28331 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260924 14:29:49.937724 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 49656.5us
I20260924 14:29:49.885145 28296 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:49.939730 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 205159us
I20260924 14:29:49.903370 27902 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:49.954908 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:49.938941 28341 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:49.938589 28350 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:49.917670 28351 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:49.957861 28316 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:49.974489 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:49.935017 27850 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:49.968117 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:49.937996 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:49.952540 28237 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:49.978292 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.978407 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.980372 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:49.960470 28325 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:49.989470 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:49.991154 26295 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:49.960732 28260 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:49.973191 28329 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:29:50.003502 28348 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260924 14:29:50.004164 28240 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:50.017117 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 181049us
I20260924 14:29:49.969120 27902 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:50.022962 27902 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:50.001539 28322 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:50.024049 28322 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:50.005545 28260 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:50.025203 28334 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:50.025992 28219 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:50.026214 28322 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:50.026656 28260 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:50.015851 28329 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:50.024044 28357 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260924 14:29:50.027681 28040 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:50.004894 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:50.028872 28329 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:49.980557 27850 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:50.030584 28242 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:49.980357 28364 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:50.031872 27850 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:50.018251 28169 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:29:50.034286 28091 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.936s	user 0.014s	sys 0.000s
I20260924 14:29:50.000474 28325 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:50.036376 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 209) took 1195 ms. Trace:
I20260924 14:29:50.037462 28325 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:50.021082 28328 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:50.015816 28167 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:50.035311 27185 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:29:50.014153 28352 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:29:50.038728 28352 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:50.039201 28352 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:29:50.036768 26325 rpcz_store.cc:276] 0924 14:29:48.840780 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:48.842307 (+  1527us) service_pool.cc:227] Handling call
0924 14:29:48.842710 (+   403us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:49.265640 (+422930us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:49.265674 (+    34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:49.287619 (+ 21945us) log.cc:842] Serialized 26 byte log entry
0924 14:29:49.708790 (+421171us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:49.709368 (+   578us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:49.709464 (+    96us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:50.036169 (+326705us) raft_consensus.cc:1671] finished
0924 14:29:50.036190 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:50.036318 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":30116,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":661637,"threads_started":2,"wal-append.queue_time_us":545205}
W20260924 14:29:50.017450 28330 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260924 14:29:50.020418 28355 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:50.019656 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:50.056149 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 225856us
W20260924 14:29:50.060544 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 86883.5us
I20260924 14:29:50.064924 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.066843 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.069336 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.076656 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:50.076844 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:50.077345 26255 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:50.070693 27194 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:49.971496 28341 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:50.080430 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:50.083992 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 240432us
I20260924 14:29:50.082957 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:50.084074 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 93025.3us
I20260924 14:29:50.086575 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.088976 26255 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:50.091015 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:50.093446 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 73887us
I20260924 14:29:50.094250 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.101537 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:50.102366 26255 raft_consensus.cc:2396] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:29:50.105031 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:50.106117 28331 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:50.106523 28260 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:50.107255 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:50.107786 28355 raft_consensus.cc:979] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: : Instructing follower 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:50.108220 28355 raft_consensus.cc:1067] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Signalling peer 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:50.107141 28260 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:50.110831 28260 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:50.114699 28363 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260924 14:29:50.115949 28334 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:50.118132 28322 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:29:50.119855 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:50.128089 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:50.129259 28293 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:50.131274 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:50.131598 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:50.131844 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:50.132417 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:50.134187 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:50.134935 27123 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:50.139642 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46245.1us
I20260924 14:29:50.145360 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:50.153474 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:50.154188 26402 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:50.167868 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:50.169898 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:50.141724 28254 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:50.142869 28356 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:50.159020 28242 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:50.149358 28328 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:50.240273 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 82765.7us
W20260924 14:29:50.340799 28362 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260924 14:29:50.341173 28341 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:50.430822 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 260809us
W20260924 14:29:50.440744 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 58465.4us
I20260924 14:29:50.239240 28169 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:50.243033 28242 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:50.431200 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:50.443887 28356 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:50.241891 28254 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:50.518851 28242 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:50.432006 28328 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:50.451870 28329 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260924 14:29:50.549609 28322 raft_consensus.cc:979] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: : Instructing follower 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:50.545424 28240 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:50.547362 26324 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:50.547684 28137 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:50.579212 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 338089us
I20260924 14:29:50.513017 28355 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260924 14:29:50.595535 28386 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:50.569943 28219 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:50.600032 28202 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:50.580673 28352 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.001s
I20260924 14:29:50.570314 26324 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:50.596853 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:50.572507 28137 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:50.636471 28328 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:50.640023 28387 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:29:50.608404 28167 raft_consensus.cc:1067] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Signalling peer 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:50.641592 26324 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:50.652664 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 55913.3us
I20260924 14:29:50.571390 28240 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:50.661196 28240 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:50.602941 28356 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:50.665669 28137 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:50.668622 28254 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:50.670007 28387 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:29:50.671773 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 483906us
W20260924 14:29:50.682060 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 442994us
I20260924 14:29:50.682267 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:50.682823 27123 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:50.686697 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 183299us
I20260924 14:29:50.693147 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.695991 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.700099 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.691983 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:50.681384 28254 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:50.705631 28137 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:50.711488 28325 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:50.711555 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:50.712097 28325 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:50.714231 28325 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:50.714900 28325 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:50.711357 28137 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:50.718299 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62690.7us
I20260924 14:29:50.718581 28137 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:50.609754 28386 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:50.720932 28386 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:50.727080 28377 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:50.727274 28370 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:50.727849 28377 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:50.727104 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44365.2us
I20260924 14:29:50.681861 28352 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:50.730412 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:50.730618 28377 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:50.670624 28387 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:50.731665 28352 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:50.715301 28325 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:50.732338 28325 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:50.732731 28369 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:50.733362 28387 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:50.734084 28387 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:50.736147 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:50.707118 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:50.737731 28364 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:50.736174 28329 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:50.740083 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:50.742086 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:50.742713 26330 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:29:50.743089 28374 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:50.744336 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:50.744393 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:50.745478 26328 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:50.743716 28374 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:50.746340 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:50.746795 28316 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:50.681411 28202 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:50.749877 28374 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:50.750226 28202 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:50.751899 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:50.753401 27188 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:50480
I20260924 14:29:50.754168 27188 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:29:50.754621 27188 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:50.734683 28387 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:50.671170 28219 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:50.767925 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.764670 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:50.765733 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.786407 27182 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0756687d99e643c094531124c9438376"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:50480
I20260924 14:29:50.787256 27182 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:29:50.787698 27182 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:50.801311 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.801728 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.772570 27188 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:50.804262 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.833972 27182 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:50.836071 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 126376us
I20260924 14:29:50.751926 28331 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:50.842626 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:50.843042 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 78460.2us
I20260924 14:29:50.844352 27188 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:50.760483 28202 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:50.711344 28254 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:50.764150 28137 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:50.846390 27182 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:50.866392 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:50.843509 28347 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.603s	user 0.000s	sys 0.010s
W20260924 14:29:50.864450 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 160881us
I20260924 14:29:50.767349 28328 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:50.864048 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 67896us
I20260924 14:29:50.881640 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.889299 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:50.891793 26250 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:50.891887 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:50.891000 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47200.9us
I20260924 14:29:50.892923 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.870826 28137 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:50.900713 28137 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:50.901230 28137 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:50.852345 28331 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:50.903034 28331 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:50.903213 28254 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:50.870112 28296 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:50.866732 28202 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:50.903957 28296 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:50.905577 28202 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:50.905898 28296 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:50.906155 28393 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:50.908387 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:50.910323 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.919160 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:50.921048 26295 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:50.922525 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:50.901676 28137 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:50.927067 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:50.928174 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:50.926937 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 55233.1us
I20260924 14:29:50.932785 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 340) took 1658 ms. Trace:
I20260924 14:29:50.909797 28040 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:50.933197 26258 rpcz_store.cc:276] 0924 14:29:49.274865 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:49.275214 (+   349us) service_pool.cc:227] Handling call
0924 14:29:49.327646 (+ 52432us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:49.327951 (+   305us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:49.327976 (+    25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:49.333549 (+  5573us) log.cc:842] Serialized 26 byte log entry
0924 14:29:49.516767 (+183218us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:49.516860 (+    93us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:49.517135 (+   275us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:50.147015 (+629880us) spinlock_profiling.cc:243] Waited 8 ms on lock 0x7b1800180d00. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11
0924 14:29:50.932522 (+785507us) raft_consensus.cc:1671] finished
0924 14:29:50.932545 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:50.932727 (+   182us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1961,"raft.queue_time_us":161851,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":22297,"spinlock_wait_cycles":18396288,"thread_start_us":47541,"threads_started":1,"wal-append.queue_time_us":119433}
I20260924 14:29:50.936573 26369 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:50.937562 27182 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:50.888690 28328 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:51.044476 28399 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:51.045143 28399 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:51.045604 28137 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:51.045976 28328 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:51.046588 26254 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:51.047399 28242 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:51.049539 28242 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:51.049849 28403 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:51.056007 26254 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
W20260924 14:29:51.056178 28402 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260924 14:29:51.061408 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.062520 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:50.927960 28355 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:51.077342 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.078510 26250 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:50.914494 28393 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:50.936259 28396 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:51.112277 26329 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:51.061650 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.128948 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:51.129798 26329 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:51.130147 26329 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:51.065347 28169 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:51.047292 28399 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:51.131039 26329 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:51.144980 26329 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:51.148670 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:51.149581 26323 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:51.149984 26323 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:29:51.152233 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 385822us
I20260924 14:29:51.178634 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.121310 28377 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:51.177249 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 236154us
I20260924 14:29:51.160916 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.185811 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.186045 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.194482 26332 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:51.063438 28040 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:51.197176 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:51.200464 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 348) took 1575 ms. Trace:
I20260924 14:29:51.126688 28202 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:51.212431 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:51.138661 28395 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:51.206522 26257 rpcz_store.cc:276] 0924 14:29:49.625041 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:49.625157 (+   116us) service_pool.cc:227] Handling call
0924 14:29:49.625442 (+   285us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:49.847851 (+222409us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:49.847897 (+    46us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:49.847954 (+    57us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:49.847996 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:51.200118 (+1352122us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54006d2ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:51.200188 (+    70us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:51.200379 (+   191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":510875520}
I20260924 14:29:51.234937 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:51.236263 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:51.237450 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:51.238054 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.158254 28392 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.178944 28219 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:51.249445 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:51.252590 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:51.250335 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:51.203655 28331 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:51.199496 28387 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:51.273793 26329 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:51.306524 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.150961 26323 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:51.327721 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 391190us
I20260924 14:29:51.232774 28367 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:51.335268 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.342059 28309 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:51.344873 28403 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.346782 28219 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.347057 28403 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:51.347543 28219 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:51.349985 28403 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:51.350061 28219 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:51.350976 28403 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.354223 28356 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.342871 28309 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:51.355266 28396 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.356415 28040 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:51.370900 28414 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:51.374532 28356 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:51.361507 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:51.362751 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:51.239107 28401 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:51.304549 27188 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:51.368749 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:51.213240 28351 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.372056 28403 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:51.390496 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.369684 28374 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:51.214859 28377 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:51.379240 28401 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:51.412300 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.439177 26298 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:51.233104 28202 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:51.412133 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.235846 28395 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:51.375855 28411 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:51.442034 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.262854 28392 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:51.455150 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:51.379000 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:51.456983 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.379350 28379 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:51.344149 28367 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:51.459921 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 220954us
I20260924 14:29:51.443893 26298 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:51.408857 28403 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:51.436717 28040 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:51.379000 28356 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:51.443112 28389 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:51.479555 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:51.470011 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:51.481393 26329 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:51.465801 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.465262 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:51.467021 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.004s	sys 0.017s
I20260924 14:29:51.450714 28401 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:29:51.482667 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.005s	sys 0.017s
I20260924 14:29:51.488709 26328 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:51.483644 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.490238 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.491035 26329 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:51.467038 28322 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:51.464406 28374 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:51.493356 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.388499 28396 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:51.515045 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 71262.4us
I20260924 14:29:51.414225 28421 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260924 14:29:51.377702 28414 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:51.475315 28420 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:51.476867 28427 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.487720 28393 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:51.409813 28351 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:51.489815 28137 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:51.484597 28356 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:51.477552 28411 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:51.538951 28417 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260924 14:29:51.539849 28421 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:51.540964 28377 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:51.484565 28389 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:51.487545 26252 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
W20260924 14:29:51.565263 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 76137.3us
I20260924 14:29:51.567006 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:51.580542 28367 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:51.580938 28396 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:51.581029 28414 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:51.481066 28379 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:51.583029 28379 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:51.583258 28411 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:51.585397 28325 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.853s	user 0.005s	sys 0.008s
I20260924 14:29:51.585916 28325 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:51.586383 28309 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:51.543720 28427 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:51.589438 28427 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:51.590010 28374 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:51.592995 28401 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:51.594298 26250 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:51.595116 28351 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:51.596626 28401 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:51.596825 28392 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:51.571135 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 158908us
W20260924 14:29:51.576401 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 91580us
W20260924 14:29:51.608278 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 145762us
I20260924 14:29:51.610380 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:51.613229 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:51.615682 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:51.617620 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.574692 28359 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:29:51.621593 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 138019us
I20260924 14:29:51.624442 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.625348 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.628646 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.628593 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.630353 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:51.598546 28395 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:51.634225 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.634956 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:51.635612 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.619887 28328 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.620416 28401 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.648146 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.643849 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.620687 28387 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.620134 28137 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:51.653615 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:51.660473 26399 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:51.662322 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.663930 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:51.667066 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:51.645299 28420 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:51.665752 28342 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:51.675987 28420 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:51.676167 28342 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:51.676940 28420 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:51.678299 28342 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:51.678821 28342 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.680567 28429 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:51.681015 28429 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:51.681195 28415 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:51.681392 28429 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:51.658754 28202 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:51.675249 28431 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:51.680575 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.694598 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:51.699627 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:51.671044 28359 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:51.703676 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.666718 28437 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:51.679247 28342 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:51.669282 28137 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:51.682611 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:51.708585 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45598.2us
I20260924 14:29:51.707305 28351 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:51.693621 28431 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:51.665166 28328 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:51.713744 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:51.700780 28369 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:51.666234 28401 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:51.704308 28424 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:51.667794 28387 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:51.704048 28325 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:51.712013 28395 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:51.716421 28431 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:51.719987 26400 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:51.731936 28387 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:51.732795 28346 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.900s	user 0.001s	sys 0.023s
W20260924 14:29:51.735998 27185 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 442) took 2213 ms (client timeout 1612 ms). Trace:
I20260924 14:29:51.736657 28342 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:51.736927 28328 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:51.737790 28137 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:51.738341 26402 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:29:51.738950 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 332) took 1300 ms. Trace:
I20260924 14:29:51.739341 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.739889 28401 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:51.740314 28385 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.144s	user 0.010s	sys 0.015s
I20260924 14:29:51.681805 28415 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:51.740625 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:51.742455 28415 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:51.736411 27185 rpcz_store.cc:269] 0924 14:29:49.522701 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:49.522861 (+   160us) service_pool.cc:227] Handling call
0924 14:29:51.735686 (+2212825us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54000028d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:51.735899 (+   213us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1061322624}
I20260924 14:29:51.739231 26402 rpcz_store.cc:276] 0924 14:29:50.438388 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:50.439121 (+   733us) service_pool.cc:227] Handling call
0924 14:29:51.737968 (+1298847us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63eeb7c 00007f09b63ee73b 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:51.738163 (+   195us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:51.738713 (+   550us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:51.738902 (+   189us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3358080}
I20260924 14:29:51.749816 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32930 (request call id 11) took 1893 ms. Trace:
I20260924 14:29:51.754642 26331 rpcz_store.cc:276] 0924 14:29:49.856958 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:49.857340 (+   382us) service_pool.cc:227] Handling call
0924 14:29:51.749499 (+1892159us) spinlock_profiling.cc:243] Waited 9.66 ms on lock 0x7b540067bea0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92e3dc0 00007f09b93489c2 00007f09be29281a 00007f09b7985d04 00007f09b7985c92 00007f09b7985bc1 00007f09b7985b3c 00007f09b7984df2 00007f09b777894c 00007f09b7777d86
0924 14:29:51.749745 (+   246us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":22220032}
I20260924 14:29:51.722798 28351 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:51.725354 28424 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:51.767697 28332 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:51.768466 28332 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:51.771088 28332 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:51.762480 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.766476 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:51.774012 28332 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:29:51.727754 28434 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:51.779660 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.780769 28401 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:51.781437 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.781649 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.782299 26403 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:29:51.783349 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.793004 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:51.801132 28401 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:51.778702 28424 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:51.810333 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:51.808406 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.748027 28395 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:51.807737 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.818394 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.798708 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:51.822077 28367 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:51.824685 28137 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:51.825548 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:51.808200 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:51.827886 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:51.786720 28411 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:51.828969 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.844909 28137 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:51.794378 28389 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:51.849175 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:51.849938 28377 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:29:51.850606 28377 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:51.853160 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.853893 28127 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:51.858770 28395 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:51.862919 28388 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.145s	user 0.009s	sys 0.010s
W20260924 14:29:51.865679 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 55425.1us
I20260924 14:29:51.875475 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.803869 26224 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:51.893462 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:51.887825 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 84046.6us
I20260924 14:29:51.901067 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:51.898165 26224 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:51.910180 26224 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:52.020104 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.022710 28452 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:52.023234 28452 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:51.822829 28367 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:52.022171 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:52.038616 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:52.029703 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 154425us
I20260924 14:29:52.043957 26256 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:29:51.901968 27188 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:29:51.853817 28350 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:52.060945 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.061332 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.062534 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:52.063601 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 400495us
I20260924 14:29:52.065423 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.039623 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.077594 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.038336 28315 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:52.057703 28202 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:52.090863 26249 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:52.021075 28137 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:52.021665 28401 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:52.021777 28328 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:52.088234 26330 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:52.105042 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.115773 26255 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:29:52.109756 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:52.117616 28447 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260924 14:29:52.096145 28202 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:52.118466 28202 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:52.118997 28392 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:52.120087 28392 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:52.121945 28450 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260924 14:29:52.112807 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:52.113631 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.025490 28452 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:52.083290 28439 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:52.090284 28315 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:52.152603 28392 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:52.138950 28342 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:29:52.175019 28367 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:52.177968 28371 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 1.644s	user 0.009s	sys 0.023s
I20260924 14:29:52.078363 28350 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:52.180660 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 67953.8us
I20260924 14:29:52.181365 28350 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:52.163821 28315 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:52.148176 26330 raft_consensus.cc:595] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:52.073921 28351 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.161480 28429 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:52.192669 28383 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:52.194468 28429 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:52.172928 28342 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:52.204607 28381 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.580s	user 0.013s	sys 0.012s
I20260924 14:29:52.167574 28439 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:52.197894 28342 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
W20260924 14:29:52.227890 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.017s	sys 0.035s
W20260924 14:29:52.229161 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.018s	sys 0.036s
I20260924 14:29:52.202019 28332 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260924 14:29:52.220649 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.021s	sys 0.056s
W20260924 14:29:52.231586 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.022s	sys 0.056s
W20260924 14:29:52.217509 28420 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.037s	sys 0.063s
I20260924 14:29:52.239818 27189 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:52.183051 28431 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.241156 28434 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:52.242063 28431 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:52.247177 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.247113 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:52.242935 28420 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.037s	sys 0.063s
I20260924 14:29:52.256013 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.265966 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.266745 26329 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:29:52.267133 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.267319 26331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:52.269732 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:52.272104 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.271430 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.274602 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.275385 27185 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:29:52.272909 27186 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:52.241842 28434 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.269682 26331 raft_consensus.cc:595] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:52.276898 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:52.336226 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:52.372912 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 289020us
I20260924 14:29:52.358317 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.359277 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:52.373469 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 191585us
W20260924 14:29:52.414808 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 78716.2us
I20260924 14:29:52.254742 28439 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:52.266577 28431 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:52.420270 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.419905 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:52.411053 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 134265us
I20260924 14:29:52.423224 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.266494 28187 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:52.428685 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.430759 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.456780 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.417749 28188 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:29:52.341219 28374 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:52.280669 28350 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:52.418206 28367 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:52.459043 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.335831 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.460274 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.462683 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:52.475469 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.461442 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.463891 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:52.462108 28424 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.590s	user 0.004s	sys 0.008s
I20260924 14:29:52.355247 28375 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260924 14:29:52.457489 28439 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:52.455888 28187 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.466418 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.473841 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.474550 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.353845 28202 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:52.493847 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.491508 26405 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:29:52.460569 28188 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:52.491809 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.477334 28462 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:52.487651 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.476856 28396 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:52.498234 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.429512 28431 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.507666 28342 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.508318 28201 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:52.499136 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:52.499991 28188 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:29:52.500916 28439 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:52.501737 28187 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:52.497466 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.504912 27188 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:52.506652 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:52.505967 26405 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:29:52.514272 28431 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:52.486665 28424 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:52.476917 28374 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:52.518584 28428 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:29:52.519126 28428 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:52.518152 28202 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:52.520061 28428 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.515058 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:52.474884 28350 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.509646 28462 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.508895 28201 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:52.525243 28428 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:29:52.533670 28201 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:52.515393 28439 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.518177 28424 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:52.539861 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.542294 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:52.542799 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.543217 26328 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:52.544116 27185 raft_consensus.cc:2396] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260924 14:29:52.546181 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.546955 27191 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:52.548041 28439 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:52.548451 26400 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:52.549894 28350 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:52.550930 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.550894 28449 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:52.503486 28315 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.527324 28374 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:52.552351 26295 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:29:52.611724 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 59773.3us
I20260924 14:29:52.608111 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:52.608788 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:52.609640 28396 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:52.618018 28396 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:52.618366 28396 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:52.619271 28396 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:52.620532 28396 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:52.610445 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:52.616580 28127 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:52.626579 26328 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:29:52.555410 28462 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:52.632874 28456 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:29:52.629281 28127 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:52.557581 28431 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:52.636073 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.637128 28460 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:29:52.599471 28342 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:52.625073 28439 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:52.553844 28414 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:52.633683 28188 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:52.628903 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:52.649187 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.640714 28127 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:52.646189 28439 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:52.651798 28439 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:52.640702 28367 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:52.661005 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:52.647526 28414 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.659924 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.661057 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 541) took 1300 ms. Trace:
I20260924 14:29:52.662472 26258 rpcz_store.cc:276] 0924 14:29:51.360510 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:51.360640 (+   130us) service_pool.cc:227] Handling call
0924 14:29:52.660365 (+1299725us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:52.660582 (+   217us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:52.660758 (+   176us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:52.660782 (+    24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:52.660822 (+    40us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:52.660850 (+    28us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:52.660899 (+    49us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:52.661004 (+   105us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9641216}
I20260924 14:29:52.642678 28460 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:52.664611 26328 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:52.641872 28434 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:52.676883 28188 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:52.677485 28414 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:52.679124 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.624102 28449 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.664448 28460 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:29:52.682488 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.678046 28469 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260924 14:29:52.686049 28449 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:52.686487 28472 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:52.687507 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:52.685271 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:52.690027 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:52.689618 28326 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:52.692085 28326 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:52.692884 28326 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:52.665076 28367 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:52.697862 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65672.4us
I20260924 14:29:52.700014 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.701541 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.702239 26328 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:52.703104 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.695626 28472 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:52.704159 28471 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:52.704278 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:52.705508 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:52.705896 28127 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:52.706504 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:52.699919 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47745.3us
I20260924 14:29:52.714892 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:52.715598 26404 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:29:52.699776 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:52.698711 28315 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:52.729592 28127 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:52.732635 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:52.736815 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:52.745042 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.005s	sys 0.037s
W20260924 14:29:52.745585 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.006s	sys 0.037s
W20260924 14:29:52.745843 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 723695us
I20260924 14:29:52.746658 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:52.736272 28127 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:52.745011 26404 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:29:52.744166 28472 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:52.752079 28342 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:52.753412 28460 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:52.754766 27191 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:52.753496 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:52.756358 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:52.762974 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:52.763157 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:52.763696 26397 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:29:52.766877 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.763561 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.761370 28479 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:29:52.768252 28479 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:52.770813 28467 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:52.772688 28367 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:52.759397 28342 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:52.772688 26397 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:29:52.772977 28467 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.777137 28467 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:52.771347 28439 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.777606 28460 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:52.769155 28479 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:52.774441 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:52.763193 28464 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:52.781790 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 54046.3us
I20260924 14:29:52.785940 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.788087 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:52.788604 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:52.789275 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:52.789837 27181 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:52.783408 28464 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:52.791879 28464 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:52.787842 28477 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:52.796093 28477 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:52.799168 28477 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:52.800490 27181 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:52.802147 26295 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:52.803349 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.806113 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:52.806888 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:52.807108 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:52.807116 26406 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:52.807852 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.808938 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.810933 28467 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:29:52.811512 28467 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:52.812387 28467 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.813212 28454 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:52.813961 28454 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:52.814075 28439 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:52.818128 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.822058 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.815179 28454 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.838658 28477 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:52.870342 28380 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 2.203s	user 0.017s	sys 0.000s
I20260924 14:29:52.870589 28477 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:52.871644 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 330) took 2893 ms. Trace:
I20260924 14:29:52.872015 26401 rpcz_store.cc:276] 0924 14:29:49.977814 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:49.979552 (+  1738us) service_pool.cc:227] Handling call
0924 14:29:49.979973 (+   421us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:49.984416 (+  4443us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:49.984446 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:50.011831 (+ 27385us) log.cc:842] Serialized 26 byte log entry
0924 14:29:50.353435 (+341604us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:50.353496 (+    61us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:50.353594 (+    98us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:52.869753 (+2516159us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:29:52.871284 (+  1531us) raft_consensus.cc:1671] finished
0924 14:29:52.871308 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:52.871564 (+   256us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4059,"raft.queue_time_us":25849,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4948,"spinlock_wait_cycles":41000704,"thread_start_us":313544,"threads_started":1,"wal-append.queue_time_us":538014}
I20260924 14:29:52.873262 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 231) took 1808 ms. Trace:
I20260924 14:29:52.876327 26399 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:52.876955 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 852) took 1202 ms. Trace:
I20260924 14:29:52.877215 26399 rpcz_store.cc:276] 0924 14:29:51.674370 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:51.674495 (+   125us) service_pool.cc:227] Handling call
0924 14:29:52.875969 (+1201474us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:52.876159 (+   190us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:52.876730 (+   571us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:52.876908 (+   178us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6713728}
I20260924 14:29:52.873682 26327 rpcz_store.cc:276] 0924 14:29:51.065229 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:51.065356 (+   127us) service_pool.cc:227] Handling call
0924 14:29:51.065627 (+   271us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:51.116530 (+ 50903us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:29:51.373891 (+257361us) log.cc:842] Serialized 14 byte log entry
0924 14:29:51.541618 (+167727us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:51.541669 (+    51us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:51.541707 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:52.873044 (+1331337us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b54007800d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:52.873093 (+    49us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:52.873207 (+   114us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17748,"prepare.queue_time_us":257685,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":112899,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":12806,"spinlock_wait_cycles":116370816,"thread_start_us":369189,"threads_started":3,"wal-append.queue_time_us":167858,"wal-append.run_cpu_time_us":1231,"wal-append.run_wall_time_us":1047044}
I20260924 14:29:52.870443 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:52.885468 26330 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:52.886665 26223 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.023s	sys 0.034s
W20260924 14:29:52.887038 26223 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.023s	sys 0.034s
I20260924 14:29:52.890205 26401 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:29:52.890661 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:29:52.893500 28234 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:29:52.895030 28441 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:29:52.895361 28441 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:52.895371 28166 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:52.900517 28464 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:52.901039 28464 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.902273 26256 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:52.902913 28464 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:52.906955 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:29:52.903144 28331 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:52.907720 28267 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:52.908149 28331 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:52.909792 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:52.910086 28331 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:52.896148 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 237) took 1732 ms. Trace:
I20260924 14:29:52.911000 26326 rpcz_store.cc:276] 0924 14:29:51.163388 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:51.177926 (+ 14538us) service_pool.cc:227] Handling call
0924 14:29:51.327547 (+149621us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:51.531073 (+203526us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:51.536858 (+  5785us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:51.543174 (+  6316us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:51.563369 (+ 20195us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:52.895885 (+1332516us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54007d3cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:52.895943 (+    58us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:52.896089 (+   146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":420495104}
I20260924 14:29:52.911965 28267 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:52.912372 28267 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:52.912679 28197 consensus_queue.cc:1046] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:29:52.913599 28197 raft_consensus.cc:3032] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:52.913136 28267 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.913949 28197 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:52.915299 28267 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:52.915577 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.914853 28197 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.896139 28441 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:52.917761 28441 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:29:52.918963 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 471) took 1462 ms. Trace:
I20260924 14:29:52.919924 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:29:52.922400 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:52.923820 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.919313 27194 rpcz_store.cc:276] 0924 14:29:51.456591 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:52.918118 (+1461527us) service_pool.cc:227] Handling call
0924 14:29:52.918469 (+   351us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:52.918638 (+   169us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:52.918664 (+    26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:52.918704 (+    40us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:52.918730 (+    26us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:52.918780 (+    50us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:52.918908 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:52.918846 28467 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260924 14:29:52.934715 28467 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:52.935094 28467 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:52.933485 28197 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:52.935896 28467 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:52.946986 28394 raft_consensus.cc:979] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: : Instructing follower 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:52.947611 28394 raft_consensus.cc:1067] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Signalling peer 7e09fbebe29a49d291d23f9d395fe335 to start an election
I20260924 14:29:52.948671 26330 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:52.954289 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:52.957779 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.958999 28365 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:52.959692 28365 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:52.960570 28365 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:52.973078 27188 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:50480
I20260924 14:29:52.973732 27188 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:29:52.974087 27188 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:52.976572 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.977353 26249 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:52.979544 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.980191 26249 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:52.981456 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:52.983793 28464 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:52.983875 26249 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:52.984448 28464 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:52.987097 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:52.986413 28464 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:53.008546 27188 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.011353 27188 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:53.015525 28378 log.cc:924] Time spent T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Append to log took a long time: real 2.459s	user 0.000s	sys 0.023s
I20260924 14:29:53.026154 28476 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:53.026924 28476 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:53.032636 28326 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:53.037990 28476 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:53.038715 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 412660us
I20260924 14:29:53.040130 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.042115 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.042697 28316 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:53.042976 28370 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:53.044682 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:53.045599 28370 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260924 14:29:53.046226 28459 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.584s	user 0.000s	sys 0.002s
I20260924 14:29:53.046972 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.047837 28316 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:53.049321 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.050402 28429 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:29:53.046340 28370 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:53.055912 28370 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:53.063174 28467 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:53.062958 28370 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.065443 28370 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:53.107810 26253 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:29:53.109812 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:53.112138 26249 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:53.112516 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:53.125003 26323 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:53.131735 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:53.134012 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:53.137684 26254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 361) took 2248 ms (client timeout 1812 ms). Trace:
I20260924 14:29:53.138128 26252 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:53.140527 26326 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:53.149083 26369 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:53.151688 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:53.138480 26254 rpcz_store.cc:269] 0924 14:29:50.889220 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:50.891588 (+  2368us) service_pool.cc:227] Handling call
0924 14:29:53.137165 (+2245577us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400716e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:53.137396 (+   231us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":353918848}
I20260924 14:29:53.156672 27188 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:53.157398 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:53.158872 28329 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:53.169040 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 958) took 1322 ms. Trace:
I20260924 14:29:53.176244 27186 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:53.181823 26249 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:53.171125 28329 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.181110 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:53.182018 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:53.191596 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:53.193990 26252 rpcz_store.cc:276] 0924 14:29:51.846689 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:51.846892 (+   203us) service_pool.cc:227] Handling call
0924 14:29:53.137541 (+1290649us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:53.138768 (+  1227us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:53.168711 (+ 29943us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400716e50. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:53.168921 (+   210us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2967107840}
I20260924 14:29:53.162577 28202 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:53.177809 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:53.218804 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:53.219206 28431 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:53.184525 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:53.189683 28344 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:53.187840 28237 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:53.183038 28467 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:53.227248 26330 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:53.228205 26324 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:53.189316 28325 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:53.220837 28202 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:53.233166 28461 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260924 14:29:53.164685 28316 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260924 14:29:53.225117 28344 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:53.238370 26328 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:53.245314 26328 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:53.241379 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 536) took 2538 ms. Trace:
I20260924 14:29:53.236050 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:53.234380 28202 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:53.254937 28332 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:53.255574 28332 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:29:53.253306 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44987.7us
W20260924 14:29:53.254205 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 48078.1us
I20260924 14:29:53.257179 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.247788 26251 rpcz_store.cc:276] 0924 14:29:50.703087 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:50.703218 (+   131us) service_pool.cc:227] Handling call
0924 14:29:50.711196 (+  7978us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:50.734110 (+ 22914us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:50.734145 (+    35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:50.734185 (+    40us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:50.734215 (+    30us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:53.240550 (+2506335us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b54000c12d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:53.240634 (+    84us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:53.241065 (+   431us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":52142848}
W20260924 14:29:53.246061 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 64371.9us
I20260924 14:29:53.255936 28332 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:53.244148 28329 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:53.274467 26401 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:53.280519 28359 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:29:53.280402 26328 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.282835 26328 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:53.265024 28237 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.275437 28431 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:53.291795 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:53.294123 26325 raft_consensus.cc:2362] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4. Re-sending same reply.
W20260924 14:29:53.294791 26325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 869) took 3081 ms (client timeout 1929 ms). Trace:
W20260924 14:29:53.295163 26325 rpcz_store.cc:269] 0924 14:29:50.213693 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:50.266204 (+ 52511us) service_pool.cc:227] Handling call
0924 14:29:53.283973 (+3017769us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:53.294699 (+ 10726us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":380711168}
I20260924 14:29:53.300290 26323 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:53.278673 28325 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:53.312299 28384 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:53.297274 28370 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:53.314724 28384 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.317055 28344 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:53.316279 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 234) took 2217 ms. Trace:
I20260924 14:29:53.299166 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.275727 28467 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.318236 26323 rpcz_store.cc:276] 0924 14:29:51.098833 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:51.111722 (+ 12889us) service_pool.cc:227] Handling call
0924 14:29:53.122692 (+2010970us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b58006b00b0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92c9f63 00007f09b932daba 00007f09b93453af 00007f09b9346b4e 00007f09b9352228 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c
0924 14:29:53.316231 (+193539us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4131952384}
I20260924 14:29:53.323349 26332 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260924 14:29:53.320506 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 136225us
I20260924 14:29:53.317694 28431 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.333698 26397 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:29:53.334112 28237 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:53.336457 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 79522.9us
I20260924 14:29:53.340934 28393 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260924 14:29:53.344009 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.299s	user 0.036s	sys 0.089s
I20260924 14:29:53.347748 28431 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:53.352723 28344 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:53.356416 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.311s	user 0.037s	sys 0.090s
I20260924 14:29:53.355405 28325 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:29:53.362732 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 1.31564e+06us
I20260924 14:29:53.369032 28344 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.371042 28467 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:53.373730 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:53.388901 28467 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:53.392050 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:53.395857 28467 raft_consensus.cc:2741] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260924 14:29:53.431058 28467 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:53.431855 28467 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:53.433750 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.436123 28373 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:53.437601 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:53.438112 28373 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:53.439675 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.441643 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.444177 28453 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:53.444552 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.444676 28467 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:53.443189 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:53.441689 28373 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:53.451556 28373 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:53.449838 28467 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:53.452976 28467 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:53.704267 28467 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.453042 28373 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:53.711686 28384 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:53.455303 28321 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:53.464427 28237 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:53.596287 28202 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:53.610383 26324 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:53.615101 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:53.715310 28395 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:53.716230 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 101280us
I20260924 14:29:53.715981 28395 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:53.717305 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.616677 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:53.718657 28202 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:53.719331 28202 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:29:53.715341 26324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 865) took 3553 ms (client timeout 2047 ms). Trace:
I20260924 14:29:53.720041 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:53.617849 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:53.619073 28474 consensus_queue.cc:1046] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:53.622766 28383 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:29:53.658542 28344 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:53.663195 28415 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:53.665834 28441 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:53.666671 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:53.669116 27192 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:53.670044 27188 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:53.673622 27186 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:29:53.676582 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:53.679680 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 309414us
I20260924 14:29:53.680122 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:53.682530 27194 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:53.447427 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:53.719934 26324 rpcz_store.cc:269] 0924 14:29:50.161979 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:50.166025 (+  4046us) service_pool.cc:227] Handling call
0924 14:29:52.755195 (+2589170us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b1800242940. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b9389856 00007f09b92dbf7e 00007f09b9345367 00007f09b9346b4e 00007f09b9352228 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c
0924 14:29:53.219283 (+464088us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b58003db8b0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92c9f63 00007f09b932daba 00007f09b93453af 00007f09b9346b4e 00007f09b9352228 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c
0924 14:29:53.715268 (+495985us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2430478976}
I20260924 14:29:53.724511 26330 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:53.728713 26323 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:53.727999 28395 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:53.729079 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:53.729916 28395 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:29:53.730638 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:53.731227 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 894) took 1209 ms. Trace:
I20260924 14:29:53.731689 28441 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.734058 28441 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:53.735150 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:53.735404 27188 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:53.735755 28344 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:53.736018 26324 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:53.736438 28344 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.736994 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:53.731706 26331 rpcz_store.cc:276] 0924 14:29:52.521904 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:52.523814 (+  1910us) service_pool.cc:227] Handling call
0924 14:29:52.524136 (+   322us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:52.529646 (+  5510us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:52.529672 (+    26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:52.529710 (+    38us) raft_consensus.cc:1631] Marking committed up to 1
0924 14:29:52.529737 (+    27us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:53.730723 (+1200986us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b540001efd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:53.730797 (+    74us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:53.731004 (+   207us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":12156672}
I20260924 14:29:53.739094 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:53.740809 26327 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:53.738557 28344 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:53.739037 27190 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
W20260924 14:29:53.744700 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 297681us
I20260924 14:29:53.720229 28202 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:53.746160 26329 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260924 14:29:53.743348 28474 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:53.746959 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.747830 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:53.748538 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:53.756820 26250 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:53.757899 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.743350 28344 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:53.745194 28384 raft_consensus.cc:3032] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:53.759676 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:53.758016 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:53.759430 26331 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260924 14:29:53.762333 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 43610.7us
I20260924 14:29:53.761636 26255 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:53.766630 26325 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:53.760672 28384 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:53.773490 26324 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:53.783414 28187 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260924 14:29:53.785929 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:53.780627 28441 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:53.788035 26327 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:53.796624 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 243) took 1534 ms. Trace:
I20260924 14:29:53.798254 26332 rpcz_store.cc:276] 0924 14:29:52.261856 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:52.270179 (+  8323us) service_pool.cc:227] Handling call
0924 14:29:53.309975 (+1039796us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:53.324466 (+ 14491us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:53.522733 (+198267us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b54005d8754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:53.788530 (+265797us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b54005d8750. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:53.796549 (+  8019us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2225321600}
I20260924 14:29:53.798254 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.762264 28415 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:53.818063 28415 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:53.808037 28474 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:53.818970 28474 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.821193 28474 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:53.822573 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 889) took 1329 ms. Trace:
I20260924 14:29:53.808252 28431 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:53.824209 28431 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:53.807261 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.823139 26329 rpcz_store.cc:276] 0924 14:29:52.493044 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:52.510187 (+ 17143us) service_pool.cc:227] Handling call
0924 14:29:53.715522 (+1205335us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:53.751293 (+ 35771us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:53.821703 (+ 70410us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b540001e0d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:53.822348 (+   645us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540001e0d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:53.822501 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2782044288}
I20260924 14:29:53.836989 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:53.838214 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:29:53.839020 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 500096us
I20260924 14:29:53.835907 28431 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:53.840379 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.806052 28384 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:53.841997 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:53.843111 28234 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:53.843604 28359 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:53.844074 28359 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:53.844460 28359 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:53.846877 26252 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
W20260924 14:29:53.849340 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 572680us
I20260924 14:29:53.850585 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:53.851243 28352 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:53.853287 28240 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:53.853354 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:53.855275 26255 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:53.855758 26256 raft_consensus.cc:3032] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:53.856036 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.858397 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:53.815084 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.860461 26257 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 1.
I20260924 14:29:53.853691 28240 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:53.861606 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.862696 28240 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:53.864646 28240 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:53.856096 26256 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:53.869021 26256 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.870318 26256 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:53.870378 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.871337 26257 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:53.855161 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.886636 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.886911 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.887988 26251 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:29:53.834879 27185 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:29:53.891631 26258 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
W20260924 14:29:53.891892 26328 consensus_meta.cc:287] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.608s	user 0.016s	sys 0.000s
I20260924 14:29:53.895383 26328 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:53.788410 28202 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260924 14:29:53.907043 26250 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:53.905093 28377 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:53.909703 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 67884.6us
I20260924 14:29:53.910625 27125 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:29:53.912353 28492 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260924 14:29:53.906068 28326 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260924 14:29:53.915800 28431 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:53.916471 28431 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:53.917006 28431 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:53.909703 28377 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:53.920393 28332 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.649s	user 0.019s	sys 0.000s
I20260924 14:29:53.920516 28393 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:53.921404 28393 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:53.920940 28332 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:53.923794 28377 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:53.924220 26399 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:29:53.923807 28393 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:53.925300 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:53.925702 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:53.926405 28267 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:53.926797 28267 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:53.927019 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:53.927205 28267 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:53.929081 28332 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:53.929953 28234 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:29:53.935544 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:29:53.907918 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 110868us
I20260924 14:29:53.942833 26256 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:53.944381 28370 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:53.946050 28370 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:53.948143 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:53.948871 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:53.947413 28370 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:53.905920 28441 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:53.946050 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:53.959905 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:53.960641 27192 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:53.965035 26397 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:53.967155 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:53.972023 27192 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:53.966363 28384 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:53.971383 28415 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:53.989785 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.003144 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.003010 28187 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260924 14:29:54.007923 26295 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:54.010253 28495 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:54.010541 28187 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:29:54.011142 28187 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:54.012027 28441 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:54.012066 28187 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:54.014549 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:54.015805 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:54.016366 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.016566 28187 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:54.018038 28464 consensus_queue.cc:1046] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:54.019212 28464 raft_consensus.cc:3032] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:54.019620 28464 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:29:54.019680 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 77555us
I20260924 14:29:54.019936 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:54.020632 28464 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:54.021234 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:54.021942 28395 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:54.022419 28395 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:54.022789 28395 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:54.024647 28202 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:29:53.995754 28415 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:54.027884 28415 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:53.981124 28359 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:54.037377 26252 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:54.045514 28367 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:54.046284 28367 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:54.048751 28367 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:54.050679 26323 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:54.056756 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:54.059304 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:54.056807 28359 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:54.062932 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.021950 28464 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:54.068298 27182 raft_consensus.cc:2379] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:54.083132 26252 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:54.084772 28267 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:54.085203 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.086086 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.087026 28267 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:54.090063 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 111497us
I20260924 14:29:54.097064 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.101598 27192 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:54.099359 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:54.067543 28393 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
W20260924 14:29:54.104485 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 92907.4us
I20260924 14:29:54.112619 26403 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:54.093995 28329 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:54.116528 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:54.129276 27185 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:29:54.134083 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:54.136457 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:54.142133 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.143152 26369 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:54.144650 26251 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:54.146330 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:54.146549 28393 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:54.147106 28393 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:54.147814 26297 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:54.148337 27182 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:54.153100 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:54.153998 26330 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:29:54.162415 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:54.162297 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:54.173673 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:54.176362 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:54.180639 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.182574 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.185981 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.183719 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:54.191234 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.188119 26221 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.197130 27190 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:54.200107 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.205117 27125 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.207374 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.149801 28329 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:54.152989 28395 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:54.199983 28367 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:54.192759 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:54.218688 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:54.227774 28366 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:54.228550 28500 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260924 14:29:54.228781 28332 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:29:54.227877 28496 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:29:54.201557 28393 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:54.230432 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 82734.6us
I20260924 14:29:54.210886 28187 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:29:54.235119 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:54.187371 28321 consensus_queue.cc:1046] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260924 14:29:54.242290 27125 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:54.239073 27182 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:54.240056 26330 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: []
I20260924 14:29:54.227612 28494 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:54.246558 28370 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:54.246587 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.247705 28321 raft_consensus.cc:3032] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:54.246778 26077 catalog_manager.cc:5495] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 1 to 5, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:29:54.248080 28321 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:54.249084 28321 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:54.249946 28211 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:54.249351 26077 catalog_manager.cc:5495] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 1 to 4, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 4 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:29:54.250361 28321 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:54.222221 28438 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260924 14:29:54.251343 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 74013.9us
I20260924 14:29:54.251717 28438 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:54.252331 28438 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:54.220129 28367 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:29:54.255924 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 112841us
I20260924 14:29:54.214932 28267 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260924 14:29:54.262068 28453 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:29:54.262750 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.263653 28344 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:54.245759 28359 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:29:54.215561 28431 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:54.286953 26078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55200 (request call id 126) took 14767 ms (client timeout 15000 ms). Trace:
I20260924 14:29:54.246994 28494 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:54.287462 26078 rpcz_store.cc:269] 0924 14:29:39.535570 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:39.550521 (+ 14951us) service_pool.cc:227] Handling call
0924 14:29:54.286700 (+14736179us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:29:49.774077 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:29:49.969112 (+195035us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:29:49.970833 (+  1721us) write_op.cc:435] Acquired schema lock
0924 14:29:49.970861 (+    28us) tablet.cc:660] Decoding operations
0924 14:29:50.028489 (+ 57628us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:29:50.028554 (+    65us) write_op.cc:641] Partition lock acquired for write op
0924 14:29:50.028586 (+    32us) tablet.cc:683] Acquiring locks for 2 operations
0924 14:29:50.028980 (+   394us) tablet.cc:699] Row locks acquired
0924 14:29:50.028997 (+    17us) write_op.cc:260] PREPARE: finished
0924 14:29:50.029154 (+   157us) write_op.cc:270] Start()
0924 14:29:50.029278 (+   124us) write_op.cc:276] Timestamp: P: 1790260190029108 usec, L: 0
0924 14:29:50.029297 (+    19us) op_driver.cc:331] REPLICATION: starting
0924 14:29:50.029975 (+   678us) log.cc:842] Serialized 934 byte log entry
0924 14:29:53.014635 (+2984660us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:29:53.032702 (+ 18067us) op_driver.cc:444] REPLICATION: finished
0924 14:29:53.034822 (+  2120us) write_op.cc:301] APPLY: starting
0924 14:29:53.034891 (+    69us) tablet.cc:1365] starting BulkCheckPresence
0924 14:29:53.035398 (+   507us) tablet.cc:1368] finished BulkCheckPresence
0924 14:29:53.035417 (+    19us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:29:53.036880 (+  1463us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:29:53.037148 (+   268us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0924 14:29:53.037358 (+   210us) write_op.cc:312] APPLY: finished
0924 14:29:53.037592 (+   234us) log.cc:842] Serialized 28 byte log entry
0924 14:29:53.038411 (+   819us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:53.038663 (+   252us) write_op.cc:454] Released schema lock
0924 14:29:53.038697 (+    34us) write_op.cc:341] FINISH: Updating metrics
0924 14:29:53.038940 (+   243us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:53.038958 (+    18us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":378881,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":1748,"apply.run_cpu_time_us":2123,"apply.run_wall_time_us":4365,"num_ops":2,"prepare.queue_time_us":1138475,"prepare.run_cpu_time_us":33414,"prepare.run_wall_time_us":398656,"raft.queue_time_us":15885,"raft.run_cpu_time_us":2002,"raft.run_wall_time_us":2003,"replication_time_us":3003304,"spinlock_wait_cycles":35041152,"thread_start_us":610607,"threads_started":4,"wal-append.queue_time_us":437006,"wal-append.run_cpu_time_us":35080,"wal-append.run_wall_time_us":3585763}]]}
I20260924 14:29:54.277956 28341 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:54.259107 28367 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:54.313463 27190 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
W20260924 14:29:54.282805 28503 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260924 14:29:54.283743 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.252743 28373 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260924 14:29:54.292876 28494 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:54.294368 26329 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:54.317510 28373 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:54.317958 28373 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:54.294835 27181 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:29:54.250363 28211 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:54.319981 28211 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:54.278138 28383 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260924 14:29:54.296149 28395 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:54.253211 28325 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260924 14:29:54.324800 28341 raft_consensus.cc:3032] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:54.325304 28341 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:54.326655 28341 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:54.328814 28431 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:54.319008 28373 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:54.306257 28329 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:54.294450 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:54.303226 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 153050us
I20260924 14:29:54.301996 26077 catalog_manager.cc:5495] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:54.331414 28373 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:54.336062 26403 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:29:54.233309 28415 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:54.352129 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 68547.4us
I20260924 14:29:54.334548 27189 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:29:54.354362 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:54.355885 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 54462.8us
I20260924 14:29:54.358384 27181 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 7: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:54.359460 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:54.344915 28341 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:54.337554 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:54.357321 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48406.3us
I20260924 14:29:54.370411 28501 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:54.371073 28501 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:54.371748 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.372857 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.373378 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.373500 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.373775 28352 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:54.374100 28474 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:54.350972 28415 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:54.374480 28352 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:54.374818 28474 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:54.376125 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.377035 28474 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:54.357072 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:54.385519 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.386974 28513 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:29:54.387648 28513 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:54.387892 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.388108 28513 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:29:54.362020 28267 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:29:54.389360 28267 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:54.389716 28267 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:29:54.362210 28332 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260924 14:29:54.387033 28309 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:29:54.391083 28332 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:54.391178 28309 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:54.391522 28332 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:54.392300 28309 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:54.392457 28332 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:54.393618 28332 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:54.362401 28240 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:54.362680 28446 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 14:29:54.399098 28202 consensus_queue.cc:1159] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c22052275034c369b7f7dca086fbd1e" responder_term: 5 status { last_received { term: 5 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 5 index: 4 } }
I20260924 14:29:54.363153 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:54.381223 26399 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:29:54.383834 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.382748 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:54.382005 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.404237 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:54.406056 28464 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:29:54.407368 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.363822 28383 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Stepping down as leader of term 5
W20260924 14:29:54.360081 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 97147.3us
I20260924 14:29:54.349951 26077 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 1 to 5, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:29:54.418450 26250 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:54.425123 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.377295 28501 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:54.428764 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:54.422663 28352 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:29:54.422650 28377 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:54.422811 28446 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:54.423137 28383 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:54.425964 28474 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:54.353925 28367 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:54.379897 28211 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:54.446553 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 64720.6us
I20260924 14:29:54.449554 26297 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:54.450516 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:54.468976 26279 debug-util.cc:398] Leaking SignalData structure 0x7b0800251ba0 after lost signal to thread 23138
W20260924 14:29:54.470350 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08002b2320 after lost signal to thread 26115
I20260924 14:29:54.471724 28202 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:54.471367 28267 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:54.472312 28202 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:54.474288 28202 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:54.474535 28509 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.067s	user 0.000s	sys 0.003s
W20260924 14:29:54.476188 28505 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260924 14:29:54.476739 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.477540 26250 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 9
I20260924 14:29:54.478626 28267 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:54.479130 28383 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:54.479786 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:54.480693 28383 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:54.443168 28446 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:54.483767 28446 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:54.484001 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:54.484639 28367 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:54.484745 26405 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:54.486222 28438 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:54.443358 28464 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:54.487649 28464 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:54.488034 28464 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:54.488667 28438 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:54.440804 28501 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:29:54.492501 26250 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:29:54.467209 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.494027 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:54.494724 28321 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:29:54.498526 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.499661 28321 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:29:54.501488 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51025.5us
I20260924 14:29:54.502156 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:29:54.507241 26279 debug-util.cc:398] Leaking SignalData structure 0x7b080037ce20 after lost signal to thread 26282
I20260924 14:29:54.507932 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:54.507997 28384 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:54.508702 27189 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:54.508841 28384 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:54.509260 28384 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:54.509442 28461 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:54.509589 28211 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:54.509963 28461 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:54.510423 28461 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:54.506839 28187 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:29:54.511898 28187 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:54.512858 28187 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:54.514717 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:54.508181 27190 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:54.515518 27189 raft_consensus.cc:2379] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:54.517438 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:54.478621 28508 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260924 14:29:54.519680 26298 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.521114 26298 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:54.522781 28237 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:54.522934 28474 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:54.523434 28474 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:54.532555 28474 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:54.523759 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:54.537668 28446 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:54.507875 28513 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:54.530402 26279 debug-util.cc:398] Leaking SignalData structure 0x7b080037ce00 after lost signal to thread 26283
W20260924 14:29:54.542815 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08001352c0 after lost signal to thread 26356
W20260924 14:29:54.543267 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08000cd440 after lost signal to thread 26357
I20260924 14:29:54.531093 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.525321 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:29:54.547494 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08000f7720 after lost signal to thread 26430
W20260924 14:29:54.583007 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 88460.7us
I20260924 14:29:54.589017 26256 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:54.592202 26250 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260924 14:29:54.633001 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08001c1b40 after lost signal to thread 26431
I20260924 14:29:54.630640 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:54.634562 26279 debug-util.cc:398] Leaking SignalData structure 0x7b080019d3e0 after lost signal to thread 27222
W20260924 14:29:54.634967 26279 debug-util.cc:398] Leaking SignalData structure 0x7b080038fc60 after lost signal to thread 27223
W20260924 14:29:54.548939 28512 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.116s	user 0.004s	sys 0.000s
I20260924 14:29:54.556222 28446 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:54.547739 26401 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:29:54.637367 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.549247 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:54.548996 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:54.594993 28395 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260924 14:29:54.550632 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:54.595134 28513 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:54.547789 28211 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:54.531080 28438 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:54.639029 28329 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:54.651430 28441 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:29:54.549496 28332 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.115s
I20260924 14:29:54.654742 28234 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:54.657456 26328 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:29:54.553170 28377 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:54.537598 28464 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:54.669597 28211 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:54.672909 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:54.677765 26323 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 2. Current term is 3. Ops: []
I20260924 14:29:54.681877 27181 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:54.684469 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.689137 28234 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:54.689730 28234 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:54.679667 28377 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:54.621800 28461 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:54.700711 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.669596 28341 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:54.703958 26257 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:54.705418 28461 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:54.705787 28234 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:54.704015 28309 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260924 14:29:54.707139 26332 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:54.715502 28494 consensus_queue.cc:1046] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:29:54.716789 28494 raft_consensus.cc:3032] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:54.717173 28494 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:54.699857 28325 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:54.717908 28325 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:54.718312 28325 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:54.678426 28415 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:54.724234 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 87471us
I20260924 14:29:54.720660 26329 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:54.726044 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.729344 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.693850 28377 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:54.694219 28211 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:54.733625 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:54.734623 26399 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:54.741226 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:54.742106 26332 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:54.743829 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:54.723214 28344 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:54.745091 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.678578 28329 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:54.718127 28494 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:54.748916 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:54.700381 28321 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260924 14:29:54.748432 28242 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:54.748579 28359 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:54.749315 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:54.734020 28341 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:54.715564 28384 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:54.764482 28341 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:54.830087 28494 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:54.748659 28453 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:54.739195 28415 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:54.844772 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.847174 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.842078 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:54.762588 28359 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:54.850461 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:54.859633 26252 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:54.861126 26327 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260924 14:29:54.863302 28384 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:54.864312 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:54.868844 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:54.875504 27190 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:54.875872 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.878392 27185 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: []
I20260924 14:29:54.858459 26329 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:54.875945 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.838668 28341 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:54.881738 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.834218 28321 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:54.892091 26328 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:29:54.869773 28359 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:54.894373 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:54.831215 28242 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260924 14:29:54.910364 26251 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:54.857633 28202 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260924 14:29:54.909271 28242 raft_consensus.cc:3032] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Stepping down as leader of term 6
W20260924 14:29:54.909409 28527 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260924 14:29:54.960427 26328 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260924 14:29:54.966463 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 124469us
I20260924 14:29:54.971545 28464 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:54.976773 28534 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:54.992329 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:29:55.048277 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 154588us
W20260924 14:29:54.984622 28503 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260924 14:29:54.897250 28529 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
W20260924 14:29:55.055963 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 79652.3us
I20260924 14:29:54.894838 28513 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:55.085712 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:55.105952 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:55.108012 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:55.108603 26297 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:55.107380 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:55.079847 28242 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:55.114235 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:29:55.113461 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 128808us
I20260924 14:29:55.119853 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:55.116758 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:55.124166 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 128599us
I20260924 14:29:55.138339 26250 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:29:55.136510 27123 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:55.104452 28513 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:55.101716 28329 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:55.115887 28384 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:29:55.123898 28461 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:29:55.095235 28211 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:55.151355 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62727.7us
I20260924 14:29:55.155409 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:55.157778 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:55.155597 28434 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:29:55.113193 28321 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260924 14:29:55.155997 26397 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:29:55.156337 28513 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:55.127951 28367 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 14:29:55.142581 28415 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:55.162967 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:55.124167 28464 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260924 14:29:55.187623 26327 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:29:55.197456 26297 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:55.199309 28234 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:29:55.200353 28530 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260924 14:29:55.200680 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:55.202457 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:55.167717 28533 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
W20260924 14:29:55.212102 28474 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.679s	user 0.009s	sys 0.004s
I20260924 14:29:55.216446 28367 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:29:55.167096 28431 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260924 14:29:55.225198 28325 consensus_meta.cc:287] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260924 14:29:55.226558 28211 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:55.227414 26405 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.564s	user 0.009s	sys 0.000s
I20260924 14:29:55.197865 28344 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:55.261610 28526 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.420s	user 0.001s	sys 0.000s
W20260924 14:29:55.266664 28536 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260924 14:29:55.270746 28242 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:55.318081 28242 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:55.315001 28533 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:55.321245 28533 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:55.322638 28513 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:55.274860 27182 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
W20260924 14:29:55.277338 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 80041.1us
W20260924 14:29:55.282754 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 82173.9us
W20260924 14:29:55.327410 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 112677us
I20260924 14:29:55.328066 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:55.306229 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:55.311230 26405 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:55.280720 28446 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:29:55.280207 28464 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:55.316097 28544 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:55.314662 28344 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:55.330634 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:55.330389 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:55.328799 26325 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:55.355877 26329 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: []
I20260924 14:29:55.357143 26327 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:29:55.365692 28539 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:55.305541 28474 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:55.369586 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:29:55.371789 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44458.3us
I20260924 14:29:55.312281 28325 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:55.372884 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 210878us
I20260924 14:29:55.390758 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:55.354485 28431 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
W20260924 14:29:55.391634 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 255402us
I20260924 14:29:55.400791 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:55.381963 28501 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260924 14:29:55.402570 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:55.405097 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:55.405835 26258 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:55.397547 28321 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:55.405830 28326 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:55.414281 28326 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:55.399161 28309 consensus_queue.cc:1046] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260924 14:29:55.405675 28549 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:29:55.401671 28187 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260924 14:29:55.424480 28309 raft_consensus.cc:3032] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:29:55.380723 28464 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:55.426491 28474 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:55.426815 28464 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:55.398892 28211 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:55.401926 28534 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:55.427489 28211 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:55.394358 28539 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:55.428630 28341 consensus_meta.cc:287] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260924 14:29:55.429060 28341 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:55.429495 28211 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:55.430084 28539 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:55.430613 26252 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:29:55.430979 28341 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:55.393469 28359 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:55.433104 28359 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:55.433845 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:55.435820 28535 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:55.436421 28535 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:55.415288 28326 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:55.420776 28344 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:55.421212 28377 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.000s
I20260924 14:29:55.430835 28547 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260924 14:29:55.422140 28325 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:55.425084 28309 raft_consensus.cc:729] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:55.438911 28377 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:55.444327 28309 consensus_queue.cc:260] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:55.445132 28377 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:55.445856 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:55.439042 28535 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:55.450858 28321 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:55.446504 26406 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:55.451887 28321 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:55.454598 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:55.455294 26253 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 4
I20260924 14:29:55.454751 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:55.457469 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:55.459600 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:55.460402 26329 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:55.461426 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:55.463186 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:55.464877 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:55.465392 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:55.467208 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:55.464404 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 55357.8us
I20260924 14:29:55.470624 26329 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:55.471104 28344 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:55.473953 28344 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:55.471416 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:55.475467 27191 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:55.472517 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:55.473085 28494 consensus_queue.cc:1046] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:29:55.416218 28329 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:55.478690 26327 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:55.446272 28309 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:55.486678 26325 raft_consensus.cc:2354] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:55.485767 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:55.497236 26329 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:55.491164 28501 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260924 14:29:55.502686 28546 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
W20260924 14:29:55.505501 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50054.2us
I20260924 14:29:55.507432 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:55.508136 26404 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:55.435660 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:29:55.513109 28558 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:55.519276 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:55.455897 28202 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260924 14:29:55.528242 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:55.529060 26252 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:55.529184 26406 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:55.530427 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:55.529091 26254 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:55.479399 28535 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260924 14:29:55.513669 28558 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:55.517210 28267 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:55.534677 28415 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:29:55.536365 28554 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:55.538188 28187 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:29:55.540391 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:55.543859 28344 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:55.546074 26221 leader_election.cc:398] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:55.546294 26253 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:55.506597 28359 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260924 14:29:55.551520 26404 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:29:55.590613 26221 leader_election.cc:401] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:29:55.552139 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:55.559121 27189 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:29:55.590641 26327 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:29:55.572202 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:55.571403 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48390.5us
I20260924 14:29:55.593187 26399 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:29:55.593811 26331 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 5
W20260924 14:29:55.597268 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 86289.8us
I20260924 14:29:55.599200 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:55.602437 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:55.603564 28529 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:29:55.604887 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:55.605571 28529 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:55.608438 28558 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:55.608974 28558 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:55.609443 28558 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:55.579432 28267 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:55.612417 28267 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:55.613580 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:55.614208 26250 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:55.614531 28344 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4.
I20260924 14:29:55.615669 28202 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:55.616062 28202 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:55.616495 28359 raft_consensus.cc:3032] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:29:55.616865 28359 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:55.616894 28202 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:55.617641 28529 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:29:55.617759 28359 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:55.618398 28529 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:29:55.580880 28187 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:55.618818 28529 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:55.615694 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:55.619622 28187 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:55.620100 28242 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:29:55.621006 28187 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:55.622082 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:55.622246 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:55.622507 26331 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:55.623534 28502 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:55.605697 28545 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:55.623960 28502 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:55.624366 28502 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:55.624202 28545 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:55.589967 28557 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:29:55.626447 28545 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:55.619159 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:55.572372 28415 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:55.648846 28557 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:29:55.649767 28359 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:29:55.652202 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50678.3us
I20260924 14:29:55.592015 28544 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:29:55.654879 26298 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:55.611315 28558 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:55.618197 28202 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:55.618497 28240 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:29:55.667130 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:55.580761 28554 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:55.657423 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:55.673652 27181 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:29:55.671069 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:55.676124 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:55.645803 28377 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:55.676786 26403 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:29:55.678675 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:55.680009 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:55.731801 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48671.2us
I20260924 14:29:55.740579 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:55.750698 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:55.920756 27191 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:29:55.670640 28554 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:55.670285 28240 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:55.803601 28415 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:55.678871 28548 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.132s
I20260924 14:29:55.791626 28474 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:55.678000 28544 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:55.766023 28367 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:55.776969 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 122156us
I20260924 14:29:55.683941 28377 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:55.619227 28529 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:55.870520 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 126964us
I20260924 14:29:55.777642 28395 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:55.852528 28539 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:55.922582 26327 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:55.654100 28242 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:55.919639 26405 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:55.914268 28535 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:55.972589 27182 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:55.982041 26331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3a94b735387741fca7f83059b64c0917"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:55.988884 28242 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:55.968266 28367 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:55.997728 28242 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:55.998206 28367 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:55.998224 28535 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:56.001811 28535 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:56.002418 28569 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260924 14:29:55.966523 28377 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:56.015525 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:56.017198 26250 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:56.018757 26371 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:56.044559 26252 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:56.029000 28332 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:29:55.986092 28539 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:56.045475 28501 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:56.072463 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 96622us
I20260924 14:29:56.065073 28544 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:56.095472 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:56.096923 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:56.097251 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:56.098310 26330 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:56.062031 28332 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:56.092631 28576 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:56.104756 26327 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:29:56.103933 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 135923us
I20260924 14:29:56.108598 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:56.109624 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:56.029625 28240 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:56.124249 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:56.127353 27181 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260924 14:29:56.129119 28508 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.061s	user 0.003s	sys 0.012s
W20260924 14:29:56.136096 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 170637us
I20260924 14:29:56.066780 28441 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260924 14:29:56.141343 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:56.107897 28576 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:56.161608 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:56.145056 27181 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:29:56.148499 28341 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260924 14:29:56.093833 28501 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:56.132236 28187 consensus_meta.cc:287] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.004s
I20260924 14:29:56.175064 28309 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:29:56.174376 28529 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:56.176789 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 53205.7us
W20260924 14:29:56.178787 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 52502.1us
I20260924 14:29:56.181811 26224 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:56.186430 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:56.191778 26330 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
W20260924 14:29:56.191934 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 56399.9us
I20260924 14:29:56.194295 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:29:56.194535 28571 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260924 14:29:56.199754 28578 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:56.200137 28539 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:29:56.200320 28202 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
I20260924 14:29:56.200975 28556 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260924 14:29:56.199782 28242 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:29:56.202623 28242 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:56.203001 28242 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:29:56.146045 28395 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:56.206209 28395 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:56.184469 28321 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:29:56.200341 28578 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:56.207757 28321 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:56.207901 28578 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:56.208207 28321 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:29:56.212385 28575 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260924 14:29:56.213732 28309 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:56.214743 28309 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:56.216603 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:56.228823 27181 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:29:56.230083 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:56.172353 28576 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:29:56.263221 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 127486us
W20260924 14:29:56.244553 28502 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.001s
W20260924 14:29:56.276042 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 163696us
I20260924 14:29:56.329679 26080 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:56.332217 26080 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 4 to 5, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 5 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:29:56.264762 28539 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:56.389528 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:56.390419 26400 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
W20260924 14:29:56.350078 28570 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
W20260924 14:29:56.380932 28571 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260924 14:29:56.405319 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:56.185187 28332 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:56.404646 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:56.415887 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:56.381309 28539 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:56.334010 26080 catalog_manager.cc:5495] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 3 to 6, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:29:56.388485 28529 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:56.462409 26331 raft_consensus.cc:595] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:29:56.321411 28502 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:56.428166 28583 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:56.430652 28539 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:56.469895 26080 catalog_manager.cc:5495] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 1 to 4, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:29:56.428300 28453 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:29:56.481750 26077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 138) took 14009 ms (client timeout 15000 ms). Trace:
W20260924 14:29:56.482041 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 164140us
I20260924 14:29:56.473618 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:56.532008 27182 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:29:56.487661 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 46714.8us
I20260924 14:29:56.496762 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:29:56.471545 28561 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260924 14:29:56.534384 26080 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:56.535424 26400 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:56.546341 27126 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:29:56.540117 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 202174us
I20260924 14:29:56.579201 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 445) took 1114 ms. Trace:
W20260924 14:29:56.581905 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 176719us
I20260924 14:29:56.579774 26257 rpcz_store.cc:276] 0924 14:29:55.464457 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:55.464583 (+   126us) service_pool.cc:227] Handling call
0924 14:29:55.495633 (+ 31050us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:55.552855 (+ 57222us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:55.552889 (+    34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:55.553346 (+   457us) log.cc:842] Serialized 26 byte log entry
0924 14:29:55.674850 (+121504us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:55.674888 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:55.675157 (+   269us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:56.578985 (+903828us) raft_consensus.cc:1671] finished
0924 14:29:56.579009 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:56.579131 (+   122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":24804,"raft.queue_time_us":58138,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":57,"thread_start_us":121154,"threads_started":1,"wal-append.queue_time_us":369925}
I20260924 14:29:56.583459 28501 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:29:56.531004 26077 rpcz_store.cc:269] 0924 14:29:42.472127 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:42.472431 (+   304us) service_pool.cc:227] Handling call
0924 14:29:56.481600 (+14009169us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:29:54.636560 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:29:54.703205 (+ 66645us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:29:54.716320 (+ 13115us) write_op.cc:435] Acquired schema lock
0924 14:29:54.716346 (+    26us) tablet.cc:660] Decoding operations
0924 14:29:54.717081 (+   735us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:29:54.717636 (+   555us) write_op.cc:641] Partition lock acquired for write op
0924 14:29:54.717676 (+    40us) tablet.cc:683] Acquiring locks for 4 operations
0924 14:29:54.718192 (+   516us) tablet.cc:699] Row locks acquired
0924 14:29:54.718229 (+    37us) write_op.cc:260] PREPARE: finished
0924 14:29:54.731578 (+ 13349us) write_op.cc:270] Start()
0924 14:29:54.747069 (+ 15491us) write_op.cc:276] Timestamp: P: 1790260194718334 usec, L: 0
0924 14:29:54.748379 (+  1310us) op_driver.cc:331] REPLICATION: starting
0924 14:29:54.749188 (+   809us) log.cc:842] Serialized 1717 byte log entry
0924 14:29:55.166387 (+417199us) op_driver.cc:444] REPLICATION: finished
0924 14:29:55.414598 (+248211us) write_op.cc:301] APPLY: starting
0924 14:29:55.414702 (+   104us) tablet.cc:1365] starting BulkCheckPresence
0924 14:29:55.414927 (+   225us) tablet.cc:1368] finished BulkCheckPresence
0924 14:29:55.415071 (+   144us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:29:55.420607 (+  5536us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:29:55.421012 (+   405us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0924 14:29:55.421108 (+    96us) write_op.cc:312] APPLY: finished
0924 14:29:55.421655 (+   547us) log.cc:842] Serialized 40 byte log entry
0924 14:29:55.421916 (+   261us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:55.422019 (+   103us) write_op.cc:454] Released schema lock
0924 14:29:55.422072 (+    53us) write_op.cc:341] FINISH: Updating metrics
0924 14:29:55.427628 (+  5556us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:55.427657 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":186456,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":247886,"apply.run_cpu_time_us":3619,"apply.run_wall_time_us":13380,"num_ops":4,"prepare.queue_time_us":121305,"prepare.run_cpu_time_us":48019,"prepare.run_wall_time_us":336119,"raft.queue_time_us":247226,"raft.run_cpu_time_us":37399,"raft.run_wall_time_us":239851,"replication_time_us":417893,"thread_start_us":641672,"threads_started":4,"wal-append.queue_time_us":144956,"wal-append.run_cpu_time_us":73075,"wal-append.run_wall_time_us":1524197}]]}
W20260924 14:29:56.594751 28377 consensus_meta.cc:287] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.001s
W20260924 14:29:56.595703 28587 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260924 14:29:56.596088 26080 catalog_manager.cc:5495] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 5, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:29:56.591545 27182 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:29:56.610098 28377 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:56.611406 26397 raft_consensus.cc:2354] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6.
I20260924 14:29:56.612304 26326 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:29:56.616266 27192 raft_consensus.cc:2354] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6.
W20260924 14:29:56.600852 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52781.9us
I20260924 14:29:56.636044 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:56.633278 26080 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 4 to 5, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 5 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:29:56.647372 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:29:56.649683 28341 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:29:56.650013 28240 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:56.650369 28341 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:56.658380 28600 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:56.659026 28377 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:56.663385 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:56.659157 28600 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:56.665213 28341 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:56.669132 28600 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:56.670622 28321 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:56.671180 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:56.496280 28583 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:56.734061 26369 leader_election.cc:398] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6.
I20260924 14:29:56.735123 26080 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 7, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 7 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
W20260924 14:29:56.676888 28554 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.002s
I20260924 14:29:56.681187 28542 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260924 14:29:56.422734 28393 consensus_queue.cc:1159] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f59f3234edf84aab852fa989fb754d31" responder_term: 6 status { last_received { term: 5 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:29:56.686364 28415 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:56.687520 26400 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:56.701339 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 422) took 1051 ms. Trace:
I20260924 14:29:56.714586 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 589) took 1051 ms. Trace:
I20260924 14:29:56.718498 28602 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:56.643565 28578 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:56.753062 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:56.744560 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 112343us
I20260924 14:29:56.749437 26397 rpcz_store.cc:276] 0924 14:29:55.649848 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:55.651595 (+  1747us) service_pool.cc:227] Handling call
0924 14:29:56.701049 (+1049454us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b54006e6ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:56.701230 (+   181us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2151167744}
I20260924 14:29:56.750291 26253 rpcz_store.cc:276] 0924 14:29:55.662836 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:55.662982 (+   146us) service_pool.cc:227] Handling call
0924 14:29:55.663292 (+   310us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:55.671768 (+  8476us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:55.671815 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:55.737224 (+ 65409us) log.cc:842] Serialized 26 byte log entry
0924 14:29:55.926839 (+189615us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b18003dfd20. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:29:56.128722 (+201883us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:56.145328 (+ 16606us) log.cc:842] Serialized 14 byte log entry
0924 14:29:56.145768 (+   440us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:56.145894 (+   126us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:56.325591 (+179697us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b18003dfd20. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:29:56.714352 (+388761us) raft_consensus.cc:1671] finished
0924 14:29:56.714376 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:56.714510 (+   134us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1272,"prepare.queue_time_us":20647,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":4367,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":54353,"spinlock_wait_cycles":50835456,"thread_start_us":201645,"threads_started":1,"wal-append.queue_time_us":622524}
I20260924 14:29:56.753203 27185 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:56.760473 26080 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 3, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 3 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:29:56.762591 26258 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:56.746949 28542 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:56.773335 28542 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:56.774138 28583 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:56.774969 28370 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:56.777020 28367 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:29:56.780165 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:56.780277 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:56.780972 26400 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:56.747179 28554 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:56.788901 27191 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:56.789418 28554 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:56.790159 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:56.791488 28578 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:56.791215 28202 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:56.792023 28202 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:56.792346 28202 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:56.793330 27182 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:56.801193 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:56.825155 26080 catalog_manager.cc:5495] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:29:56.837090 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47021.3us
I20260924 14:29:56.838716 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:56.836966 26080 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 4, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:29:56.842133 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:56.842846 26401 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:29:56.846798 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 263401us
I20260924 14:29:56.852361 28609 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:56.852747 27191 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:56.852985 28609 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:56.836179 28595 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:56.854295 28595 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:56.854662 28595 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:56.854836 28542 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:56.733372 28242 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:56.855185 28609 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:56.857167 28542 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:56.836472 28332 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:56.857662 28242 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:56.858441 28242 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:56.859010 28535 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:29:56.859413 28242 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:56.860399 28601 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:29:56.860785 28601 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:56.862872 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:29:56.869796 26400 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:29:56.871644 27126 leader_election.cc:398] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6.
I20260924 14:29:56.872454 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:56.873242 26253 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:56.873745 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:56.879098 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:56.879878 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:56.880699 26406 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:56.879904 26397 raft_consensus.cc:2354] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6.
I20260924 14:29:56.882642 27126 leader_election.cc:398] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6.
I20260924 14:29:56.883169 27126 leader_election.cc:401] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260924 14:29:56.870337 28601 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:56.883671 28373 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:29:56.883924 28601 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:29:56.887125 28344 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:56.887746 28344 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:56.889259 27182 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:56.897027 26406 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:29:56.900887 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:56.904748 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:56.905262 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:29:56.905982 27126 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:56.907555 26326 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 2: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:56.908272 26324 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:56.909041 26324 raft_consensus.cc:595] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:56.909436 26324 raft_consensus.cc:603] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:29:56.915880 26258 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:29:56.920230 26253 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:29:56.924903 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:29:56.974074 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 57094.9us
I20260924 14:29:56.759779 28602 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:56.985637 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:56.982899 26400 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:57.023813 26224 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:57.037463 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:57.044636 27185 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:29:57.051908 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:29:57.054852 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 441587us
I20260924 14:29:56.920738 28321 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:56.975435 28453 raft_consensus.cc:2741] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260924 14:29:57.077353 26328 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:56.917205 28393 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:56.991895 28242 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:57.098584 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:57.099350 26253 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:29:57.120950 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 49718.3us
I20260924 14:29:57.134264 26253 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:57.028641 28237 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:57.136476 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:57.076324 28494 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:57.077121 28502 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:57.099606 28558 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:29:57.084337 28321 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:57.162072 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:57.124262 28610 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:57.158345 26402 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:57.097776 28344 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:57.120785 28341 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:57.162894 26404 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:29:57.124138 28446 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:57.151883 26326 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:29:57.167832 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:29:57.882122 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 728981us
I20260924 14:29:57.894011 26324 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:57.144282 28395 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:57.169255 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:29:57.896133 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 726938us
I20260924 14:29:57.149878 28596 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:29:57.173849 26328 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260924 14:29:57.197178 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 217202us
I20260924 14:29:57.230288 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:57.157258 28187 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:57.122316 28267 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260924 14:29:57.100569 28393 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:57.136811 28237 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:57.172888 28558 raft_consensus.cc:686] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:29:57.303573 28541 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:57.262151 28326 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:57.160620 28494 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:57.985075 28494 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:57.200181 28613 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6.
I20260924 14:29:57.173121 28321 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:57.206398 28341 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:57.175539 28610 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:57.596367 26401 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.000s
I20260924 14:29:57.674012 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 317) took 1015 ms. Trace:
I20260924 14:29:57.628926 28529 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:57.293771 28576 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.972s	user 0.007s	sys 0.007s
I20260924 14:29:57.287990 28600 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:29:57.407837 28615 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260924 14:29:57.408636 28614 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
W20260924 14:29:57.323578 28539 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.785s	user 0.007s	sys 0.004s
W20260924 14:29:57.458161 28620 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260924 14:29:58.103310 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 173076us
W20260924 14:29:57.731490 28595 consensus_meta.cc:287] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.814s	user 0.006s	sys 0.006s
W20260924 14:29:57.693783 28617 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
W20260924 14:29:57.692855 28616 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260924 14:29:57.987989 28332 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:57.458269 28609 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:57.406862 28619 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:57.474318 28453 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:57.768651 28464 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:57.974812 28611 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:57.971279 28187 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:57.924824 28513 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.233395 28513 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.234611 28321 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:57.970122 28474 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:57.981735 28558 consensus_queue.cc:237] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:57.303740 28605 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.243882 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 932) took 2059 ms. Trace:
I20260924 14:29:58.026152 28529 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:57.982914 28326 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:57.903899 28415 consensus_meta.cc:287] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.144s	user 0.010s	sys 0.010s
I20260924 14:29:57.845155 28606 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:57.333308 28367 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:57.908083 26331 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:29:57.463778 28242 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:57.291473 28592 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 1.051s
I20260924 14:29:58.107648 28539 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:57.989775 26401 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
W20260924 14:29:58.279753 26401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 442) took 1438 ms (client timeout 1737 ms). Trace:
I20260924 14:29:58.279886 26405 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:29:57.990144 27182 rpcz_store.cc:276] 0924 14:29:56.658756 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:56.660776 (+  2020us) service_pool.cc:227] Handling call
0924 14:29:57.673724 (+1012948us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54000771d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:57.673891 (+   167us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":281218304}
I20260924 14:29:58.093883 26326 raft_consensus.cc:595] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:57.355396 28524 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:57.911137 28446 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:57.807633 28373 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.923s	user 0.010s	sys 0.003s
I20260924 14:29:57.911379 28395 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.103235 28588 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.105226 28600 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.107151 28434 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.114735 28599 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.157629 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:29:57.988879 28541 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.206115 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 592) took 1305 ms. Trace:
I20260924 14:29:58.213011 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.112154 28626 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:29:58.148010 28535 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.100757 28576 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:58.244656 26327 rpcz_store.cc:276] 0924 14:29:56.184598 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:56.184796 (+   198us) service_pool.cc:227] Handling call
0924 14:29:56.196215 (+ 11419us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:56.196536 (+   321us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:56.196562 (+    26us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:56.433789 (+237227us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b2c001e1fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:29:56.434253 (+   464us) log.cc:842] Serialized 26 byte log entry
0924 14:29:56.477224 (+ 42971us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b48000c21d8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:29:56.739942 (+262718us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:56.739997 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:56.770410 (+ 30413us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:58.243602 (+1473192us) raft_consensus.cc:1671] finished
0924 14:29:58.243635 (+    33us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:58.243813 (+   178us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":53355,"raft.queue_time_us":16644,"raft.run_cpu_time_us":4006,"raft.run_wall_time_us":264196,"spinlock_wait_cycles":117192704,"thread_start_us":213071,"threads_started":1,"wal-append.queue_time_us":451594}
I20260924 14:29:58.151592 28596 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:29:58.274830 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 299) took 1103 ms. Trace:
I20260924 14:29:58.193601 28602 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:57.996270 26324 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:58.293525 28618 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:58.195652 28533 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:58.278065 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 316) took 1863 ms. Trace:
I20260924 14:29:58.333760 28187 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:58.345541 27189 rpcz_store.cc:276] 0924 14:29:56.409032 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:56.409176 (+   144us) service_pool.cc:227] Handling call
0924 14:29:56.413195 (+  4019us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:56.464197 (+ 51002us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:56.464330 (+   133us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:56.468641 (+  4311us) log.cc:842] Serialized 26 byte log entry
0924 14:29:57.040589 (+571948us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:57.040637 (+    48us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:57.040705 (+    68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:58.272372 (+1231667us) raft_consensus.cc:1671] finished
0924 14:29:58.272392 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:58.272524 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":49825,"raft.queue_time_us":54467,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":439882,"threads_started":1,"wal-append.queue_time_us":593092}
I20260924 14:29:58.235401 28321 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.277578 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 389) took 1853 ms. Trace:
W20260924 14:29:58.280016 26401 rpcz_store.cc:269] 0924 14:29:56.841619 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:56.841834 (+   215us) service_pool.cc:227] Handling call
0924 14:29:58.279705 (+1437871us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:58.153704 28332 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:58.156754 28609 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:58.278321 28601 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.394s	user 0.006s	sys 0.007s
I20260924 14:29:58.289675 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32930 (request call id 16) took 1135 ms. Trace:
I20260924 14:29:58.258664 28593 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.113566 28595 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:58.272473 28367 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:29:58.209173 28464 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.277369 28544 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.237605 28611 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.223472 28453 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.202076 28619 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:58.251797 28474 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.289762 28373 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:29:58.297072 28588 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.282092 28583 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.334306 26331 rpcz_store.cc:276] 0924 14:29:57.171382 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:57.171591 (+   209us) service_pool.cc:227] Handling call
0924 14:29:57.907631 (+736040us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:58.274496 (+366865us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:58.274754 (+   258us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:58.289613 28415 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.288133 28606 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.333027 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 302) took 1100 ms. Trace:
I20260924 14:29:58.520304 26325 rpcz_store.cc:276] 0924 14:29:57.229468 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:57.231041 (+  1573us) service_pool.cc:227] Handling call
0924 14:29:58.157477 (+926436us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:29:58.329341 (+171864us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":10225920}
I20260924 14:29:58.336037 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 934) took 1194 ms. Trace:
I20260924 14:29:58.521696 26324 rpcz_store.cc:276] 0924 14:29:57.141267 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:57.141424 (+   157us) service_pool.cc:227] Handling call
0924 14:29:58.335964 (+1194540us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:29:58.353195 28533 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.336046 26255 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:29:58.337143 28535 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:58.285398 28524 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:58.341418 28627 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:29:58.288456 28513 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.299530 26328 rpcz_store.cc:276] 0924 14:29:56.900247 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:56.900375 (+   128us) service_pool.cc:227] Handling call
0924 14:29:56.916754 (+ 16379us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:57.903704 (+986950us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:58.205734 (+302030us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54007d3cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:58.206028 (+   294us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":369031424}
I20260924 14:29:58.279089 28242 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.298532 28592 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
W20260924 14:29:58.349645 28621 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260924 14:29:58.353379 26406 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:58.293833 28578 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:58.257019 28605 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.425606 28644 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:58.347702 27181 rpcz_store.cc:276] 0924 14:29:56.404563 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:56.405896 (+  1333us) service_pool.cc:227] Handling call
0924 14:29:56.406946 (+  1050us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:56.530602 (+123656us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:56.530635 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:56.531067 (+   432us) log.cc:842] Serialized 26 byte log entry
0924 14:29:57.046042 (+514975us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:57.046103 (+    61us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:57.046205 (+   102us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:58.257779 (+1211574us) raft_consensus.cc:1671] finished
0924 14:29:58.257821 (+    42us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:58.257989 (+   168us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":118361,"raft.queue_time_us":125134,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":33882,"thread_start_us":414411,"threads_started":1,"wal-append.queue_time_us":702330}
I20260924 14:29:58.421586 26326 rpcz_store.cc:276] 0924 14:29:57.150517 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:57.151727 (+  1210us) service_pool.cc:227] Handling call
0924 14:29:58.093631 (+941904us) spinlock_profiling.cc:243] Waited 9.93 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a7330 00007f09be292dc9 00007f09be29268c 00007f09b7985d04
0924 14:29:58.285987 (+192356us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":22836992}
I20260924 14:29:58.426369 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:58.336257 28630 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:58.343561 28393 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:58.436753 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 91139.9us
I20260924 14:29:58.531878 28441 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:29:58.451236 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.002s	sys 0.001s
W20260924 14:29:58.451941 26369 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.005s	sys 0.009s
W20260924 14:29:58.452790 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 258262us
I20260924 14:29:58.420476 28309 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.300191 28434 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.462924 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 917) took 1434 ms. Trace:
I20260924 14:29:58.427160 28560 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.603607 28560 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:58.579722 28513 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.347565 28321 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.432375 28341 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:58.605134 28513 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:58.197547 28237 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.440550 28591 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.606520 28591 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:58.525105 28415 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:58.583461 28446 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.445911 28581 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.608271 28581 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:58.609192 28415 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:58.327355 28599 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.610059 28638 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:58.610730 28638 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:58.611718 28632 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.612238 28632 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.615185 28434 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.616364 28309 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.616783 28513 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:58.617549 28513 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.618114 28513 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:29:58.553228 26296 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.005s	sys 0.045s
W20260924 14:29:58.619127 26296 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.006s	sys 0.045s
I20260924 14:29:58.583339 28630 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.600029 28601 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.621098 28601 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.623030 28601 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.348234 28618 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.471226 28542 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:58.560969 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 300) took 1360 ms. Trace:
I20260924 14:29:58.475191 28326 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.505601 28623 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.533360 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.641433 26325 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:29:58.454692 28544 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.573139 28535 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:58.572961 28644 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:58.575559 26372 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.016s	sys 0.066s
W20260924 14:29:58.650153 26372 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.017s	sys 0.067s
I20260924 14:29:58.579700 27185 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:29:58.344524 28626 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:58.438692 28593 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.594497 26406 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:58.554519 28234 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:58.595818 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.004s	sys 0.001s
I20260924 14:29:58.623508 28513 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:58.552647 28647 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:29:58.580336 27126 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 1.505s (SENT) [suppressed 9 similar messages]
I20260924 14:29:58.618546 28309 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:29:58.712565 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 65157.3us
I20260924 14:29:58.460824 28610 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.453523 28631 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:58.715575 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.556396 28592 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:29:58.599121 26405 rpcz_store.cc:276] 0924 14:29:57.021893 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:57.022210 (+   317us) service_pool.cc:227] Handling call
0924 14:29:58.279702 (+1257492us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:58.282882 (+  3180us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:58.455641 (+172759us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54006e78d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:58.455783 (+   142us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2804695936}
I20260924 14:29:58.624269 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 440) took 1844 ms. Trace:
I20260924 14:29:58.712590 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:58.528357 28202 consensus_meta.cc:287] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.731s	user 0.014s	sys 0.010s
I20260924 14:29:58.521840 28373 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.593174 28393 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.530778 28583 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:58.638893 26222 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.025s	sys 0.036s
W20260924 14:29:58.733167 26222 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.026s	sys 0.037s
I20260924 14:29:58.571369 28539 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:58.636039 26323 rpcz_store.cc:276] 0924 14:29:57.200281 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:57.200460 (+   179us) service_pool.cc:227] Handling call
0924 14:29:58.560342 (+1359882us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:29:58.560513 (+   171us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:58.560678 (+   165us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:58.560702 (+    24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:58.560737 (+    35us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:58.560781 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:58.560824 (+    43us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:58.560918 (+    94us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":10227456}
I20260924 14:29:58.643998 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.646162 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:58.566210 28529 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.656035 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:58.661304 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:58.598289 26369 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.006s	sys 0.009s
I20260924 14:29:58.635461 28501 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:58.580099 28578 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.674654 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.530406 28596 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:58.637048 28601 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.584467 28545 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260924 14:29:58.681454 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:58.645587 28600 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.699807 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:58.856318 28309 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.857193 28309 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.858384 28578 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.650028 28326 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:58.859828 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:58.623735 28441 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:58.860172 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.861024 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:58.861254 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.861624 27186 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:58.862287 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.862535 28441 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:58.862846 26253 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:29:58.719005 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.863624 26249 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:29:58.855599 28601 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.727099 26256 raft_consensus.cc:2396] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260924 14:29:58.728117 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:58.641795 28542 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:58.683053 28332 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.682174 28434 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:58.651072 28541 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.708371 28535 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.733286 28560 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:58.844271 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.846252 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.910063 26250 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:58.847554 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:58.911803 27185 raft_consensus.cc:2396] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260924 14:29:58.841737 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 195615us
I20260924 14:29:58.848224 28625 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:58.848786 28637 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:58.849068 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.849260 28650 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.843884 28638 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.647135 28623 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.849628 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.850005 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.850581 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.850103 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.841568 28606 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:58.845467 28595 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.733709 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.733503 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.717427 26403 rpcz_store.cc:276] 0924 14:29:56.780091 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:56.781506 (+  1415us) service_pool.cc:227] Handling call
0924 14:29:56.781827 (+   321us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:56.782160 (+   333us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:56.782187 (+    27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:56.782589 (+   402us) log.cc:842] Serialized 26 byte log entry
0924 14:29:57.883094 (+1100505us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:29:57.883136 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:57.883198 (+    62us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:29:58.624057 (+740859us) raft_consensus.cc:1671] finished
0924 14:29:58.624077 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:58.624212 (+   135us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"thread_start_us":392327,"threads_started":1,"wal-append.queue_time_us":1453290}
I20260924 14:29:58.744427 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.850078 28641 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.855597 28594 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:58.700701 28234 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:58.720098 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 298) took 1555 ms. Trace:
I20260924 14:29:58.699057 28635 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:58.698280 28237 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:58.869138 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.732995 28202 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:58.876621 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.849110 28619 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:58.880417 28629 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:29:58.880676 28474 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:58.880465 28646 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.874950 28630 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.881435 28453 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:58.882166 28601 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.884730 28593 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.885519 28529 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.885581 28636 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:58.886060 28618 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.885982 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.851416 28513 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:58.896463 28524 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:58.869545 28501 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:58.851286 28576 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.851119 28632 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.851426 28581 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:58.901013 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:29:58.902714 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:58.899067 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 171063us
I20260924 14:29:58.816214 28464 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.817521 28588 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:58.953024 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:29:58.815716 28609 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:58.729812 28267 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:58.846679 28605 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:58.832142 28373 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:58.859935 28326 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.856422 27194 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:29:58.735038 28599 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:58.715796 28647 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:58.675590 28626 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:59.073623 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:29:59.058249 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 358752us
I20260924 14:29:59.022867 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.354583 26330 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
W20260924 14:29:59.065526 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 485166us
I20260924 14:29:59.178545 26257 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:29:59.165617 26298 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:59.179847 26252 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:29:58.957070 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.176620 26332 rpcz_store.cc:276] 0924 14:29:57.164776 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:57.165088 (+   312us) service_pool.cc:227] Handling call
0924 14:29:57.262734 (+ 97646us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:57.992671 (+729937us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:57.992716 (+    45us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:57.992768 (+    52us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:29:57.992822 (+    54us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:58.719876 (+727054us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54005bb8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:58.719924 (+    48us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:58.720034 (+   110us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1678024448}
I20260924 14:29:59.177439 26324 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:58.731545 28631 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:58.867309 28591 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.222690 28641 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:58.870584 28611 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.878170 28592 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:59.291755 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.297773 26254 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260924 14:29:59.224751 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48594.4us
I20260924 14:29:59.317752 26256 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 7
I20260924 14:29:59.320524 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.953536 28539 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:59.324055 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:58.873967 28393 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:59.327800 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:58.918614 28434 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:59.188227 28629 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:59.337652 28373 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:59.290902 28606 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:59.339922 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:59.340174 26255 raft_consensus.cc:2396] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260924 14:29:59.341179 28326 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:58.978709 28637 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:59.046897 28560 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:59.350378 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:58.957371 28533 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:59.162657 28542 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.057796 28309 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:59.212888 28359 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:59.067778 28332 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:29:59.358114 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 284572us
I20260924 14:29:59.232362 28529 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:29:59.356650 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 191094us
I20260924 14:29:59.362039 27126 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:29:59.305586 28501 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:58.951462 28535 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:59.213743 28383 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.147s
I20260924 14:29:59.211479 28237 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:59.211400 28635 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:59.241904 28638 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:59.164546 28321 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:59.235083 28513 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:29:59.427240 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 46265.6us
W20260924 14:29:59.433103 26258 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.024s	sys 0.067s
W20260924 14:29:59.595392 26258 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.031s	sys 0.069s
I20260924 14:29:59.598448 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.427433 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:59.600121 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.600953 26400 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:29:59.603386 26323 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:29:59.550159 26295 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:59.427043 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.607601 26330 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:29:59.221807 28594 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:59.568617 26251 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:29:59.546887 26327 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:29:59.611318 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.323846 28639 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:59.225989 28646 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:59.449249 28583 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:59.380040 28599 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:29:59.609356 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.610013 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 479) took 1012 ms. Trace:
I20260924 14:29:59.233171 28650 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:59.360911 28464 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.253s
I20260924 14:29:59.613991 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:59.614118 28619 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:29:59.642144 28619 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:59.642634 28619 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:59.645413 28395 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:59.614270 28641 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:59.568184 28606 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:59.649636 28606 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:59.651583 28594 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.651979 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 579) took 1011 ms. Trace:
I20260924 14:29:59.652906 28626 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.654237 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.654677 27123 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:29:59.655370 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.655792 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:59.656004 26324 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:29:59.656726 28610 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:59.657109 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.657274 28542 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260924 14:29:59.657771 28634 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.234s	user 0.003s	sys 0.002s
I20260924 14:29:59.615084 28547 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:59.616727 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.674253 26397 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:29:59.617930 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:59.618768 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.521371 28644 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.619254 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:59.675519 27194 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:29:59.612879 28539 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:59.579350 28629 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:29:59.619618 28588 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:29:59.620364 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.691426 26405 raft_consensus.cc:2354] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:29:59.686826 28539 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:59.620361 28434 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:59.572041 28332 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:59.622346 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:59.623878 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.624336 26325 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:59.462060 28372 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:29:59.625087 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:59.625306 26256 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
W20260924 14:29:59.576370 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 214507us
I20260924 14:29:59.627020 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.679812 28513 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:59.047252 28625 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:29:59.326424 28267 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:59.632791 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.712796 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.713205 26328 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 2.
I20260924 14:29:59.713449 26405 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:29:59.632452 26403 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:29:59.714111 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.636109 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.714613 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.690147 28623 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.657965 28593 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:29:59.657799 28542 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:59.658695 28631 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:59.659996 28650 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:59.624027 28630 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
W20260924 14:29:59.614511 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 64465.5us
I20260924 14:29:59.591207 28638 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:59.672446 28599 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:29:59.673015 28611 raft_consensus.cc:979] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:29:59.652371 27190 rpcz_store.cc:276] 0924 14:29:58.640793 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:58.640956 (+   163us) service_pool.cc:227] Handling call
0924 14:29:58.665005 (+ 24049us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:58.863656 (+198651us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:58.863688 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:58.863726 (+    38us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:58.863764 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:59.651750 (+787986us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54006c5054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:59.651796 (+    46us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:59.651927 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":456320256}
I20260924 14:29:59.632812 26258 rpcz_store.cc:276] 0924 14:29:58.597969 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:58.599728 (+  1759us) service_pool.cc:227] Handling call
0924 14:29:58.600049 (+   321us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:58.600307 (+   258us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:29:58.904225 (+303918us) log.cc:842] Serialized 14 byte log entry
0924 14:29:59.595930 (+691705us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:59.595999 (+    69us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:29:59.596060 (+    61us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:59.596144 (+    84us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:59.609948 (+ 13804us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":700841,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":6984,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":6590,"thread_start_us":992272,"threads_started":2}
I20260924 14:29:59.686228 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:29:59.636384 28309 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:59.694453 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:29:59.673156 28202 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:59.695906 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.700455 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.703387 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:59.657232 28610 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:59.590903 28560 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:59.764386 27194 raft_consensus.cc:2354] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:29:59.590518 28359 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:59.710597 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.223951 28636 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:59.641425 28646 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:59.715191 26330 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:29:59.549722 28558 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260924 14:29:59.736939 28648 raft_consensus.cc:1067] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:29:59.715866 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:59.630753 28639 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:59.717603 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.717694 26298 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:59.683082 28547 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:59.709137 28372 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:29:59.801251 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 83601.3us
I20260924 14:29:59.721123 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.721593 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.704861 28234 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:59.736241 26295 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:59.720171 28593 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:59.737608 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.735793 28187 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:29:59.739912 28332 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:59.719897 28542 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:29:59.722316 28513 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:59.736176 28647 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.741580 28599 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:59.806226 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:29:59.807413 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.808553 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.809134 26403 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:29:59.809401 28611 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:29:59.809790 28611 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:29:59.809906 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.810155 28611 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:29:59.810983 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.811848 28187 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:59.812412 28513 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:29:59.813941 28187 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:59.814430 28513 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:59.735735 28309 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:29:59.815165 28637 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:59.816344 28309 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:59.742734 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.742905 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:59.817230 27190 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:29:59.817581 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.818257 26397 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:29:59.743412 26402 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:29:59.819068 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.744057 28638 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:59.791247 28539 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:29:59.746209 28367 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.292s	user 0.012s	sys 0.000s
W20260924 14:29:59.746043 27192 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 396) took 2752 ms (client timeout 1752 ms). Trace:
I20260924 14:29:59.746006 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 923) took 1160 ms. Trace:
I20260924 14:29:59.747946 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 572) took 1451 ms. Trace:
I20260924 14:29:59.748805 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 401) took 1166 ms. Trace:
I20260924 14:29:59.751098 28544 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:59.752262 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.752131 28629 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:29:59.754359 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.755252 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 925) took 1053 ms. Trace:
I20260924 14:29:59.756947 28267 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.757510 28630 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:29:59.765257 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 70898.8us
I20260924 14:29:59.766448 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:59.768559 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.766193 28524 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:29:59.715327 26400 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:29:59.777657 28202 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:59.785312 28650 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:59.788312 28583 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:59.788746 28359 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:29:59.789458 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.789819 28586 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:59.790462 28529 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:29:59.778093 28393 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:29:59.791782 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:29:59.792552 28660 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:59.792706 28581 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:29:59.793979 28242 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.795850 28625 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.796203 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 481) took 1079 ms. Trace:
I20260924 14:29:59.790048 28591 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:29:59.802393 26328 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:29:59.743804 28237 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:29:59.804766 26298 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:29:59.803651 28635 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:00.134877 28234 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:00.009371 26402 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:00.144636 26251 rpcz_store.cc:276] 0924 14:29:58.717147 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:58.717897 (+   750us) service_pool.cc:227] Handling call
0924 14:29:59.796003 (+1078106us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b5400716e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:59.796116 (+   113us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":897759616}
I20260924 14:30:00.145365 28663 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:00.145542 28234 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:00.146116 28663 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:00.146545 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.146700 28650 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:00.147253 26251 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:00.148509 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:00.147994 28234 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:00.111526 28558 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:00.148419 28663 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:00.034375 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:29:59.943539 26406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 446) took 3015 ms (client timeout 1505 ms). Trace:
I20260924 14:30:00.170495 26253 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
W20260924 14:30:00.064877 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 261807us
I20260924 14:30:00.130797 27125 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:59.803844 28372 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:29:59.804498 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 68318.6us
I20260924 14:29:59.823544 27126 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:29:59.872175 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.869804 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.176307 27188 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:00.176548 26405 raft_consensus.cc:2396] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260924 14:30:00.034893 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.177383 26250 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:29:59.975776 28578 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:59.901100 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1031) took 1242 ms. Trace:
W20260924 14:30:00.144650 27192 rpcz_store.cc:269] 0924 14:29:56.993393 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:56.993540 (+   147us) service_pool.cc:227] Handling call
0924 14:29:59.745856 (+2752316us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b540000da54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:59.745955 (+    99us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4886994304}
I20260924 14:30:00.160121 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.186416 26400 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
I20260924 14:30:00.026577 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 397) took 1549 ms. Trace:
I20260924 14:29:59.997208 28560 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:00.144060 26397 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:00.149807 26324 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:00.188254 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.149227 26401 rpcz_store.cc:276] 0924 14:29:58.585877 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:58.586017 (+   140us) service_pool.cc:227] Handling call
0924 14:29:58.586365 (+   348us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:58.613285 (+ 26920us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:58.613328 (+    43us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:58.613402 (+    74us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:29:58.613490 (+    88us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:59.745794 (+1132304us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b5400477054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:59.745840 (+    46us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:59.745953 (+   113us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":61297408}
I20260924 14:30:00.152598 26404 rpcz_store.cc:276] 0924 14:29:58.701715 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:58.702971 (+  1256us) service_pool.cc:227] Handling call
0924 14:29:58.707309 (+  4338us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:58.865165 (+157856us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:58.865192 (+    27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:58.865223 (+    31us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:58.865250 (+    27us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:59.755020 (+889770us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400336654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:59.755074 (+    54us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:59.755186 (+   112us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":362567424}
I20260924 14:30:00.165087 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:00.148751 27181 rpcz_store.cc:276] 0924 14:29:58.582775 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:58.583540 (+   765us) service_pool.cc:227] Handling call
0924 14:29:58.583831 (+   291us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:59.291265 (+707434us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:59.291295 (+    30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:59.291335 (+    40us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:29:59.291363 (+    28us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:59.748593 (+457230us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b540014ffd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:59.748638 (+    45us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:59.748750 (+   112us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1626428672}
I20260924 14:30:00.144732 27182 rpcz_store.cc:276] 0924 14:29:58.297151 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:58.297442 (+   291us) service_pool.cc:227] Handling call
0924 14:29:58.297830 (+   388us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:58.904080 (+606250us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:58.904111 (+    31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:58.904151 (+    40us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:29:58.904188 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:59.747721 (+843533us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b540079f9d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:59.747770 (+    49us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:29:59.747890 (+   120us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1393714048}
I20260924 14:29:59.883869 27189 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:00.023084 28610 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:00.170521 26406 rpcz_store.cc:269] 0924 14:29:56.924331 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:56.924552 (+   221us) service_pool.cc:227] Handling call
0924 14:29:59.939198 (+3014646us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54001c1e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:29:59.939344 (+   146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3282976128}
I20260924 14:29:59.804973 28609 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:29:59.804196 28593 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:00.198434 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:00.178377 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:00.179088 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 355632us
I20260924 14:30:00.179085 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:29:59.821208 26399 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260924 14:29:59.992628 28592 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:29:59.942309 28187 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:29:59.943840 28533 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:29:59.996411 28547 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:00.012616 28613 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:00.181116 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 79465.3us
W20260924 14:30:00.150825 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 346188us
I20260924 14:30:00.189155 26251 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:00.022397 28639 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:00.111306 28646 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:00.187168 27191 rpcz_store.cc:276] 0924 14:29:58.477201 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:58.479590 (+  2389us) service_pool.cc:227] Handling call
0924 14:29:58.550322 (+ 70732us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:58.859206 (+308884us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:58.859242 (+    36us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:29:58.859299 (+    57us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:29:58.859347 (+    48us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:00.026257 (+1166910us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540000d554. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:00.026322 (+    65us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:00.026506 (+   184us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":709827968}
I20260924 14:30:00.192924 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.277642 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.193183 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.194429 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.279477 26252 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:00.194833 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.280499 26258 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:00.159322 28237 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260924 14:30:00.172196 28541 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.013s	sys 0.006s
I20260924 14:30:00.197402 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.169720 28524 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:00.221885 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:00.282196 26404 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:00.185959 26254 rpcz_store.cc:276] 0924 14:29:58.659129 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:58.662097 (+  2968us) service_pool.cc:227] Handling call
0924 14:29:58.662490 (+   393us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:29:59.382349 (+719859us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:29:59.900834 (+518485us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54000c17d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:29:59.901034 (+   200us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1437848320}
I20260924 14:30:00.283026 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.258906 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:00.285259 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.228461 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:00.160413 28586 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:00.262032 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.155766 28367 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:00.267097 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.293918 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.189644 28267 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:00.201687 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:00.199806 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:00.227715 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52083.6us
I20260924 14:30:00.168627 28581 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:00.190469 28644 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:00.153766 28591 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:00.153062 28202 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:00.272338 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:00.324466 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.273993 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.153707 28583 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:00.325166 26328 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:30:00.178580 28558 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:00.280351 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:29:59.869060 28636 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:00.239859 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:00.227262 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:00.289410 27123 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:00.293443 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 324395us
I20260924 14:30:00.262436 26399 pending_rounds.cc:76] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 2
W20260924 14:30:00.228078 28619 consensus_meta.cc:287] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.581s	user 0.004s	sys 0.007s
I20260924 14:30:00.297580 26250 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:00.439841 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.420186 26323 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:00.441030 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.294373 26257 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
W20260924 14:30:00.299691 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 98095.7us
I20260924 14:30:00.193562 28663 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:00.264642 27126 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:00.300734 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.301522 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.301848 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.258671 28383 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:00.325747 28560 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:00.227399 28610 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:00.305761 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.311851 27189 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
W20260924 14:30:00.293845 28541 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.013s	sys 0.006s
I20260924 14:30:00.195067 28535 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:00.269619 28609 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:00.340286 26405 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:00.453864 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.180606 28544 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:00.350075 28591 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:00.346025 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.347681 28650 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:00.347796 28367 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:00.348106 27223 heartbeater.cc:644] Failed to heartbeat to 127.22.151.254:43789 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.22.151.254:43789 timed out after 15.000s (SENT)
I20260924 14:30:00.278825 28187 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:00.299937 28630 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:00.462230 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.173269 28660 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:00.195662 28578 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:00.355594 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:00.356410 26400 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:00.378296 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.464653 26332 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:00.465049 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 405) took 1097 ms. Trace:
I20260924 14:30:00.465224 26255 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:00.386981 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.465610 27189 rpcz_store.cc:276] 0924 14:29:59.367807 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:59.624808 (+257001us) service_pool.cc:227] Handling call
0924 14:30:00.464902 (+840094us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54002c51d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:00.464995 (+    93us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":407721216}
I20260924 14:30:00.466215 26249 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:00.467119 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:00.385923 27188 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 582) took 1722 ms (client timeout 2265 ms). Trace:
I20260924 14:30:00.429934 28642 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:30:00.469376 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.430893 26328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:00.470634 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:00.471079 26328 raft_consensus.cc:595] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:00.471673 26256 raft_consensus.cc:2354] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6.
I20260924 14:30:00.448438 28636 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:00.448832 28560 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:00.304510 28267 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:00.395699 27185 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:30:00.442430 26369 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:00.457391 28609 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:00.483889 26369 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:00.484671 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.489076 26249 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:00.489609 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:00.443770 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 154428us
I20260924 14:30:00.420966 26399 pending_rounds.cc:97] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260924 14:30:00.492216 26399 raft_consensus.cc:2866] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:00.303572 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.447412 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.324790 28644 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:00.493942 27182 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:30:00.448168 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.276750 28533 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:00.450749 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 186563us
I20260924 14:30:00.505363 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:00.446838 28383 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:00.447708 28663 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:00.453264 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.361261 28202 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:00.286923 28613 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:00.292356 28237 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:00.294445 28646 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:00.458537 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.363698 28586 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:00.459687 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.462965 27192 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:00.540213 26254 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
W20260924 14:30:00.468055 27188 rpcz_store.cc:269] 0924 14:29:58.663206 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:58.663331 (+   125us) service_pool.cc:227] Handling call
0924 14:30:00.385561 (+1722230us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540070f8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:00.385793 (+   232us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3021304704}
I20260924 14:30:00.355756 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.541360 27191 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:00.541491 26254 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:00.378257 28529 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:00.481015 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.546075 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:00.457093 28632 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:00.297653 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.552198 26258 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:00.458088 28544 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:00.501705 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.557533 26255 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:00.508754 28535 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:00.510788 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.563196 26249 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:00.489940 28629 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:00.511219 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.444192 28619 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:00.528187 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.479825 28642 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:00.543044 28237 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:00.517796 26256 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:00.483860 28560 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:00.549741 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:00.496235 28267 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:00.511111 26253 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:00.565086 26224 leader_election.cc:398] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:00.677806 26224 leader_election.cc:401] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260924 14:30:00.678325 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 113470us
I20260924 14:30:00.557857 28591 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:00.558091 28632 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:00.518728 28383 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:00.559202 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.566236 26252 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:00.560469 26257 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:49618
I20260924 14:30:00.574874 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:00.698349 26257 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:00.698746 26257 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:30:00.701515 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 393035us
I20260924 14:30:00.702989 26224 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:00.557477 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.511871 28393 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:00.539780 28370 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:00.633643 28187 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:00.633453 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:00.726752 26332 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 1.
I20260924 14:30:00.635816 28613 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:00.639032 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.658936 28642 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:00.640478 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.660954 27123 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:00.565357 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.767001 27191 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:30:00.683825 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:00.562778 28535 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:00.483799 28636 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:00.576444 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:00.696753 26326 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
W20260924 14:30:00.773237 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 196992us
I20260924 14:30:00.704396 28632 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:00.640211 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 404) took 1273 ms. Trace:
I20260924 14:30:00.781527 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:00.684477 28671 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260924 14:30:00.720019 28393 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:00.764120 28578 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:00.555070 28529 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:00.772534 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 111711us
W20260924 14:30:00.663902 28611 consensus_meta.cc:287] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.853s	user 0.016s	sys 0.000s
I20260924 14:30:00.694314 28541 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:00.783414 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 459) took 1053 ms. Trace:
I20260924 14:30:00.797773 26401 rpcz_store.cc:276] 0924 14:29:59.730516 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:00.286317 (+555801us) service_pool.cc:227] Handling call
0924 14:30:00.286614 (+   297us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:00.493330 (+206716us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:00.493362 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:00.493436 (+    74us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:00.493478 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:00.783102 (+289624us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540011dd54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:00.783165 (+    63us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:00.783325 (+   160us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":474865408}
I20260924 14:30:00.688962 28591 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:00.700645 28644 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:00.787586 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 901) took 1206 ms. Trace:
I20260924 14:30:00.781466 27185 rpcz_store.cc:276] 0924 14:29:59.367205 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:59.702621 (+335416us) service_pool.cc:227] Handling call
0924 14:30:00.640063 (+937442us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540000d054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:00.640161 (+    98us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1376726400}
I20260924 14:30:00.789394 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:00.785210 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:00.793422 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:00.794584 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 936) took 1065 ms. Trace:
I20260924 14:30:00.800652 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:00.771873 28535 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:00.802958 26399 rpcz_store.cc:276] 0924 14:29:59.580930 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:59.581117 (+   187us) service_pool.cc:227] Handling call
0924 14:29:59.581442 (+   325us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:00.492927 (+911485us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:00.787268 (+294341us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400450454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:00.787511 (+   243us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":550351744}
I20260924 14:30:00.804389 26406 rpcz_store.cc:276] 0924 14:29:59.728699 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:00.260058 (+531359us) service_pool.cc:227] Handling call
0924 14:30:00.260404 (+   346us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:00.260746 (+   342us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:00.260774 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:00.268214 (+  7440us) log.cc:842] Serialized 26 byte log entry
0924 14:30:00.572257 (+304043us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:00.572298 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:00.572362 (+    64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:00.794348 (+221986us) raft_consensus.cc:1671] finished
0924 14:30:00.794382 (+    34us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:00.794515 (+   133us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":775,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":14654,"thread_start_us":230953,"threads_started":1,"wal-append.queue_time_us":463271}
I20260924 14:30:00.812529 26295 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:00.786154 28359 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:00.786389 28640 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:00.827795 26250 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:00.827885 26295 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:00.801681 28542 consensus_meta.cc:287] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.986s	user 0.008s	sys 0.008s
I20260924 14:30:00.827831 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 316) took 1602 ms. Trace:
I20260924 14:30:00.791229 28393 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:00.825654 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 358431us
I20260924 14:30:00.831722 26331 rpcz_store.cc:276] 0924 14:29:59.225782 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:59.548199 (+322417us) service_pool.cc:227] Handling call
0924 14:29:59.548640 (+   441us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:29:59.596462 (+ 47822us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:29:59.596506 (+    44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:29:59.620587 (+ 24081us) log.cc:842] Serialized 26 byte log entry
0924 14:30:00.043927 (+423340us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b18003260e0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:30:00.299233 (+255306us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:00.299272 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:00.299343 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:00.812096 (+512753us) raft_consensus.cc:1671] finished
0924 14:30:00.812116 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:00.827761 (+ 15645us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":47346,"raft.queue_time_us":60949,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5223,"spinlock_wait_cycles":8414592,"thread_start_us":255088,"threads_started":1,"wal-append.queue_time_us":923145}
I20260924 14:30:00.757104 28629 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.136s
I20260924 14:30:00.785997 28583 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:00.801599 28558 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:00.802083 28581 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:00.831590 28640 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:00.802779 28591 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:00.836398 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:00.828666 28541 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:00.824252 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:00.823764 28610 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:00.831907 28644 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:00.809077 28611 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:00.826571 28359 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:00.887980 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 308999us
W20260924 14:30:00.904732 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 80552.6us
W20260924 14:30:00.899422 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 63085.8us
I20260924 14:30:00.901554 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.902616 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 607) took 1201 ms. Trace:
I20260924 14:30:00.943630 26326 rpcz_store.cc:276] 0924 14:29:59.701379 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:59.754109 (+ 52730us) service_pool.cc:227] Handling call
0924 14:30:00.902440 (+1148331us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54007800d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:00.902550 (+   110us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1251593600}
I20260924 14:30:00.958845 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.937896 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:00.865665 28242 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:00.837807 28542 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:00.790934 28544 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:00.893165 28640 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:00.869872 28676 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:00.892786 28541 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:00.980039 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 194942us
I20260924 14:30:00.973330 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:00.936046 28202 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:00.953352 28359 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:30:00.987797 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 484386us
I20260924 14:30:00.916680 28501 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:00.916069 28367 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:00.890918 28663 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:00.912894 28644 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:00.991735 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:00.991791 26221 leader_election.cc:398] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
W20260924 14:30:01.036742 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45077.2us
W20260924 14:30:01.036911 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 207649us
W20260924 14:30:01.036780 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45113.2us
I20260924 14:30:01.037756 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:00.983831 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.976639 28646 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:01.044514 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:01.045573 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:00.984728 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.999393 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:01.009859 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:01.026757 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:00.975009 28638 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:01.034729 27194 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:00.975773 28533 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:00.952374 28610 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:01.054008 26295 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:01.040509 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:00.979606 28592 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.710s	user 0.010s	sys 0.003s
I20260924 14:30:01.047886 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:01.056571 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 231740us
I20260924 14:30:01.047088 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:01.049635 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:00.996135 28541 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:01.050822 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 81948.2us
I20260924 14:30:01.063791 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:01.068676 26398 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:01.068899 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:01.022421 28501 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:01.010226 28332 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:00.993340 28676 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:00.992820 28660 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:00.975217 28547 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260924 14:30:01.059324 28367 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:01.069564 26400 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:01.042783 28644 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:00.983835 28242 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:01.056124 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:01.084335 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:01.062274 27123 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:01.061474 28619 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:01.098444 28611 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:01.122731 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:01.085294 28501 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:01.069772 28581 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:30:01.117017 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 54846.9us
I20260924 14:30:01.123678 26250 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:01.063416 28533 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:01.130425 27123 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:01.114372 28372 consensus_meta.cc:287] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.922s	user 0.007s	sys 0.004s
I20260924 14:30:01.085304 28539 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:01.093305 28367 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:01.111836 28636 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:01.095115 28592 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:01.088672 28332 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:01.138928 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:01.088129 28676 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:01.063591 28610 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:01.141571 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:01.176669 27189 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:01.116559 28529 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:01.145503 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:01.147270 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:01.118208 28646 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:01.121377 28234 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:01.150564 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 471) took 1356 ms. Trace:
I20260924 14:30:01.194801 26257 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:01.153254 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:01.155205 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.155682 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.156280 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.118024 28688 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:01.174954 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:01.178720 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 122654us
I20260924 14:30:01.206246 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:01.192404 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:01.193060 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45860.2us
I20260924 14:30:01.185168 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.199309 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:01.205070 26328 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:01.200014 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:01.192618 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 147111us
I20260924 14:30:01.202001 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:01.205659 27186 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:01.204027 26326 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:30:01.209095 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:01.201067 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 56910.8us
I20260924 14:30:01.202989 26404 rpcz_store.cc:276] 0924 14:29:59.794515 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:00.306763 (+512248us) service_pool.cc:227] Handling call
0924 14:30:00.354346 (+ 47583us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:00.354530 (+   184us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:30:00.796065 (+441535us) log.cc:842] Serialized 14 byte log entry
0924 14:30:01.150150 (+354085us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:01.150220 (+    70us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:01.150286 (+    66us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:01.150368 (+    82us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:01.150499 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":597995,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":39769,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":83,"thread_start_us":794903,"threads_started":3,"wal-append.queue_time_us":339458}
I20260924 14:30:01.203756 26254 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:30:01.210736 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:01.211788 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:01.211974 28605 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:01.212543 26403 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:30:01.212597 28605 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:01.213259 28605 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:01.213508 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:01.213928 26404 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:30:01.204336 26250 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:01.211725 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:01.215169 26397 raft_consensus.cc:2362] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7. Re-sending same reply.
I20260924 14:30:01.215759 28529 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:01.216053 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:01.217732 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:01.217919 26224 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:01.219756 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:01.150242 28367 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:01.134608 28663 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260924 14:30:01.225129 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:01.174307 28332 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:01.164172 28592 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:01.230834 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 420569us
I20260924 14:30:01.234803 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:01.236323 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:01.176769 28687 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:01.231297 26250 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:30:01.233215 26257 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:01.202947 28234 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:01.202507 28539 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:01.240502 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:01.219699 28647 raft_consensus.cc:2741] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260924 14:30:01.252379 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:01.253199 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:01.225628 28690 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:01.141268 28372 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:01.237475 28681 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:01.229954 28529 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:01.238202 28646 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:01.276589 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:01.261972 28693 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260924 14:30:01.262574 28684 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260924 14:30:01.263007 28359 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:01.263330 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:01.255496 28234 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:01.266309 28663 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:30:01.241439 28592 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:01.289280 28698 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260924 14:30:01.270452 28609 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.753s	user 0.005s	sys 0.003s
I20260924 14:30:01.272893 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:01.237869 28542 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:01.278304 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:01.280332 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 74211.7us
I20260924 14:30:01.280529 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:01.298403 26257 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:01.257393 28638 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:01.288707 26325 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:30:01.300259 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:01.284293 27125 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:01.256105 28539 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:01.290925 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:01.294108 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 621) took 1189 ms. Trace:
I20260924 14:30:01.289479 26371 leader_election.cc:398] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6.
I20260924 14:30:01.301422 26326 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
W20260924 14:30:01.296623 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61931.6us
I20260924 14:30:01.303247 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:01.306878 26326 raft_consensus.cc:595] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:01.301627 26253 rpcz_store.cc:276] 0924 14:30:00.104584 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:00.176320 (+ 71736us) service_pool.cc:227] Handling call
0924 14:30:01.293923 (+1117603us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54003827d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:01.294045 (+   122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":527355776}
I20260924 14:30:01.298524 27188 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260924 14:30:01.308275 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:01.306041 26371 leader_election.cc:401] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260924 14:30:01.308501 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:01.295244 28593 consensus_meta.cc:287] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.030s	user 0.013s	sys 0.000s
I20260924 14:30:01.274950 28529 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:01.279757 28646 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:01.306133 27125 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:01.275218 28636 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260924 14:30:01.266449 28267 consensus_meta.cc:287] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.000s
I20260924 14:30:01.310292 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:01.309592 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:01.332000 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:01.298935 26250 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:01.305461 28539 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:01.311738 28703 raft_consensus.cc:2741] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260924 14:30:01.309677 28696 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:01.335964 28581 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:01.340135 28581 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:01.340425 28632 consensus_meta.cc:287] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.529s	user 0.006s	sys 0.004s
I20260924 14:30:01.312333 28697 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:01.312616 28610 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:01.312686 28630 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:01.310042 28542 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:01.314519 28242 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:01.300570 28638 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:01.343669 28638 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:01.316668 28359 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:01.335108 28560 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.000s
I20260924 14:30:01.294456 28690 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:01.329264 27125 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:01.273401 28642 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:01.386407 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 53301.1us
I20260924 14:30:01.310976 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:01.389477 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 78570.5us
I20260924 14:30:01.390166 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:01.330312 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:30:01.337620 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:01.391690 27185 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
W20260924 14:30:01.334694 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49154us
I20260924 14:30:01.345820 26257 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:01.340493 28581 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:01.326531 28529 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:01.327988 28646 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:30:01.385793 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 149924us
I20260924 14:30:01.330385 28187 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:01.338887 28393 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:01.393813 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:01.393100 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:01.342867 28541 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:01.326789 28593 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:01.301273 28609 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:01.396806 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 67627.9us
I20260924 14:30:01.399855 26401 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:01.336229 28660 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:01.403066 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:01.406977 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 408) took 1908 ms. Trace:
I20260924 14:30:01.341028 28632 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:01.396697 28242 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:01.394270 28688 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:01.338057 28267 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:01.394178 28690 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:01.410996 27194 rpcz_store.cc:276] 0924 14:29:59.498752 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:59.778058 (+279306us) service_pool.cc:227] Handling call
0924 14:30:01.406785 (+1628727us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b5400776854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:01.406920 (+   135us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2047757184}
I20260924 14:30:01.410899 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:01.421617 26401 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:01.414692 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:01.400687 28630 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:01.402788 28703 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:01.418550 27189 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:01.418728 26406 raft_consensus.cc:2354] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:01.400513 28187 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:01.420204 27125 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:01.420812 27182 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:01.409523 28600 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260924 14:30:01.424682 28699 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260924 14:30:01.425067 28694 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260924 14:30:01.425424 28642 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:01.427753 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:01.428436 28383 consensus_meta.cc:287] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.669s	user 0.004s	sys 0.007s
I20260924 14:30:01.422530 28613 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260924 14:30:01.431617 28367 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:01.536463 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 116314us
I20260924 14:30:01.537882 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:01.409721 28696 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260924 14:30:01.433058 28593 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:01.433449 28702 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:01.433972 27190 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:30:01.434664 28267 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:01.387288 28560 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:01.436578 28541 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:01.437881 28393 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:01.416303 28707 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:01.440201 28632 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:01.441845 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:01.549659 26397 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:01.410372 28693 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:01.445434 28592 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:01.422884 28690 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:01.459964 26401 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
W20260924 14:30:01.461474 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 46888us
I20260924 14:30:01.556645 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:01.415899 28372 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:01.451936 28703 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:01.509684 28644 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:01.452018 28688 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:01.510780 28332 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:01.495988 28237 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.792s	user 0.012s	sys 0.000s
I20260924 14:30:01.461849 28187 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:01.512091 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:01.538792 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 83140.1us
I20260924 14:30:01.418042 28629 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.106s
I20260924 14:30:01.498183 28685 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:30:01.551301 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 606) took 1858 ms. Trace:
I20260924 14:30:01.631582 26325 rpcz_store.cc:276] 0924 14:29:59.692571 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:59.761803 (+ 69232us) service_pool.cc:227] Handling call
0924 14:29:59.762112 (+   309us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:01.300646 (+1538534us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:01.550971 (+250325us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540001efd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:01.551215 (+   244us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3510353792}
I20260924 14:30:01.451238 28501 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:01.557698 26224 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:01.561774 28707 raft_consensus.cc:2741] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260924 14:30:01.560153 27125 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:01.562824 28591 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.627s	user 0.003s	sys 0.007s
I20260924 14:30:01.521399 28383 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:01.576269 28703 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:01.553646 28535 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.722s	user 0.010s	sys 0.001s
I20260924 14:30:01.653204 27181 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:01.547113 28642 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:01.544894 28630 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:01.553059 28693 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:01.631022 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:01.638273 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:01.640184 26401 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 2. Current term is 3. Ops: []
I20260924 14:30:01.559368 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:01.665161 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 105872us
I20260924 14:30:01.632534 28685 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:01.548699 28702 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:01.642943 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 82839.7us
W20260924 14:30:01.644340 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 86725.5us
I20260924 14:30:01.647603 28494 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:01.641223 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1056) took 1151 ms. Trace:
I20260924 14:30:01.705844 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:01.704741 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 66540.1us
I20260924 14:30:01.710608 26399 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:01.710938 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:01.712441 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:01.713385 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 82439.5us
I20260924 14:30:01.709105 26249 rpcz_store.cc:276] 0924 14:30:00.490399 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:00.564164 (+ 73765us) service_pool.cc:227] Handling call
0924 14:30:01.294287 (+730123us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:01.294463 (+   176us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:01.294487 (+    24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:01.294534 (+    47us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:01.294553 (+    19us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:01.294588 (+    35us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:01.641041 (+346453us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54003827d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:01.641162 (+   121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1678258560}
I20260924 14:30:01.646718 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:01.641781 28234 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:01.706756 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:01.648447 28242 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:30:01.705196 27125 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:01.612040 28605 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:01.666476 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:01.721773 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 55398.2us
I20260924 14:30:01.722872 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:01.665087 28535 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:01.637042 28237 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:01.707508 28698 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6.
I20260924 14:30:01.643383 28648 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260924 14:30:01.702219 28685 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:30:01.716822 26402 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260924 14:30:01.707708 28611 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:01.727342 27125 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:01.704661 28693 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:01.768350 26405 raft_consensus.cc:2354] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:01.708410 28609 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:01.729609 28541 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260924 14:30:01.754091 28704 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260924 14:30:01.704538 28706 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:01.736321 28592 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:01.759614 28638 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:01.712007 28494 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:01.648289 28591 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:01.720037 28533 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:01.728070 28372 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 14:30:01.769771 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:01.770584 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:01.772079 26399 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:30:01.730816 28267 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:01.717996 28700 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:01.726084 28558 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:01.778244 28541 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:01.772034 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 59687.7us
I20260924 14:30:01.889400 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:01.891902 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.893370 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.894157 26326 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:01.894271 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.894659 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:01.895427 26328 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:01.896152 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:01.896695 26328 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:01.900660 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:01.902275 26325 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:01.774230 28547 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260924 14:30:01.832844 26400 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
W20260924 14:30:01.840201 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 97742.1us
I20260924 14:30:01.729902 28529 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:01.873972 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 104276us
I20260924 14:30:01.777581 28542 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:01.878332 28595 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:01.752434 26397 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:01.903155 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:01.901945 28700 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:01.916145 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.916411 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.860460 28533 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:01.917775 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.918211 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:01.991663 26323 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:01.919399 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:01.993872 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:01.994978 26256 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:01.836429 28494 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:01.832890 28367 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:01.787701 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 80999.4us
W20260924 14:30:01.942351 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 48509.3us
I20260924 14:30:01.876495 26403 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:01.953779 26325 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:01.874424 28560 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:01.958463 28267 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:01.961969 28640 consensus_meta.cc:287] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.938s	user 0.011s	sys 0.002s
W20260924 14:30:02.006052 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 255929us
I20260924 14:30:01.897436 28541 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:01.962198 26328 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:01.964329 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:01.961817 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.021551 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:01.971607 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 241316us
I20260924 14:30:02.022989 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:01.971838 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:01.874614 28706 consensus_queue.cc:1046] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.143s
I20260924 14:30:01.974396 26253 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:02.030885 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:02.033443 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:01.953032 26398 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:01.951395 28542 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:01.922626 28644 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:02.066177 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:02.067028 26249 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:02.010489 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:02.025554 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:01.972896 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:02.026372 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:02.022059 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:02.068653 26250 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:01.878577 28558 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:02.072562 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.073403 26298 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:02.034149 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 506) took 1048 ms. Trace:
I20260924 14:30:02.075219 26258 rpcz_store.cc:276] 0924 14:30:00.985491 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:00.985667 (+   176us) service_pool.cc:227] Handling call
0924 14:30:00.986040 (+   373us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:01.137270 (+151230us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:01.137303 (+    33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:01.137347 (+    44us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:01.137395 (+    48us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:02.033853 (+896458us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540015e5d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:02.033920 (+    67us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:02.034076 (+   156us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":347284736}
W20260924 14:30:02.007222 28704 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260924 14:30:02.001914 28533 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:02.088964 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:02.086172 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:02.090584 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:01.963532 28332 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:02.091359 26258 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:02.098596 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:02.002668 28383 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:01.991884 28501 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:02.069178 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:02.111603 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 917) took 1064 ms. Trace:
I20260924 14:30:02.111801 26254 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:02.112093 26406 rpcz_store.cc:276] 0924 14:30:01.046802 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.047574 (+   772us) service_pool.cc:227] Handling call
0924 14:30:02.111237 (+1063663us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b540035cfd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:02.111533 (+   296us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":818648320}
I20260924 14:30:02.069208 26325 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:02.115058 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:02.115778 28267 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:02.116302 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:02.117353 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:02.070221 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:02.072566 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:01.986171 28700 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:02.118309 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:02.118631 28700 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:02.080749 26371 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.514s (SENT)
I20260924 14:30:02.081365 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:02.095021 26250 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:02.055960 28242 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:02.105393 28676 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.903s	user 0.005s	sys 0.006s
I20260924 14:30:02.106316 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:02.108136 28723 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260924 14:30:02.107929 28685 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:02.038061 28702 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:02.029968 28640 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:02.119235 28591 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:02.197049 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.119366 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:02.441793 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 351) took 1170 ms. Trace:
I20260924 14:30:02.350165 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.565012 27181 rpcz_store.cc:276] 0924 14:30:01.271880 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.272285 (+   405us) service_pool.cc:227] Handling call
0924 14:30:01.272548 (+   263us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:01.664814 (+392266us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:02.441434 (+776620us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54003269d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:02.441710 (+   276us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":874867968}
I20260924 14:30:02.385684 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 345) took 1194 ms. Trace:
I20260924 14:30:02.566660 27182 rpcz_store.cc:276] 0924 14:30:01.191804 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.192107 (+   303us) service_pool.cc:227] Handling call
0924 14:30:02.385380 (+1193273us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54006c5054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:02.385604 (+   224us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":485956992}
I20260924 14:30:02.418107 26325 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:02.567830 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.568652 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:02.569789 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.197077 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:02.066565 28542 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:02.571696 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:02.326615 26249 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:02.571895 26253 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 7
I20260924 14:30:02.273061 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:02.410626 27191 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
W20260924 14:30:02.442709 26405 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 919) took 1395 ms (client timeout 1511 ms). Trace:
I20260924 14:30:02.082118 28558 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:02.566978 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:02.573994 26323 consensus_meta.cc:287] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.582s	user 0.024s	sys 0.023s
I20260924 14:30:02.054261 28367 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:02.369151 28187 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.728s	user 0.007s	sys 0.003s
W20260924 14:30:02.167841 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 135053us
I20260924 14:30:02.576097 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:02.238557 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:02.594995 27126 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:02.163920 28592 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:30:02.597620 27182 raft_consensus.cc:2354] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
W20260924 14:30:02.579154 26405 rpcz_store.cc:269] 0924 14:30:01.047350 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.049342 (+  1992us) service_pool.cc:227] Handling call
0924 14:30:02.442466 (+1393124us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b5400123ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:02.442634 (+   168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1618039040}
I20260924 14:30:02.575372 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:02.576122 28581 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.176s	user 0.012s	sys 0.000s
I20260924 14:30:02.266179 28605 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:02.612830 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:02.612988 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:02.599586 26332 raft_consensus.cc:2354] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
W20260924 14:30:02.614348 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 533585us
I20260924 14:30:02.340548 28595 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:02.570755 28237 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:02.584081 28646 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.177s	user 0.011s	sys 0.000s
I20260924 14:30:02.584527 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.586200 26399 raft_consensus.cc:2354] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:02.587020 26323 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 3.
W20260924 14:30:02.428377 28703 consensus_meta.cc:287] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.710s	user 0.007s	sys 0.005s
I20260924 14:30:02.572569 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.341336 28638 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:02.264768 28383 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:02.570385 28535 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:02.669305 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.077816 28234 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:02.096323 28648 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.495s
I20260924 14:30:02.599282 26258 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:30:02.411419 28529 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:02.668969 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.618392 28655 raft_consensus.cc:979] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:02.621783 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:02.622124 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 504) took 1732 ms. Trace:
I20260924 14:30:02.681252 26252 rpcz_store.cc:276] 0924 14:30:00.889438 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.290641 (+401203us) service_pool.cc:227] Handling call
0924 14:30:02.621937 (+1331296us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54004538d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:02.622066 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2749992832}
W20260924 14:30:02.622432 28700 consensus_meta.cc:287] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.000s
W20260924 14:30:02.684124 28494 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.658s	user 0.008s	sys 0.002s
I20260924 14:30:02.683780 28242 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:02.684646 28494 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:02.683827 28700 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:02.687255 26328 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:02.623893 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:02.616636 28632 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:02.624645 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:02.626232 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:02.700893 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 74725.6us
I20260924 14:30:02.626806 28685 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:02.210448 28687 consensus_queue.cc:1046] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.406s
W20260924 14:30:02.632032 28690 consensus_meta.cc:287] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.048s	user 0.013s	sys 0.000s
I20260924 14:30:02.633003 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:02.601526 28187 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:02.633663 28640 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:02.634001 28732 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260924 14:30:02.575248 28676 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:02.656114 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:02.752496 27182 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:02.659214 26253 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:02.671077 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:02.754953 26323 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:02.755833 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.756739 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.757305 26323 raft_consensus.cc:2396] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260924 14:30:02.678150 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.758657 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:02.759146 26325 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:02.624559 28592 raft_consensus.cc:686] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:02.617273 28539 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.279s	user 0.010s	sys 0.002s
I20260924 14:30:02.699028 27194 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
W20260924 14:30:02.767401 28629 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.005s	sys 0.005s
I20260924 14:30:02.777756 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:02.626847 28681 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:02.778312 28629 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.005s	sys 0.005s
I20260924 14:30:02.778664 26402 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:02.778620 28681 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:02.706418 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 74239.8us
I20260924 14:30:02.653841 28646 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:02.718008 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:02.704260 26297 connection.cc:232] RPC call timeout handler was delayed by -1.0200414070004626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2989s for a response
W20260924 14:30:02.748472 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 126047us
I20260924 14:30:02.762967 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.616242 28581 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:02.793283 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.793511 28592 consensus_queue.cc:237] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:02.794523 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.794699 28581 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:02.796221 28581 consensus_queue.cc:1046] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:02.796918 28578 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:02.764629 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.699577 28494 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:02.799350 26323 raft_consensus.cc:2396] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260924 14:30:02.769227 27185 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:02.717722 28237 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:02.616197 28332 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:02.688138 28700 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:02.714488 28558 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:02.783124 26326 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:02.783599 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:02.755173 28541 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.701s	user 0.008s	sys 0.003s
I20260924 14:30:02.716434 27192 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:30:02.750794 28690 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:02.674099 28383 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:02.751608 28686 raft_consensus.cc:1067] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:02.745280 28632 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:02.763608 28713 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260924 14:30:02.787617 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:02.789062 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:02.797202 28581 raft_consensus.cc:3032] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:02.596309 28367 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:02.863114 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:02.799224 28494 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:02.804123 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:02.670462 28703 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:02.811350 26404 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:02.821539 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:02.823781 28646 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:02.936949 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:02.822989 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 250054us
I20260924 14:30:02.940147 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:02.822268 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.000s	sys 0.015s
I20260924 14:30:02.786347 28595 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:02.841269 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:02.843868 26254 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: []
W20260924 14:30:02.862319 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 74807us
W20260924 14:30:02.943598 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 102470us
I20260924 14:30:02.943957 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:02.945161 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:02.946256 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:02.941105 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.000s	sys 0.016s
W20260924 14:30:02.862627 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 102137us
I20260924 14:30:02.793285 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.788707 28539 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:02.881141 26406 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:02.879467 26331 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
W20260924 14:30:02.912106 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 142169us
I20260924 14:30:02.822646 26257 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:02.830368 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49630us
I20260924 14:30:02.951710 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:02.849284 28681 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:02.865037 28581 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:02.819375 28541 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:02.958120 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:02.959271 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:02.880664 28741 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:02.960268 28529 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:02.962502 28703 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:02.963846 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:02.965679 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:02.966601 28629 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:02.963163 26249 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:02.994308 26253 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:02.956423 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:02.979894 28332 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:02.976895 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:02.985683 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:02.911705 28494 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:02.999461 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:02.993142 26224 leader_election.cc:398] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:03.000893 26224 leader_election.cc:401] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:30:02.936448 28640 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:03.017174 26256 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:03.019665 27182 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:03.023118 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:03.023825 26323 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:30:03.023777 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:03.026835 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:03.028820 26252 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:02.940811 28740 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:03.034399 28332 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:03.034896 28740 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:03.035385 26325 raft_consensus.cc:2354] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:30:03.035887 28332 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:03.036077 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 946) took 1397 ms. Trace:
I20260924 14:30:03.036504 26398 rpcz_store.cc:276] 0924 14:30:01.638668 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.639670 (+  1002us) service_pool.cc:227] Handling call
0924 14:30:01.952726 (+313056us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:02.054717 (+101991us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.035885 (+981168us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54002fcbd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:03.036015 (+   130us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":705555840}
I20260924 14:30:03.037103 28740 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:03.037173 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:03.037961 26325 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:03.043927 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 97737.1us
I20260924 14:30:03.036499 28332 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:03.045084 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:03.046346 28332 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:03.047009 28332 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:03.050093 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:03.051008 26330 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:30:03.051400 26330 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:03.031791 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 64619.5us
W20260924 14:30:03.031890 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46301.2us
I20260924 14:30:03.052362 26330 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:03.053921 26330 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:03.056501 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 628) took 1033 ms. Trace:
I20260924 14:30:03.056730 26250 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:03.056926 26256 rpcz_store.cc:276] 0924 14:30:02.022903 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.026048 (+  3145us) service_pool.cc:227] Handling call
0924 14:30:03.056296 (+1030248us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400782dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:03.056445 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2150196608}
I20260924 14:30:03.058118 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:02.984328 28686 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:03.059842 28686 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:03.060055 26325 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:03.060267 28686 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:03.061128 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:03.062536 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:03.062731 28655 raft_consensus.cc:3032] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:30:03.034063 28741 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:03.063575 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:03.063882 28741 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:30:03.064383 28741 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:03.064626 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:03.065279 28741 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:03.065699 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:03.066403 26325 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:03.051316 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:03.068464 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 637) took 1042 ms. Trace:
I20260924 14:30:03.068996 26258 rpcz_store.cc:276] 0924 14:30:02.025750 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.164569 (+138819us) service_pool.cc:227] Handling call
0924 14:30:02.165073 (+   504us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:02.678561 (+513488us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.068162 (+389601us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b5400382cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:03.068387 (+   225us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":992994048}
W20260924 14:30:03.076653 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 83704.3us
I20260924 14:30:03.029788 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:03.077822 26224 leader_election.cc:398] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:03.078431 26253 raft_consensus.cc:2354] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6.
I20260924 14:30:03.078531 26224 leader_election.cc:401] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:30:03.059156 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:03.085531 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:03.003808 28242 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:03.076813 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:03.089236 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:03.102404 26256 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:03.104018 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:03.106333 26256 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
W20260924 14:30:03.106851 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 54253.3us
I20260924 14:30:03.107686 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:03.109781 26405 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:46098
I20260924 14:30:03.110370 26405 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:03.119783 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:03.171530 26405 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:03.171954 26253 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:30:03.173502 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 95858.8us
I20260924 14:30:03.063320 28655 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:03.184798 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 77206.8us
I20260924 14:30:03.185849 26221 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:03.187297 26221 leader_election.cc:398] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:30:03.190920 26221 leader_election.cc:401] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260924 14:30:03.197292 26330 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:30:03.199569 26330 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: []
W20260924 14:30:03.202325 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 171922us
I20260924 14:30:03.047389 28332 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:03.095475 28646 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 7
I20260924 14:30:03.017417 28629 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:03.219084 26323 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:30:03.228152 26325 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 1. Current term is 2. Ops: []
I20260924 14:30:03.228682 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:03.232461 26371 leader_election.cc:334] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.610s (SENT)
I20260924 14:30:03.087177 28736 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:03.105195 28578 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:03.203307 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:03.028390 28640 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:03.221508 28542 consensus_meta.cc:287] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.634s	user 0.008s	sys 0.003s
I20260924 14:30:03.178579 28494 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:03.267176 28542 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:03.202543 28703 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:03.106549 28581 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:03.091670 28632 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:03.319932 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 368312us
W20260924 14:30:03.340966 28751 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260924 14:30:03.444064 26325 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:03.444522 28681 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:03.269847 28647 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:30:03.237370 28676 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:03.145486 28741 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:03.447253 27192 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
W20260924 14:30:03.452324 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 248826us
I20260924 14:30:03.440378 26405 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:03.233515 28752 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260924 14:30:03.434370 28533 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.237s	user 0.012s	sys 0.000s
I20260924 14:30:03.463675 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:03.440363 28367 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:03.230275 28730 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
W20260924 14:30:03.446295 28731 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.593s	user 0.001s	sys 0.001s
I20260924 14:30:03.468806 28541 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:03.395349 28693 consensus_meta.cc:287] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.572s	user 0.008s	sys 0.006s
I20260924 14:30:03.564446 28693 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:03.567457 28693 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:03.569088 28749 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260924 14:30:03.272086 28578 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:03.221323 28529 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:30:03.447947 28744 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260924 14:30:03.202971 28734 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:03.438525 28690 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:03.404531 28695 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:03.582500 28578 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:03.584254 28695 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:03.446503 28735 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:03.290397 28558 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:03.473538 28539 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:03.473773 28494 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:03.342039 28383 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:03.587467 28646 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
W20260924 14:30:03.592036 28706 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.539s	user 0.021s	sys 0.045s
W20260924 14:30:03.592619 28706 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.539s	user 0.022s	sys 0.045s
I20260924 14:30:03.588406 28735 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:03.598101 28735 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:03.289539 28742 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:03.329479 28655 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:03.599794 28655 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:30:03.520339 28759 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260924 14:30:03.470674 28703 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:03.605315 28703 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:03.465742 28542 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:03.457278 28740 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:03.608387 28740 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:03.546646 28725 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:03.547561 28629 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:03.550220 28640 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:03.554185 28686 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:03.613092 28686 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:03.493503 28533 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:03.615482 28533 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:03.561506 28753 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260924 14:30:03.572855 28741 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:03.616776 28741 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:03.617231 28740 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:03.618403 28695 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:03.619328 28703 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:03.414602 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 176176us
I20260924 14:30:03.676817 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 517) took 1710 ms. Trace:
I20260924 14:30:03.670085 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:03.677565 26251 rpcz_store.cc:276] 0924 14:30:01.965700 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.976468 (+ 10768us) service_pool.cc:227] Handling call
0924 14:30:01.976943 (+   475us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:01.991761 (+ 14818us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:01.991799 (+    38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:02.005773 (+ 13974us) log.cc:842] Serialized 26 byte log entry
0924 14:30:02.520582 (+514809us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:02.520657 (+    75us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:02.520806 (+   149us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:03.675738 (+1154932us) raft_consensus.cc:1671] finished
0924 14:30:03.675799 (+    61us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:03.676195 (+   396us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14373,"raft.queue_time_us":24553,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":30860,"thread_start_us":454800,"threads_started":1,"wal-append.queue_time_us":753072}
I20260924 14:30:03.479413 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:03.701601 26327 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:30:03.448561 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 354677us
W20260924 14:30:03.703847 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 224394us
I20260924 14:30:03.705303 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:03.706212 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:03.279651 26330 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:03.708827 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 962) took 1118 ms. Trace:
I20260924 14:30:03.709131 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:03.709244 26330 rpcz_store.cc:276] 0924 14:30:02.590579 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:03.224996 (+634417us) service_pool.cc:227] Handling call
0924 14:30:03.279480 (+ 54484us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:03.708099 (+428619us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.708580 (+   481us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:03.712175 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:03.466154 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 488388us
I20260924 14:30:03.483242 26405 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:03.719362 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:03.721357 27189 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:03.515065 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 591) took 3188 ms. Trace:
I20260924 14:30:03.723719 27190 rpcz_store.cc:276] 0924 14:30:00.302621 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:00.302738 (+   117us) service_pool.cc:227] Handling call
0924 14:30:00.303008 (+   270us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:01.546692 (+1243684us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.490680 (+1943988us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400074c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:03.490921 (+   241us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2600512384}
W20260924 14:30:03.728121 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 264380us
I20260924 14:30:03.728487 27186 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:03.728540 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:03.732298 27182 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:30:03.220780 28187 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:03.737712 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:03.744325 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:03.447016 28242 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:03.226783 28332 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:03.753099 28524 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:03.754436 28646 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:30:03.755923 28535 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:03.756016 28539 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:03.757197 28535 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:03.759408 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:03.764627 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 59719.8us
I20260924 14:30:03.754274 28524 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:03.779426 27188 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:03.779914 28592 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:03.747004 28242 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:03.789093 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:03.788779 28676 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:03.791653 28242 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:03.793334 28524 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:03.794441 28740 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:03.796269 28742 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:03.797243 28742 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:03.783982 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:03.798451 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:03.799088 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:03.806409 26330 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:30:03.807464 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 627) took 1189 ms. Trace:
I20260924 14:30:03.807889 26330 rpcz_store.cc:276] 0924 14:30:02.617669 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:03.711141 (+1093472us) service_pool.cc:227] Handling call
0924 14:30:03.805990 (+ 94849us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:03.807125 (+  1135us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.807370 (+   245us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:03.809895 28524 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:30:03.810309 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 967) took 1182 ms. Trace:
I20260924 14:30:03.810650 28524 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:03.813350 28592 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:03.814373 28592 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:03.757236 28539 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:03.810855 26330 rpcz_store.cc:276] 0924 14:30:02.628158 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:03.808743 (+1180585us) service_pool.cc:227] Handling call
0924 14:30:03.809289 (+   546us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:03.809839 (+   550us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:03.809886 (+    47us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:03.809962 (+    76us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:03.810032 (+    70us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.810096 (+    64us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:03.810225 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:30:03.801354 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 65289.2us
I20260924 14:30:03.798391 28242 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:03.825856 27182 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:03.826699 28539 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:03.812479 28524 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:03.829211 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:03.829492 28703 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:30:03.832399 28690 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:03.840971 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 640) took 1814 ms. Trace:
I20260924 14:30:03.841518 26249 rpcz_store.cc:276] 0924 14:30:02.026531 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.577518 (+550987us) service_pool.cc:227] Handling call
0924 14:30:02.577776 (+   258us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:02.999225 (+421449us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.840430 (+841205us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b54007823d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:03.840792 (+   362us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":885729152}
I20260924 14:30:03.842723 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:30:03.844507 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 528) took 1214 ms. Trace:
I20260924 14:30:03.844911 26252 rpcz_store.cc:276] 0924 14:30:02.629928 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.691166 (+ 61238us) service_pool.cc:227] Handling call
0924 14:30:02.691678 (+   512us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:03.034692 (+343014us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.844041 (+809349us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400288f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:03.844346 (+   305us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":774775040}
I20260924 14:30:03.850010 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:03.854305 28760 consensus_queue.cc:1046] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:03.856050 28760 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:03.856492 28760 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:03.858569 28760 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:03.862749 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:03.863771 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 609) took 2038 ms. Trace:
I20260924 14:30:03.864820 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 966) took 1083 ms. Trace:
W20260924 14:30:03.890647 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 178636us
I20260924 14:30:03.892038 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:03.893157 28690 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:30:03.896245 26252 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260924 14:30:03.881702 27186 rpcz_store.cc:276] 0924 14:30:01.825115 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.827118 (+  2003us) service_pool.cc:227] Handling call
0924 14:30:01.827402 (+   284us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:03.729284 (+1901882us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.863408 (+134124us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b54000028d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:03.863682 (+   274us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4369818496}
W20260924 14:30:03.936579 26332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 614) took 2780 ms (client timeout 1749 ms). Trace:
I20260924 14:30:03.882249 26406 rpcz_store.cc:276] 0924 14:30:02.780973 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.782120 (+  1147us) service_pool.cc:227] Handling call
0924 14:30:02.782417 (+   297us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:02.949225 (+166808us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.864120 (+914895us) spinlock_profiling.cc:243] Waited 82.9 ms on lock 0x7b5400476154. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:03.864531 (+   411us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":190760704}
W20260924 14:30:03.937316 26332 rpcz_store.cc:269] 0924 14:30:01.155490 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.155613 (+   123us) service_pool.cc:227] Handling call
0924 14:30:03.936017 (+2780404us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540012eb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:03.936272 (+   255us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3192908160}
I20260924 14:30:03.821611 28592 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:03.817494 26250 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:03.950806 28742 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:03.951264 26080 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 4) took 18756 ms (client timeout 15000 ms). Trace:
I20260924 14:30:03.961122 28635 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:30:03.965740 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:03.965752 28372 raft_consensus.cc:979] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:03.967767 26253 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:03.957183 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:03.969007 26252 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:03.967774 28372 raft_consensus.cc:1067] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:03.970418 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:03.970907 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 79106us
I20260924 14:30:03.958557 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 607) took 2299 ms. Trace:
I20260924 14:30:03.973927 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:03.981087 28760 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:03.952865 26080 rpcz_store.cc:269] 0924 14:29:45.194636 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:45.195575 (+   939us) service_pool.cc:227] Handling call
0924 14:30:03.950874 (+18755299us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:29:58.296717 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:29:58.338351 (+ 41634us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:29:58.338393 (+    42us) write_op.cc:435] Acquired schema lock
0924 14:29:58.338404 (+    11us) tablet.cc:660] Decoding operations
0924 14:29:58.339390 (+   986us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:29:58.339470 (+    80us) write_op.cc:641] Partition lock acquired for write op
0924 14:29:58.339494 (+    24us) tablet.cc:683] Acquiring locks for 11 operations
0924 14:29:58.340389 (+   895us) tablet.cc:699] Row locks acquired
0924 14:29:58.340402 (+    13us) write_op.cc:260] PREPARE: finished
0924 14:29:58.340542 (+   140us) write_op.cc:270] Start()
0924 14:29:58.340652 (+   110us) write_op.cc:276] Timestamp: P: 1790260198340496 usec, L: 0
0924 14:29:58.340673 (+    21us) op_driver.cc:331] REPLICATION: starting
0924 14:29:58.341478 (+   805us) log.cc:842] Serialized 4446 byte log entry
0924 14:29:59.130119 (+788641us) op_driver.cc:444] REPLICATION: finished
0924 14:29:59.716942 (+586823us) write_op.cc:301] APPLY: starting
0924 14:29:59.718725 (+  1783us) tablet.cc:1365] starting BulkCheckPresence
0924 14:29:59.759217 (+ 40492us) tablet.cc:1368] finished BulkCheckPresence
0924 14:29:59.759356 (+   139us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:29:59.760750 (+  1394us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:29:59.761055 (+   305us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0924 14:29:59.761108 (+    53us) write_op.cc:312] APPLY: finished
0924 14:29:59.761432 (+   324us) log.cc:842] Serialized 82 byte log entry
0924 14:29:59.761606 (+   174us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:59.761755 (+   149us) write_op.cc:454] Released schema lock
0924 14:29:59.761820 (+    65us) write_op.cc:341] FINISH: Updating metrics
0924 14:29:59.762159 (+   339us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:29:59.762190 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":665655,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":515329,"apply.run_cpu_time_us":3993,"apply.run_wall_time_us":51673,"num_ops":11,"prepare.queue_time_us":1147527,"prepare.run_cpu_time_us":54699,"prepare.run_wall_time_us":342396,"raft.queue_time_us":271065,"raft.run_cpu_time_us":10515,"raft.run_wall_time_us":590704,"replication_time_us":724995,"thread_start_us":1003480,"threads_started":4,"wal-append.queue_time_us":306347,"wal-append.run_cpu_time_us":38326,"wal-append.run_wall_time_us":2108180}]]}
I20260924 14:30:03.963130 28635 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:03.972676 27185 rpcz_store.cc:276] 0924 14:30:01.637559 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.638763 (+  1204us) service_pool.cc:227] Handling call
0924 14:30:01.639032 (+   269us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:02.799910 (+1160878us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.845056 (+1045146us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540079f9d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:03.936463 (+ 91407us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2598721408}
I20260924 14:30:03.983687 28635 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:03.956400 28524 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:03.985333 28635 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:03.988904 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:03.990279 26404 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:03.990362 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 496) took 2536 ms. Trace:
I20260924 14:30:04.005976 26252 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260924 14:30:04.006745 26257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 631) took 1982 ms (client timeout 1610 ms). Trace:
W20260924 14:30:04.007243 26257 rpcz_store.cc:269] 0924 14:30:02.024165 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.024964 (+   799us) service_pool.cc:227] Handling call
0924 14:30:04.006226 (+1981262us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54002a1054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.006519 (+   293us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1619831168}
I20260924 14:30:03.991143 26403 rpcz_store.cc:276] 0924 14:30:01.454018 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.454703 (+   685us) service_pool.cc:227] Handling call
0924 14:30:01.454952 (+   249us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:02.006679 (+551727us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.989721 (+1983042us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54001c1454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:03.990133 (+   412us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":968898816}
W20260924 14:30:04.013149 26329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 953) took 1816 ms (client timeout 2213 ms). Trace:
W20260924 14:30:04.013674 26329 rpcz_store.cc:269] 0924 14:30:02.196352 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.196544 (+   192us) service_pool.cc:227] Handling call
0924 14:30:04.012701 (+1816157us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b540001efd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.012930 (+   229us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1333892480}
I20260924 14:30:04.019699 27188 raft_consensus.cc:2396] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260924 14:30:04.032653 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 349398us
I20260924 14:30:04.038761 26253 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:04.044252 26252 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260924 14:30:04.048261 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 253188us
I20260924 14:30:04.050050 28372 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:04.051848 28592 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:04.052300 28592 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:04.053660 28592 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:04.055351 28592 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:04.061636 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 64470.8us
I20260924 14:30:04.058202 28635 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:04.065327 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.075484 26403 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
W20260924 14:30:04.077862 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 95061.7us
W20260924 14:30:04.081368 28609 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260924 14:30:04.083011 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 346) took 2138 ms. Trace:
I20260924 14:30:04.083489 26324 rpcz_store.cc:276] 0924 14:30:01.944919 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.034550 (+ 89631us) service_pool.cc:227] Handling call
0924 14:30:02.088512 (+ 53962us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:02.938167 (+849655us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:02.938201 (+    34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:02.938257 (+    56us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:02.938286 (+    29us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:04.082523 (+1144237us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b5400498254. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:04.082589 (+    66us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:04.082797 (+   208us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1953430912}
I20260924 14:30:04.086542 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:04.089689 26249 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:04.096279 28529 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:04.097268 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 337) took 2202 ms. Trace:
I20260924 14:30:04.097822 28529 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:04.098129 28734 raft_consensus.cc:3032] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:30:04.098697 28734 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:04.100365 26323 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:04.100104 28734 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:04.101940 28734 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:04.102293 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 963) took 1511 ms. Trace:
I20260924 14:30:04.097836 26397 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260924 14:30:04.102804 26323 rpcz_store.cc:276] 0924 14:30:02.590859 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:03.228195 (+637336us) service_pool.cc:227] Handling call
0924 14:30:04.100026 (+871831us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:04.100928 (+   902us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:04.102029 (+  1101us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54005bb3d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.102198 (+   169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1581896576}
I20260924 14:30:04.097738 26327 rpcz_store.cc:276] 0924 14:30:01.894806 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.894933 (+   127us) service_pool.cc:227] Handling call
0924 14:30:04.096955 (+2202022us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54005bb3d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.097191 (+   236us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3943280896}
I20260924 14:30:04.107484 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:04.107753 28619 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:04.108430 28619 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.111099 28619 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:04.079738 28719 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.465s	user 0.007s	sys 0.049s
I20260924 14:30:04.115221 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.116089 28372 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:04.098752 28529 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:04.119707 28554 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:04.121176 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:04.119410 28719 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.544s	user 0.008s	sys 0.050s
W20260924 14:30:04.129101 26255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 627) took 2106 ms (client timeout 2206 ms). Trace:
I20260924 14:30:04.128665 28554 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:04.123797 28372 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:04.130733 28642 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:04.132254 28642 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:04.132444 28554 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:04.132680 28642 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:04.133368 26406 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:04.133798 28642 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:04.134646 28581 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:04.135864 28642 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:04.137143 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 604) took 2837 ms. Trace:
W20260924 14:30:04.129619 26255 rpcz_store.cc:269] 0924 14:30:02.022555 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.023532 (+   977us) service_pool.cc:227] Handling call
0924 14:30:04.128648 (+2105116us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54004f9f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.128918 (+   270us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1537328384}
I20260924 14:30:04.137593 27191 rpcz_store.cc:276] 0924 14:30:01.299436 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.299603 (+   167us) service_pool.cc:227] Handling call
0924 14:30:02.385954 (+1086351us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:02.582358 (+196404us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:04.136699 (+1554341us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54006c5050. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.136983 (+   284us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2497481472}
I20260924 14:30:04.141517 28529 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:04.144609 28372 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:04.146286 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 610) took 2319 ms. Trace:
I20260924 14:30:04.146723 27194 rpcz_store.cc:276] 0924 14:30:01.827250 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.827392 (+   142us) service_pool.cc:227] Handling call
0924 14:30:01.876219 (+ 48827us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:02.778004 (+901785us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:04.145855 (+1367851us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b540000d054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:04.146108 (+   253us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1869876096}
I20260924 14:30:04.147303 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.148454 26401 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:04.149546 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:04.150471 26258 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:04.150838 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1068) took 1200 ms. Trace:
I20260924 14:30:04.152547 28234 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:04.151166 26402 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:46098
I20260924 14:30:04.156486 28644 raft_consensus.cc:3032] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:30:04.156667 26402 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:04.157027 28644 raft_consensus.cc:729] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:04.157254 26402 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:04.157614 26369 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:04.158665 28638 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:04.158216 28644 consensus_queue.cc:260] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:04.157294 26079 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 5, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 5 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:04.151160 26258 rpcz_store.cc:276] 0924 14:30:02.950135 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:03.070273 (+120138us) service_pool.cc:227] Handling call
0924 14:30:04.149245 (+1078972us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:30:04.150784 (+  1539us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13711616}
I20260924 14:30:04.160307 26079 catalog_manager.cc:5495] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
W20260924 14:30:04.156363 26297 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.521s (SENT) [suppressed 32 similar messages]
I20260924 14:30:04.162911 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:04.166196 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:04.162678 26079 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
W20260924 14:30:04.172114 26326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 341) took 2275 ms (client timeout 2138 ms). Trace:
W20260924 14:30:04.172953 26331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 955) took 1817 ms (client timeout 1590 ms). Trace:
I20260924 14:30:04.173162 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:04.173379 26331 rpcz_store.cc:269] 0924 14:30:02.355169 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.355411 (+   242us) service_pool.cc:227] Handling call
0924 14:30:04.172648 (+1817237us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540001e5d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.172780 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":711552768}
I20260924 14:30:04.172617 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 615) took 1312 ms. Trace:
I20260924 14:30:04.174852 27192 rpcz_store.cc:276] 0924 14:30:02.859985 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.861896 (+  1911us) service_pool.cc:227] Handling call
0924 14:30:02.916522 (+ 54626us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:03.481340 (+564818us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:04.172143 (+690803us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b540000f854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:04.172439 (+   296us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1219965696}
I20260924 14:30:04.176170 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 636) took 2150 ms. Trace:
I20260924 14:30:04.176623 26325 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:30:04.176568 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1066) took 1227 ms. Trace:
W20260924 14:30:04.172312 26328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 619) took 1602 ms (client timeout 1839 ms). Trace:
I20260924 14:30:04.177230 28332 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:04.179611 26250 rpcz_store.cc:276] 0924 14:30:02.949564 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:03.103687 (+154123us) service_pool.cc:227] Handling call
0924 14:30:04.176374 (+1072687us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b54002882d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.176512 (+   138us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1307272448}
I20260924 14:30:04.180730 26398 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
W20260924 14:30:04.179126 26328 rpcz_store.cc:269] 0924 14:30:02.569425 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.569555 (+   130us) service_pool.cc:227] Handling call
0924 14:30:04.171890 (+1602335us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400497d54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.172125 (+   235us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":268091264}
W20260924 14:30:04.172796 26326 rpcz_store.cc:269] 0924 14:30:01.896203 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:01.900387 (+  4184us) service_pool.cc:227] Handling call
0924 14:30:04.171663 (+2271276us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54005582d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.171889 (+   226us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2027355520}
I20260924 14:30:04.182796 26254 rpcz_store.cc:276] 0924 14:30:02.025491 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:02.112968 (+ 87477us) service_pool.cc:227] Handling call
0924 14:30:02.113390 (+   422us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:02.943360 (+829970us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:04.175642 (+1232282us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54003822d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:04.175977 (+   335us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1679113984}
I20260924 14:30:04.184506 28636 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:04.185153 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 970) took 1113 ms. Trace:
I20260924 14:30:04.185402 28636 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:04.185664 26398 rpcz_store.cc:276] 0924 14:30:03.078706 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:03.079170 (+   464us) service_pool.cc:227] Handling call
0924 14:30:03.110075 (+ 30905us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:04.181163 (+1071088us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:04.184836 (+  3673us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002fc6d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:04.185079 (+   243us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2461424384}
I20260924 14:30:04.187906 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 352) took 1208 ms. Trace:
I20260924 14:30:04.188477 26330 rpcz_store.cc:276] 0924 14:30:02.979405 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:03.942657 (+963252us) service_pool.cc:227] Handling call
0924 14:30:04.178365 (+235708us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:04.178806 (+   441us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:04.178848 (+    42us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:04.178923 (+    75us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:04.178985 (+    62us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:04.179047 (+    62us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:04.187615 (+  8568us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540001e5d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.187814 (+   199us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":362293632}
I20260924 14:30:04.169265 28644 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:04.189229 28636 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:04.196158 28763 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:04.203771 27125 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:04.203600 28234 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:04.205029 26402 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.216516 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:04.216883 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:04.218916 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:04.222215 26402 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:04.205679 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 46430.1us
I20260924 14:30:04.232834 28638 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:30:04.176431 27123 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:04.211272 28529 consensus_queue.cc:1046] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:04.211776 28332 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:04.213150 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:04.236451 28529 raft_consensus.cc:3032] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:04.213528 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:04.178133 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 626) took 1584 ms. Trace:
I20260924 14:30:04.214998 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:04.235261 28698 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:30:04.236686 26254 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:04.239432 28698 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:04.239646 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:04.236855 28529 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:04.237571 26325 rpcz_store.cc:276] 0924 14:30:02.603198 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:03.469983 (+866785us) service_pool.cc:227] Handling call
0924 14:30:04.176221 (+706238us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:04.177114 (+   893us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:04.177835 (+   721us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54005582d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:04.178062 (+   227us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1623167232}
I20260924 14:30:04.240442 28698 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:04.238381 26258 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:04.241465 28529 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:04.206187 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1069) took 1138 ms. Trace:
I20260924 14:30:04.240363 26252 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:04.242148 28698 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:30:04.250300 28635 consensus_queue.cc:1046] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:04.252148 28648 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:04.254892 28326 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:04.255373 28326 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:04.256582 28326 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:04.258442 28326 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:04.258162 28332 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:04.250818 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:04.250396 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:04.268517 28763 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:04.268712 28648 raft_consensus.cc:3032] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:04.269189 28648 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:04.261993 28740 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:04.269992 28740 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:04.270139 28648 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:04.271121 28740 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.272581 28740 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:04.273736 28635 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:04.274187 28635 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:04.274571 28635 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:04.277704 28763 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:04.267264 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61740.1us
W20260924 14:30:04.284910 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 128706us
I20260924 14:30:04.272758 26079 catalog_manager.cc:5495] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:04.286909 26252 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:04.288833 28613 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:04.289590 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:04.289522 28613 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:04.289971 28648 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:04.290705 28691 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:04.291749 28595 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260924 14:30:04.292336 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 162190us
I20260924 14:30:04.289597 28524 consensus_queue.cc:1046] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260924 14:30:04.242937 26256 rpcz_store.cc:276] 0924 14:30:02.950486 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:03.165229 (+214743us) service_pool.cc:227] Handling call
0924 14:30:03.165675 (+   446us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:03.674746 (+509071us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:03.674893 (+   147us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:03.676203 (+  1310us) log.cc:842] Serialized 26 byte log entry
0924 14:30:03.798385 (+122182us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:03.952414 (+154029us) log.cc:842] Serialized 14 byte log entry
0924 14:30:03.958665 (+  6251us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:03.999890 (+ 41225us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:04.088456 (+ 88566us) raft_consensus.cc:1671] finished
0924 14:30:04.088475 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:04.088629 (+   154us) inbound_call.cc:173] Queueing success response
0924 14:30:04.167030 (+ 78401us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b180024c580. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
Metrics: {"mutex_wait_us":56086,"prepare.queue_time_us":242525,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":163876,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":63446,"spinlock_wait_cycles":54368128,"thread_start_us":649714,"threads_started":3,"wal-append.queue_time_us":281405}
I20260924 14:30:04.302155 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.302757 28676 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260924 14:30:04.304121 26254 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:04.303001 26402 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:30:04.284121 28529 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:04.310158 26252 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:04.293561 28332 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:04.309857 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:04.321784 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:04.308797 26079 catalog_manager.cc:5495] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:04.324442 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:04.346630 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:04.338052 27125 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:04.355710 26079 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:04.357774 26079 catalog_manager.cc:5495] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 5, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 5 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:04.359589 28494 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:04.360174 28494 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:04.360600 28494 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:04.364378 28763 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:04.340303 28613 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:04.375994 28728 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260924 14:30:04.360626 28581 consensus_queue.cc:1046] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260924 14:30:04.347895 28592 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:04.345230 28638 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260924 14:30:04.379115 28581 raft_consensus.cc:3032] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:04.321868 28332 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.342140 28686 raft_consensus.cc:3032] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:04.389474 26079 catalog_manager.cc:5495] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
W20260924 14:30:04.390635 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52694.6us
I20260924 14:30:04.395561 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:04.415352 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:04.422750 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:04.423897 26253 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:04.424933 28578 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:30:04.422422 28595 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:04.426195 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:04.426995 26255 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:04.426265 26079 catalog_manager.cc:5495] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 6, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 6 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:04.429257 26079 catalog_manager.cc:5495] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 5, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 5 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:04.430840 26079 catalog_manager.cc:5495] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:04.428318 26252 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:30:04.379935 28581 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:04.433064 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 108732us
I20260924 14:30:04.435398 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:04.436542 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:04.446259 26256 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:04.389783 28686 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:04.453611 26255 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:04.459947 27192 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:30:04.453846 27194 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
I20260924 14:30:04.456305 28613 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:04.465565 28613 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:04.506486 28710 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:04.507112 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:04.507217 28710 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.508262 28676 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:04.508487 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:04.509366 28710 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:04.509428 28676 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:04.509893 28332 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:04.480798 28613 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.518404 26253 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:04.527966 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:04.528434 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:04.529045 27194 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:04.529739 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:04.530176 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:04.530789 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 107805us
I20260924 14:30:04.531920 26397 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:04.532106 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.532841 28710 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:04.533036 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:04.533643 28710 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:04.537335 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:04.539213 28372 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:04.539781 28372 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:04.540218 28372 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:04.540561 28760 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:04.541006 28760 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:04.541893 28760 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:04.528077 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:04.533651 26401 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:04.529222 26403 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:30:04.547237 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 129554us
I20260924 14:30:04.471546 28648 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:04.516144 26079 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:04.553787 28740 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:04.543346 28760 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:04.572184 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:04.573230 28591 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:04.574067 28591 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:04.523574 28769 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:04.580833 28610 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:04.581377 28610 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:04.582388 28610 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:04.552780 28332 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:04.584244 26398 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:30:04.600746 26079 catalog_manager.cc:5495] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 5, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:04.606570 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.607848 26399 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:04.609076 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.577584 28754 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:04.583650 28648 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:04.570755 28740 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.626292 26079 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:04.622951 27123 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:04.522658 28581 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.653497 26401 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:04.616895 28754 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:04.662987 26079 catalog_manager.cc:5495] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 6, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 6 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:04.642731 28610 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:04.599378 28332 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:04.672868 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:04.585529 28769 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:04.691056 26400 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:30:04.651662 28372 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:04.699088 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 194475us
I20260924 14:30:04.681062 28698 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:04.700098 26079 catalog_manager.cc:5495] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 3, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 3 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:04.705031 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:04.711654 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.692487 28595 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:04.718570 26402 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260924 14:30:04.718693 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 138178us
I20260924 14:30:04.715579 26079 catalog_manager.cc:5495] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:04.721271 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:04.598984 28686 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:04.732551 26369 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 1.812s (SENT)
I20260924 14:30:04.733079 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:04.741992 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:04.744544 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:04.728425 28754 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.738276 28591 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:04.748649 28372 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:04.749655 28494 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:04.738153 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:04.752108 28592 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:04.752446 28635 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.752938 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:04.753363 28592 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:04.753778 28592 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:04.754401 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:04.754611 28635 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:04.755213 28691 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:04.755947 28691 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:30:04.756969 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:04.758085 28740 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:04.758229 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:04.759722 28332 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:04.759884 27185 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:04.759994 26325 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:30:04.760318 28332 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:04.761204 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:04.761844 27186 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 9
I20260924 14:30:04.761727 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.756836 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:04.758304 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:04.762699 26399 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:04.739619 26079 catalog_manager.cc:5495] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 4, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:04.760068 26326 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:04.761395 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:04.766875 26331 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:04.767969 26328 raft_consensus.cc:2354] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:04.777498 26079 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:04.778891 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:04.776778 27194 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:04.714713 28698 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:04.785280 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:04.786437 26330 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:04.754663 28592 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:04.790459 28769 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:04.790943 28592 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:04.796602 28698 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:04.799963 26224 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:04.801110 28529 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:30:04.801702 28529 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:04.801777 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:04.806493 26403 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:04.819420 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:04.823173 26328 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260924 14:30:04.832556 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 90669.5us
I20260924 14:30:04.781522 28581 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:04.786187 28763 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:30:04.840704 26399 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:04.847095 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:04.852185 26253 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:04.834980 27194 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:04.842784 26402 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:04.844820 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:04.845610 26401 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:04.848363 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:04.855239 26398 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:04.781585 28636 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260924 14:30:04.840723 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.802815 28529 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:04.846491 28724 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:30:04.871089 27123 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:04.880348 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:04.881121 26331 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:04.882802 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:04.891966 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:04.862435 28494 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:04.850296 28734 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:04.910970 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:04.894388 28635 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:04.913162 26331 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:04.927636 28789 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
W20260924 14:30:04.937443 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 88391.3us
I20260924 14:30:04.935042 26398 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:04.909129 28724 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:04.942471 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 138746us
I20260924 14:30:04.948101 28630 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:04.948344 28724 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:04.952389 28789 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:04.953194 28789 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:04.955832 28789 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:04.965286 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:04.966466 28740 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:04.967696 28740 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:04.968281 28740 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:04.968580 28740 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:04.969367 28740 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:04.970474 28740 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:30:04.975958 28787 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260924 14:30:04.980999 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:04.982230 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:04.967343 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:05.005167 27186 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:05.010533 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.017544 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:05.003610 28630 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:05.037685 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 305145us
I20260924 14:30:05.008198 28760 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:05.018803 28789 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:05.031883 28648 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.032711 28635 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:05.045701 28648 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.050485 28648 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:05.052237 28533 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260924 14:30:05.055685 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 88439.9us
I20260924 14:30:05.051301 28760 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:05.056452 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 150074us
W20260924 14:30:05.057279 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 114033us
I20260924 14:30:05.060681 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:05.056105 28791 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260924 14:30:05.061645 28734 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:05.070545 26251 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:05.059397 28635 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:05.079581 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:05.086030 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:05.078951 28630 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:05.067550 28372 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:05.096319 28760 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:05.102011 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:05.097676 28734 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.085548 26330 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:05.101980 28636 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260924 14:30:05.104565 28635 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:05.108130 26224 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:05.108568 26404 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:05.100208 28642 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.112233 28642 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:05.113622 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 67711.3us
I20260924 14:30:05.113804 28638 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:05.114188 28642 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:05.114467 28638 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:05.114537 26297 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:05.118878 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 205865us
I20260924 14:30:05.122655 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:05.123265 26403 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:30:05.129572 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:05.131299 26402 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:05.133073 26399 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:30:05.136264 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:05.137286 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:05.137959 26403 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:05.126461 28698 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:05.141992 28698 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:05.142994 28698 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.121794 28760 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260924 14:30:05.150214 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.131923 28642 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:05.123596 28648 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260924 14:30:05.138480 28592 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260924 14:30:05.180725 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 66311.6us
I20260924 14:30:05.156937 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:05.186617 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 112815us
I20260924 14:30:05.182299 26251 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:05.149402 28372 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:05.127022 28591 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260924 14:30:05.149087 28797 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:05.213166 28524 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:05.218502 28591 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.161540 28636 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:05.165422 28754 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:05.227314 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:05.240475 26403 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:05.242533 28610 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260924 14:30:05.244901 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 115679us
I20260924 14:30:05.247069 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:05.248740 27123 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:05.260649 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:05.222932 28524 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:05.141243 28638 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.230108 28529 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:05.230352 28372 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:05.279716 28640 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
W20260924 14:30:05.280426 28798 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260924 14:30:05.281647 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 46929.8us
W20260924 14:30:05.282330 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 210122us
I20260924 14:30:05.282773 28234 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:30:05.159838 28793 consensus_queue.cc:1046] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.141s
I20260924 14:30:05.244653 28719 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:05.251533 28636 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:05.270699 28524 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:05.287288 28640 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:05.302084 28793 raft_consensus.cc:3032] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:05.301883 28805 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:05.293277 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:05.303822 26326 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:05.270433 28595 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:05.236130 28591 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:05.310881 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 135057us
I20260924 14:30:05.302433 28719 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:05.330719 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:05.331902 26330 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:05.330950 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:05.339915 28648 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:05.340755 28591 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:05.340531 28648 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.325717 28595 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:05.341634 28595 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:05.314065 28793 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:05.349406 28800 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260924 14:30:05.349562 28638 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:05.350052 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:05.284818 28797 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260924 14:30:05.359884 26221 leader_election.cc:398] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:30:05.360181 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:05.360221 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 99713.2us
I20260924 14:30:05.334194 28686 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:05.334357 28592 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:05.363183 26330 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:05.308525 28529 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.369561 27126 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:05.369779 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:05.387426 26331 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:05.365341 28592 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:05.390753 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 88049.8us
I20260924 14:30:05.365053 28640 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.392282 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:05.391649 28592 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:05.375898 28635 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.393863 28635 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:05.395268 28799 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:05.396200 28799 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:05.396353 28524 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.397758 28799 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:30:05.398216 28799 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:05.398516 28524 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:05.392791 28640 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:05.404270 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:05.421331 26399 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:05.404040 28799 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:05.388484 28728 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:05.409485 28795 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:05.413337 28740 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.388376 28793 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:05.385573 28809 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:30:05.414034 28797 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.415750 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:05.375599 28636 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.423774 28814 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.418543 28719 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:05.435062 28797 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:05.446484 28797 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:05.436443 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:05.434494 28793 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:05.447626 26400 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:05.450147 26298 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.431970 28740 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:05.399895 28592 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:05.453271 28740 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:05.438644 28814 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.455322 28635 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:05.456072 28635 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:05.456432 28635 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:05.456748 28635 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:05.457280 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.457651 28814 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:05.457979 27186 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:30:05.459218 26295 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.464283 26404 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:05.467744 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:05.468199 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:05.468890 26331 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:05.469558 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:05.463894 26399 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:30:05.471037 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:05.471369 26402 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260924 14:30:05.473800 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:05.474066 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.475158 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:05.477666 26401 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 3.
I20260924 14:30:05.485226 28719 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:05.487720 28691 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:30:05.488533 28691 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:05.489655 28691 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.440090 28648 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:05.364964 28372 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.498363 26298 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.501709 28805 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:05.502828 26325 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:05.506438 26325 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:30:05.508342 28648 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:05.508859 28648 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:05.509212 28529 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:05.509305 28648 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:05.514212 28635 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.515771 28799 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:05.516536 27186 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:05.428740 28595 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:05.517205 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.517848 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.518626 27194 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:05.518488 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:05.519129 28595 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:05.519593 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:05.519917 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 116869us
I20260924 14:30:05.522267 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:05.516531 28529 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:05.523021 26251 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:05.523061 28529 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:05.525795 28529 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:05.520730 28595 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:05.470412 28524 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:05.527635 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:05.530321 26330 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:05.533550 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.506037 28814 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.558789 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:05.561019 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.562593 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:05.563272 26402 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:05.531207 28524 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:05.555727 28740 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:05.567687 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.567801 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:05.577186 27191 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:05.577409 28754 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:05.577639 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.577986 28754 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:05.578847 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.578984 28754 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.549554 28610 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.564790 28814 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:05.582114 28814 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:05.585700 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.579824 28789 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
W20260924 14:30:05.569455 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 48143.6us
I20260924 14:30:05.589488 27194 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:05.532966 28638 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260924 14:30:05.589767 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:05.591413 26325 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:05.566332 28797 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:30:05.566746 28524 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:05.557760 28529 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260924 14:30:05.601224 26251 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:05.575224 28740 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:05.602185 27191 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:05.609714 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:05.614705 26402 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:05.618922 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.620147 27191 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 4: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:05.620877 26325 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:05.622638 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:05.622778 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:05.623714 26327 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:30:05.624814 28820 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260924 14:30:05.625015 26328 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:05.621028 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:05.627705 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:05.628512 26329 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:05.629220 28814 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.629778 28814 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.630956 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:05.620904 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:05.617981 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:05.631948 28814 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:05.635740 26325 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:05.632009 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:05.603253 28789 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:05.591416 28581 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260924 14:30:05.614712 28826 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.656823 28807 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.661645 27191 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:30:05.615981 28740 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:05.588122 28610 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:05.619083 28636 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:05.679201 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.679839 27194 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:05.666579 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:05.667330 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 77621.4us
I20260924 14:30:05.668978 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:05.635017 28797 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:30:05.664430 28769 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:05.667224 28635 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:05.690696 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.676681 28698 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:05.695688 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.696089 26402 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:05.697423 26221 leader_election.cc:398] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:05.699644 26331 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:05.682857 28760 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260924 14:30:05.701217 26298 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.686189 28797 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:05.739485 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.740898 26329 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:05.738919 26326 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:05.747700 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.749461 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.693542 28529 raft_consensus.cc:2741] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260924 14:30:05.746145 26330 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:05.694916 28769 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:05.750738 26332 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:05.752503 26252 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260924 14:30:05.748692 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47591.5us
W20260924 14:30:05.754242 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 56933.6us
I20260924 14:30:05.755251 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:05.753484 28640 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:05.761291 28809 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:05.763299 28638 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:05.690376 28636 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260924 14:30:05.675617 28807 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.770226 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:05.767803 28640 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:05.734581 28830 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.779925 28830 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:05.780360 26258 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:30:05.771867 28794 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:05.781141 28794 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:05.781556 28794 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:05.782385 28830 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:05.783411 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.784111 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:05.784237 28698 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:05.784307 27191 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:05.784747 28698 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:05.763942 28638 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:05.785295 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:05.785564 28638 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:05.786263 28734 raft_consensus.cc:2741] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260924 14:30:05.786859 28734 raft_consensus.cc:2741] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260924 14:30:05.788107 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.789077 28799 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:05.676524 28826 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:05.789620 28799 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:05.707336 28814 consensus_queue.cc:1046] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260924 14:30:05.793190 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:05.794991 26331 raft_consensus.cc:2379] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:05.794811 26295 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:05.790637 28799 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:05.767256 28592 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.802124 26323 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260924 14:30:05.805114 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:05.805927 26252 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:05.802492 26323 pending_rounds.cc:76] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 3
I20260924 14:30:05.799898 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:05.811630 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.812297 27188 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:05.768548 26330 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:05.812475 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.816172 26256 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
W20260924 14:30:05.816387 28829 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260924 14:30:05.817209 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:05.817879 26254 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:05.817660 28797 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.780689 28524 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:05.818984 28797 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:05.820504 28640 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.821242 28728 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:05.821928 27185 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:05.823527 28524 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:05.849041 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:05.850080 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:05.824820 28833 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:05.851673 28833 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:05.852078 28833 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:05.809278 26323 pending_rounds.cc:97] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260924 14:30:05.855690 26323 raft_consensus.cc:2866] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:05.827466 28805 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:05.859879 26253 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:05.828951 28807 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:05.830224 28648 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.834568 28529 raft_consensus.cc:3032] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:05.835223 28610 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:05.757664 28789 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:05.811661 28769 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:05.844515 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:05.847245 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.884122 26256 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:05.817090 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:05.884205 27185 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:05.802754 28826 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:05.878450 26324 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:05.886281 26254 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:05.888614 26250 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:05.882663 28529 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:05.898099 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:05.907692 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 157143us
I20260924 14:30:05.907729 27194 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:05.932761 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:05.904037 26252 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:30:05.960584 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:05.961620 26252 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:05.880543 28728 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260924 14:30:05.962622 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:05.937752 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:05.964726 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 77042.5us
I20260924 14:30:05.965014 28808 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:05.967777 28630 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:05.969525 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:05.940824 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:05.971974 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:05.974967 26252 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:05.944909 26403 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260924 14:30:05.975996 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 85077.5us
I20260924 14:30:05.945549 26258 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:05.811113 28592 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:05.885710 28234 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260924 14:30:05.946868 28816 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.947501 28610 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:05.948583 28807 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:05.949173 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:05.949268 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:05.950348 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:05.998677 27191 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:05.950451 28648 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:05.940828 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:05.910188 26249 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260924 14:30:05.976886 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:05.981492 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 232066us
I20260924 14:30:05.982178 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:06.001595 27194 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:06.001724 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:05.991324 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:05.997308 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:06.004428 26254 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:06.004189 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:06.005615 27188 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:30:06.006254 26249 pending_rounds.cc:76] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 1
I20260924 14:30:06.006629 26249 pending_rounds.cc:97] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260924 14:30:06.007123 26249 raft_consensus.cc:2866] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:06.004338 27185 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:06.009133 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:06.009835 26326 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:06.011135 26295 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:06.012238 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:06.013979 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:05.897727 28529 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:06.017398 28796 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:06.014729 28814 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:06.017921 28529 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:06.018302 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.018486 26404 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260924 14:30:06.018155 28814 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:06.019840 26323 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:30:06.020277 28814 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:06.000012 28613 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260924 14:30:06.020977 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:05.980926 28581 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260924 14:30:06.022071 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:05.996851 28728 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:06.024503 28728 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:06.025399 28728 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:06.026593 28728 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:06.028664 28592 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:06.031626 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 62208.3us
I20260924 14:30:06.033576 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:06.034221 26330 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:30:06.034860 26325 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:06.035998 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:06.036463 28638 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.037582 26328 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260924 14:30:06.038479 28638 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:06.039088 28740 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.041170 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:06.015977 28610 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.042524 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:06.043162 26326 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:30:06.043223 26249 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:06.043689 28610 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:05.997759 28816 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:06.044661 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:06.045922 27123 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:06.046227 28816 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:06.046700 28763 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:05.996876 28234 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260924 14:30:06.007834 28805 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.050741 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:06.016031 26252 raft_consensus.cc:2362] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6. Re-sending same reply.
I20260924 14:30:06.053849 26403 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
W20260924 14:30:06.054514 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 49155.4us
I20260924 14:30:06.047142 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:06.055958 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51153.7us
I20260924 14:30:06.048723 28648 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:06.041204 28740 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:06.048705 28805 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:06.005244 28826 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:06.058354 26402 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:30:05.988498 28793 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260924 14:30:06.059495 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:06.059806 28805 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:06.008257 28807 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:06.057606 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:06.062108 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:06.062673 26256 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:06.063903 26255 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:06.064736 26298 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:06.026779 28833 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:06.064731 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.069907 26326 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:06.068250 28648 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:06.071849 28648 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.076359 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:06.077046 28794 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.079581 26401 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:06.079845 28814 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:06.082751 28807 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:06.066129 28524 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260924 14:30:06.083642 28793 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:06.088980 28808 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:06.092296 28698 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.094194 26400 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:06.094817 26401 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:06.096359 28794 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:06.098724 28809 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:06.106174 26253 raft_consensus.cc:2396] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260924 14:30:06.108002 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.108505 28833 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:06.070113 26221 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.117949 28638 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:06.118263 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:06.120622 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 73548.3us
I20260924 14:30:06.123032 27192 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:06.117539 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:06.127094 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:06.130820 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:06.133222 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:06.136072 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 71412.6us
I20260924 14:30:06.148650 28638 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:06.145221 28524 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:06.150005 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.150861 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:06.125957 28808 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:06.154237 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:06.154491 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 84449.7us
I20260924 14:30:06.155164 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.126354 28794 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:06.128314 28793 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:06.130287 28797 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:06.144423 28740 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:06.163527 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:06.165500 26402 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:06.164482 28234 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:06.153399 28578 consensus_queue.cc:1046] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.019s
I20260924 14:30:06.172349 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:06.172855 28578 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:06.173467 28642 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:06.174154 28578 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:06.157927 28830 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:06.156348 28760 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260924 14:30:06.154143 28796 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260924 14:30:06.175282 26398 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:06.186664 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:06.181066 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:06.130215 28613 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260924 14:30:06.191391 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:06.208562 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49970.8us
I20260924 14:30:06.162940 28640 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:06.218434 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:06.151051 28809 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260924 14:30:06.174201 28729 raft_consensus.cc:3032] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:06.161880 28524 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:06.226332 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:06.242530 26400 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:06.161557 28814 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:06.229020 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:06.223568 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:06.232383 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.192463 28698 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:06.234635 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.234947 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:06.177978 28740 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:06.236929 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:06.267088 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:06.238541 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:06.268061 26331 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:06.238936 26257 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260924 14:30:06.269142 26257 pending_rounds.cc:76] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 2
I20260924 14:30:06.269439 26257 pending_rounds.cc:97] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260924 14:30:06.269734 26257 raft_consensus.cc:2866] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:06.229727 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:06.243705 28838 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:06.188647 28578 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:06.246416 26326 raft_consensus.cc:2379] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:06.233352 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:06.282269 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 58801.7us
I20260924 14:30:06.283557 26077 catalog_manager.cc:5495] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 4 to 7, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 7 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:30:06.285326 26077 catalog_manager.cc:5495] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:06.277980 28610 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.287950 28610 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.290167 28610 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:06.290829 28698 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:06.246632 28648 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.291505 28698 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:06.244488 28794 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.235889 26256 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260924 14:30:06.210412 28595 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:06.237354 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.246866 28636 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.265971 26255 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:06.267091 26325 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260924 14:30:06.266224 26079 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 138) took 16273 ms (client timeout 15000 ms). Trace:
I20260924 14:30:06.209395 28797 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:06.240598 28729 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:06.274746 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:06.222774 28805 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:06.357344 28805 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:06.357777 28805 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:06.287045 26077 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 6, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:06.279019 26258 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:06.164151 28808 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:06.300657 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:06.291831 28648 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:06.279381 28838 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:06.334654 28581 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.222699 28529 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260924 14:30:06.263588 28789 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:06.335039 26224 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:06.292450 28799 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:06.276942 28740 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:06.356021 28729 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.266594 28793 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260924 14:30:06.304872 28754 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.305656 28595 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:06.369884 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:06.372396 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 109033us
I20260924 14:30:06.390905 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.394452 27185 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:06.366585 26077 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 3 to 4, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:06.327566 26326 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:06.223407 28816 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
W20260924 14:30:06.349274 26079 rpcz_store.cc:269] 0924 14:29:49.972916 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:29:49.976375 (+  3459us) service_pool.cc:227] Handling call
0924 14:30:06.246131 (+16269756us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:30:05.302502 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:30:05.374118 (+ 71616us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:30:05.374183 (+    65us) write_op.cc:435] Acquired schema lock
0924 14:30:05.374197 (+    14us) tablet.cc:660] Decoding operations
0924 14:30:05.396047 (+ 21850us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:30:05.396181 (+   134us) write_op.cc:641] Partition lock acquired for write op
0924 14:30:05.396208 (+    27us) tablet.cc:683] Acquiring locks for 19 operations
0924 14:30:05.397796 (+  1588us) tablet.cc:699] Row locks acquired
0924 14:30:05.397812 (+    16us) write_op.cc:260] PREPARE: finished
0924 14:30:05.397992 (+   180us) write_op.cc:270] Start()
0924 14:30:05.398130 (+   138us) write_op.cc:276] Timestamp: P: 1790260205397921 usec, L: 0
0924 14:30:05.398161 (+    31us) op_driver.cc:331] REPLICATION: starting
0924 14:30:05.398904 (+   743us) log.cc:842] Serialized 7582 byte log entry
0924 14:30:05.833094 (+434190us) op_driver.cc:444] REPLICATION: finished
0924 14:30:06.067949 (+234855us) write_op.cc:301] APPLY: starting
0924 14:30:06.068050 (+   101us) tablet.cc:1365] starting BulkCheckPresence
0924 14:30:06.071027 (+  2977us) tablet.cc:1368] finished BulkCheckPresence
0924 14:30:06.071160 (+   133us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:30:06.073163 (+  2003us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:30:06.073477 (+   314us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0924 14:30:06.073520 (+    43us) write_op.cc:312] APPLY: finished
0924 14:30:06.081680 (+  8160us) log.cc:842] Serialized 131 byte log entry
0924 14:30:06.083305 (+  1625us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:06.087749 (+  4444us) write_op.cc:454] Released schema lock
0924 14:30:06.087813 (+    64us) write_op.cc:341] FINISH: Updating metrics
0924 14:30:06.091794 (+  3981us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:06.091829 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1211,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":234532,"apply.run_cpu_time_us":5111,"apply.run_wall_time_us":24349,"num_ops":19,"prepare.queue_time_us":156832,"prepare.run_cpu_time_us":30801,"prepare.run_wall_time_us":322500,"raft.queue_time_us":157578,"raft.run_cpu_time_us":22306,"raft.run_wall_time_us":153400,"replication_time_us":434827,"thread_start_us":506020,"threads_started":4,"wal-append.queue_time_us":213143}]]}
I20260924 14:30:06.305958 28814 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260924 14:30:06.382403 27192 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:06.353574 28797 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:06.396503 26326 raft_consensus.cc:595] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:06.327770 28636 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:06.429579 28797 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:06.431159 28636 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:06.431913 28636 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:06.432273 28636 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:06.432610 28636 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:06.363282 28234 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:06.334399 28794 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.398124 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:06.398638 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:06.457827 27191 raft_consensus.cc:2354] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5.
I20260924 14:30:06.389330 28841 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:06.382982 28838 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:06.388283 28799 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:06.429620 26326 raft_consensus.cc:603] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:06.439203 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:06.459751 26077 catalog_manager.cc:5495] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 2 to 3, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 3 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
W20260924 14:30:06.461213 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 186538us
I20260924 14:30:06.359843 28698 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:06.477458 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:06.465840 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 93602.2us
I20260924 14:30:06.485042 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.385778 28581 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.461938 27185 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:06.364283 28524 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:06.456650 28729 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:06.433506 28595 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:06.478439 28799 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:06.462962 28610 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:30:06.426326 28754 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:06.417457 28578 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:06.528756 26077 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 4 to 5, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:30:06.537066 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 236492us
W20260924 14:30:06.505699 28848 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260924 14:30:06.520558 28849 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260924 14:30:06.511358 28850 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260924 14:30:06.555902 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:06.508472 28816 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:06.526562 28640 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:06.560575 26401 raft_consensus.cc:2396] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260924 14:30:06.515712 28794 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:06.562896 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 76212.5us
W20260924 14:30:06.576472 28846 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260924 14:30:06.576076 26077 catalog_manager.cc:5495] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 3 to 4, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 4 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:30:06.578071 26326 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:06.578784 28754 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:06.549077 28830 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.578094 26077 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 2 to 4, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 4 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:06.581890 26400 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:30:06.581496 26077 catalog_manager.cc:5495] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:30:06.582628 28529 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:06.579886 28830 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:06.594772 28613 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:06.606447 26325 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:06.617065 26331 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:06.619294 26330 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:30:06.623852 26328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:06.610759 28754 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.628067 28754 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:06.628449 28814 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:06.628654 28797 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:06.629129 28814 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:06.561232 28816 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:06.630339 28816 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:06.630887 28797 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:06.632429 26327 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:06.636417 26328 raft_consensus.cc:595] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:06.604681 26077 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 6, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:06.637064 26328 raft_consensus.cc:603] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:06.468964 28808 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.642663 26077 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 9, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 9 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
W20260924 14:30:06.652580 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 137265us
I20260924 14:30:06.630295 28754 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:06.663609 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:06.666770 26323 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:06.665652 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:06.667608 26331 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:06.665800 26327 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:06.668583 26077 catalog_manager.cc:5495] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 3 to 4, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 4 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:06.667526 28830 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:06.674829 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:06.675516 26323 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:06.665273 28728 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260924 14:30:06.685046 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:06.686641 26077 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 3 to 4, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:06.691545 28760 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.692332 28760 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:06.695019 28760 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:06.700330 28855 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.110s
I20260924 14:30:06.590492 28807 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260924 14:30:06.663267 28638 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:06.590736 28826 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260924 14:30:06.707751 28581 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:06.667232 28648 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:06.663301 28591 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
W20260924 14:30:06.729252 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 64706.5us
I20260924 14:30:06.729163 26077 catalog_manager.cc:5495] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:06.716742 26324 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:06.735487 26324 raft_consensus.cc:595] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:06.683508 28853 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260924 14:30:06.777217 28724 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:06.777771 28724 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:06.779743 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49728.9us
I20260924 14:30:06.760496 28805 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:06.729718 28592 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.767642 28799 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:06.768805 26077 catalog_manager.cc:5495] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 6, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 6 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:06.762449 28728 raft_consensus.cc:3032] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:06.801046 28728 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:06.802174 28728 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:06.804919 28799 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:06.807658 28728 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:06.816785 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 339370us
I20260924 14:30:06.820305 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:06.822005 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:06.826272 28636 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:06.827627 26256 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260924 14:30:06.834957 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 500024us
I20260924 14:30:06.698026 28816 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:06.847590 26331 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:06.851970 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:06.855657 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 270815us
I20260924 14:30:06.868999 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:06.869982 26401 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:06.892948 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:06.894632 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:06.901305 26331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:06.831584 28807 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:06.902140 26331 raft_consensus.cc:595] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
W20260924 14:30:06.904603 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 68497.7us
I20260924 14:30:06.905370 26331 raft_consensus.cc:603] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260924 14:30:06.906428 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 54535.5us
I20260924 14:30:06.908843 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.909216 28529 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:06.911582 26251 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:06.911851 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:06.911590 26403 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:06.803022 28814 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:06.921005 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.916383 28857 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:06.917273 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:06.848389 28799 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:06.917665 26256 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:06.924559 27185 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:06.918797 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:06.924754 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.925372 27188 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:06.919627 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.919898 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.927238 26257 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:06.927312 26255 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260924 14:30:06.836447 28841 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:06.912515 26401 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 2.
I20260924 14:30:06.920171 26404 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:30:06.910902 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:06.928177 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 227298us
I20260924 14:30:06.926868 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:06.937569 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:06.937752 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.819973 28592 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:06.945142 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.901335 28724 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:06.902818 28648 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.961858 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:06.903816 28686 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.950462 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:06.951762 26323 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:06.912942 28760 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.959215 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:06.961475 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.963953 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:06.987473 26249 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:06.964949 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.966248 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:06.993595 26257 raft_consensus.cc:2354] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7.
I20260924 14:30:06.987468 26401 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:06.966687 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:06.995754 26258 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:06.923732 28610 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.987250 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:06.996795 26254 raft_consensus.cc:2354] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8.
I20260924 14:30:07.002315 26249 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:07.004055 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:07.039701 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 146554us
I20260924 14:30:06.920518 28830 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260924 14:30:06.967368 28595 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:07.075856 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:06.936177 28854 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.123s
I20260924 14:30:07.006047 28789 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:06.952641 28754 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260924 14:30:07.005160 26323 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:07.098619 26255 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:06.989662 28648 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:07.003893 28638 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
I20260924 14:30:07.006229 28698 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260924 14:30:07.086297 28797 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:06.999732 28838 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:06.999316 28760 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:07.001487 28610 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:07.110667 28847 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260924 14:30:07.114346 27186 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:07.129789 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:07.218679 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 97661.2us
I20260924 14:30:07.053104 28581 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260924 14:30:07.263247 26323 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:30:07.121066 28789 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:07.003556 28686 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:07.254902 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 328085us
W20260924 14:30:07.226374 28740 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.828s	user 0.005s	sys 0.011s
I20260924 14:30:07.005843 28856 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.125s
I20260924 14:30:07.286053 26251 raft_consensus.cc:2354] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5.
I20260924 14:30:07.127215 28867 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:07.220839 28805 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:07.282714 26331 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
W20260924 14:30:07.290673 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 214892us
I20260924 14:30:07.298274 26369 leader_election.cc:398] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5.
I20260924 14:30:07.298735 26369 leader_election.cc:401] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260924 14:30:07.302245 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:07.302964 26323 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:07.303709 26323 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:07.304311 26323 raft_consensus.cc:595] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:07.304668 26323 raft_consensus.cc:603] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260924 14:30:07.297705 27124 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.009s	sys 0.016s
W20260924 14:30:07.308350 27124 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.010s	sys 0.017s
W20260924 14:30:07.310559 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 385863us
I20260924 14:30:07.256713 28865 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:07.318621 26295 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:07.285101 28636 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:07.291744 28724 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:07.350049 26323 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:07.102699 28841 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:07.238420 28797 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:07.299734 28816 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:07.378371 27126 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:07.387771 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 69262.8us
I20260924 14:30:07.326853 28865 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:07.288029 28740 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:07.390558 28592 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:07.246183 28864 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.156s
I20260924 14:30:07.459465 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:30:07.577426 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:30:07.578506 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260924 14:30:07.456560 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 78306.8us
I20260924 14:30:07.463501 26257 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:07.402006 28830 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:07.520848 28870 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:07.647040 28816 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:07.571497 28878 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260924 14:30:07.573508 28876 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260924 14:30:07.557832 28883 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:07.519583 28805 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:07.648478 28610 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:07.463775 28865 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:07.459547 28795 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:07.746559 28760 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:07.520108 28873 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
W20260924 14:30:07.747759 28872 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260924 14:30:07.492132 28875 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:07.730258 28648 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:07.459904 28854 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:07.711077 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 85741.5us
W20260924 14:30:07.704070 26332 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.002s	sys 0.006s
W20260924 14:30:07.760684 26332 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.003s	sys 0.006s
I20260924 14:30:07.707566 28807 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:07.435727 28724 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:07.778218 26077 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.000s	sys 0.011s
I20260924 14:30:07.729031 28867 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:07.647531 28591 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.108s
W20260924 14:30:07.786850 26077 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.001s	sys 0.011s
W20260924 14:30:07.796415 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 498278us
I20260924 14:30:07.806408 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:07.717726 28592 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260924 14:30:07.731407 28686 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:07.800035 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.041s	sys 0.033s
I20260924 14:30:07.742702 28885 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:07.701527 28830 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:07.820868 28867 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:07.775521 28838 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:07.749459 28875 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:07.828369 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 382) took 1258 ms. Trace:
I20260924 14:30:07.833508 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 379) took 1254 ms. Trace:
I20260924 14:30:07.721959 28816 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:07.833103 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 387) took 1056 ms. Trace:
I20260924 14:30:07.792964 28640 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 14:30:07.829123 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.042s	sys 0.033s
I20260924 14:30:07.755082 28805 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:07.860483 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 386) took 1084 ms. Trace:
I20260924 14:30:07.876307 26324 rpcz_store.cc:276] 0924 14:30:06.776659 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.779695 (+  3036us) service_pool.cc:227] Handling call
0924 14:30:06.780141 (+   446us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:06.781003 (+   862us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:06.781032 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:06.781491 (+   459us) log.cc:842] Serialized 26 byte log entry
0924 14:30:07.138368 (+356877us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:07.300818 (+162450us) log.cc:842] Serialized 14 byte log entry
0924 14:30:07.788225 (+487407us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:07.788307 (+    82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:07.856357 (+ 68050us) raft_consensus.cc:1671] finished
0924 14:30:07.856397 (+    40us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:07.860394 (+  3997us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":29376,"prepare.queue_time_us":617057,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":379178,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":32284,"thread_start_us":852010,"threads_started":3,"wal-append.queue_time_us":505971}
I20260924 14:30:07.877790 28805 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:07.828270 28636 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:07.878501 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 597479us
I20260924 14:30:07.878742 28636 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:07.879595 28610 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:07.879803 26298 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:07.880684 28724 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:07.881012 28648 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:30:07.852694 26330 rpcz_store.cc:276] 0924 14:30:06.569769 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.640231 (+ 70462us) service_pool.cc:227] Handling call
0924 14:30:06.657248 (+ 17017us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:06.668019 (+ 10771us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:06.668050 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:06.668504 (+   454us) log.cc:842] Serialized 26 byte log entry
0924 14:30:06.933598 (+265094us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:06.933636 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:06.933715 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:07.828048 (+894333us) raft_consensus.cc:1671] finished
0924 14:30:07.828079 (+    31us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:07.828281 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9844,"raft.queue_time_us":12128,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":13568,"thread_start_us":226330,"threads_started":1,"wal-append.queue_time_us":571838}
I20260924 14:30:07.881434 28648 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:07.881822 28648 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:07.885376 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:07.886190 26330 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:07.886577 26323 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:07.887235 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:07.782315 28807 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:07.888741 26330 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:07.857182 28698 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.014s	sys 0.010s
I20260924 14:30:07.777398 28854 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:07.852756 26332 rpcz_store.cc:276] 0924 14:30:06.568805 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.568935 (+   130us) service_pool.cc:227] Handling call
0924 14:30:06.582773 (+ 13838us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:06.589727 (+  6954us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:06.589771 (+    44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:06.595334 (+  5563us) log.cc:842] Serialized 26 byte log entry
0924 14:30:07.092875 (+497541us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:07.768422 (+675547us) log.cc:842] Serialized 14 byte log entry
0924 14:30:07.818651 (+ 50229us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:07.819293 (+   642us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:07.819838 (+   545us) raft_consensus.cc:1671] finished
0924 14:30:07.819991 (+   153us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:07.823301 (+  3310us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":54807,"prepare.queue_time_us":426074,"raft.queue_time_us":66235,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":29104,"spinlock_wait_cycles":15836928,"thread_start_us":1139853,"threads_started":2,"wal-append.queue_time_us":917202}
I20260924 14:30:07.718954 28870 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:07.920276 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.008s	sys 0.042s
W20260924 14:30:07.920862 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.008s	sys 0.042s
W20260924 14:30:07.921502 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 609014us
I20260924 14:30:07.924093 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:07.746232 28883 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:07.900645 28698 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.014s	sys 0.011s
W20260924 14:30:07.899181 28789 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.002s	sys 0.029s
W20260924 14:30:07.945956 26258 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.944s	user 0.009s	sys 0.001s
I20260924 14:30:07.881029 28636 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:07.953179 28866 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.617s	user 0.002s	sys 0.000s
W20260924 14:30:07.953496 26296 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.010s	sys 0.007s
W20260924 14:30:07.953972 26296 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.010s	sys 0.007s
I20260924 14:30:07.954505 28902 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:07.954972 28877 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260924 14:30:07.955331 28881 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260924 14:30:07.952363 26258 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:07.957620 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 381) took 1388 ms. Trace:
I20260924 14:30:07.920086 28740 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:07.964382 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1101) took 1022 ms. Trace:
I20260924 14:30:07.964931 26251 rpcz_store.cc:276] 0924 14:30:06.937102 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.937239 (+   137us) service_pool.cc:227] Handling call
0924 14:30:07.959508 (+1022269us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b540002b554. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:07.959680 (+   172us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":798949248}
I20260924 14:30:07.927419 28591 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:07.967527 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1102) took 1020 ms. Trace:
I20260924 14:30:07.842211 28885 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:07.942416 28789 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.002s	sys 0.029s
I20260924 14:30:07.927501 28797 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:07.970705 26258 rpcz_store.cc:276] 0924 14:30:06.939195 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.939356 (+   161us) service_pool.cc:227] Handling call
0924 14:30:07.959643 (+1020287us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:07.960368 26326 rpcz_store.cc:276] 0924 14:30:06.569447 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.578865 (+  9418us) service_pool.cc:227] Handling call
0924 14:30:06.579196 (+   331us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:06.584708 (+  5512us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:06.584756 (+    48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:06.585205 (+   449us) log.cc:842] Serialized 26 byte log entry
0924 14:30:06.816806 (+231601us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b1800177280. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:30:07.129237 (+312431us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:07.669601 (+540364us) log.cc:842] Serialized 14 byte log entry
0924 14:30:07.713956 (+ 44355us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:07.714041 (+    85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:07.957229 (+243188us) raft_consensus.cc:1671] finished
0924 14:30:07.957378 (+   149us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:07.957553 (+   175us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":120923,"prepare.queue_time_us":573127,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":96350,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":18023,"spinlock_wait_cycles":4073216,"thread_start_us":671424,"threads_started":2,"wal-append.queue_time_us":946148}
W20260924 14:30:07.975936 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 169661us
I20260924 14:30:07.992339 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:07.963672 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 152035us
I20260924 14:30:07.950609 28610 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:07.996220 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:07.999766 26330 raft_consensus.cc:2396] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:07.932457 28686 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260924 14:30:08.014168 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.042656 27185 raft_consensus.cc:2396] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260924 14:30:07.921434 28884 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.125s
I20260924 14:30:07.955106 28902 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:08.058231 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.074569 27192 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:08.099622 27194 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:08.084702 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:08.053272 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.101384 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.028384 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:08.015795 27188 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:30:08.082103 26257 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:08.110970 27181 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
W20260924 14:30:08.080566 28889 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260924 14:30:08.009802 28740 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.128691 28885 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:08.129024 28883 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:08.129281 28740 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:08.131178 28903 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.132032 28897 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:08.131759 28903 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:08.132680 28897 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.133376 28914 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.076933 28816 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:07.985102 28591 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:08.133992 28903 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:08.133922 28914 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:08.134640 28903 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:08.135017 28903 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:08.135334 28903 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:08.135807 28591 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:08.137857 28868 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:08.138103 28911 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:08.138447 28868 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:08.138653 28911 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:08.139611 28898 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.139859 28926 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:08.140213 28926 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:08.140601 28926 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:08.140154 28898 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:07.985611 28789 raft_consensus.cc:686] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:08.125237 28902 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:08.126889 28913 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260924 14:30:07.960326 28867 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:08.127277 28686 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.076370 28807 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:08.133102 28897 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:08.119745 28610 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.012666 28865 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:08.142228 28648 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:08.142359 28898 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:08.245633 28882 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
I20260924 14:30:08.198735 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.144343 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 380) took 1575 ms. Trace:
I20260924 14:30:08.247572 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 385) took 1525 ms. Trace:
W20260924 14:30:08.249480 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 143397us
I20260924 14:30:08.220570 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.257741 27181 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:08.079567 28896 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5.
W20260924 14:30:08.127315 28888 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
I20260924 14:30:08.265024 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:08.247646 28889 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260924 14:30:08.113245 28899 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.013676 28698 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.076205 28854 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:08.082927 28861 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:08.013154 28797 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.254671 26327 rpcz_store.cc:276] 0924 14:30:06.722050 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.723096 (+  1046us) service_pool.cc:227] Handling call
0924 14:30:06.746417 (+ 23321us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:06.747133 (+   716us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:06.747179 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:06.790843 (+ 43664us) log.cc:842] Serialized 26 byte log entry
0924 14:30:07.245896 (+455053us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:07.728659 (+482763us) log.cc:842] Serialized 14 byte log entry
0924 14:30:07.728911 (+   252us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:07.728992 (+    81us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:08.247286 (+518294us) raft_consensus.cc:1671] finished
0924 14:30:08.247308 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:08.247502 (+   194us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":89679,"prepare.queue_time_us":606166,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":42506,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":94,"spinlock_wait_cycles":13997952,"thread_start_us":659262,"threads_started":2,"wal-append.queue_time_us":1013161}
I20260924 14:30:08.552244 26324 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260924 14:30:08.250974 26329 rpcz_store.cc:276] 0924 14:30:06.569132 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.569259 (+   127us) service_pool.cc:227] Handling call
0924 14:30:06.583310 (+ 14051us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:06.883497 (+300187us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:06.883563 (+    66us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:06.933160 (+ 49597us) log.cc:842] Serialized 26 byte log entry
0924 14:30:07.575169 (+642009us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:07.862173 (+287004us) log.cc:842] Serialized 14 byte log entry
0924 14:30:07.862428 (+   255us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:07.862515 (+    87us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:08.144080 (+281565us) raft_consensus.cc:1671] finished
0924 14:30:08.144106 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:08.144271 (+   165us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4477,"prepare.queue_time_us":313027,"raft.queue_time_us":27060,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":33285,"spinlock_wait_cycles":10026240,"thread_start_us":855901,"threads_started":3,"wal-append.queue_time_us":1024539}
I20260924 14:30:08.270869 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:08.713610 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:08.671128 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 668) took 1762 ms. Trace:
I20260924 14:30:08.812558 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.881937 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.268989 26323 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:08.836804 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.532370 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.808125 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:08.628736 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 360455us
W20260924 14:30:08.940642 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 132607us
I20260924 14:30:08.941483 26295 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:08.918119 26369 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.000s	sys 0.009s
W20260924 14:30:08.945252 26369 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.000s	sys 0.010s
I20260924 14:30:08.947001 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:08.950989 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:08.945565 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 917235us
I20260924 14:30:08.952059 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:08.932149 26253 rpcz_store.cc:276] 0924 14:30:06.908975 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.912155 (+  3180us) service_pool.cc:227] Handling call
0924 14:30:07.450208 (+538053us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eebce 000000000034aea9 00007f09be21aa30 00007f09be2e07b4 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:30:07.450453 (+   245us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:07.510070 (+ 59617us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:07.510168 (+    98us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:07.526736 (+ 16568us) log.cc:842] Serialized 26 byte log entry
0924 14:30:07.842022 (+315286us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:07.842060 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:07.842128 (+    68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:08.670701 (+828573us) raft_consensus.cc:1671] finished
0924 14:30:08.670735 (+    34us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:08.671046 (+   311us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25873,"raft.queue_time_us":67399,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":2996096,"thread_start_us":150167,"threads_started":1,"wal-append.queue_time_us":374587}
I20260924 14:30:08.945616 26295 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:08.856192 28869 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.535375 28807 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.955616 28807 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:08.261265 28686 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:08.957697 28807 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:08.958226 28807 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.958532 28686 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:08.501008 28816 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.944154 28728 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.959614 28797 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:08.960472 28728 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:08.961153 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:08.961836 26327 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:08.963055 26401 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:08.583346 28883 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.959820 28816 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:08.963689 28883 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:08.963989 28928 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:08.964643 28928 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:08.965093 28928 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:08.965329 28816 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:08.965875 28883 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:08.965960 28816 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.855726 28895 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.217831 28903 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:08.972054 27192 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:08.973719 28903 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:08.974619 28903 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.939953 28932 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.975739 28816 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:08.976166 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:08.976253 28932 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:08.977916 28816 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:08.978482 28816 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:08.978504 28932 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:08.268340 28902 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.745733 28754 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:08.980504 28754 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:08.939253 28861 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:08.981343 28754 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:08.982436 28754 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:08.984158 28915 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:08.944411 28885 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:08.951767 28937 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.749449 28638 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.987370 28797 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:08.987030 28937 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:08.988025 28883 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:08.987779 28638 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:08.988442 28883 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:08.988787 28883 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:08.989171 27188 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:08.989517 28937 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:08.990242 28902 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:08.991807 28686 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:30:08.975142 28903 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:08.995196 28903 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:08.958683 28807 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:08.259224 28875 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:08.890362 28936 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.968411 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:08.955327 28869 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:08.744485 28890 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.615s	user 0.000s	sys 0.001s
I20260924 14:30:08.966833 28895 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:08.854753 28931 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.263824 28830 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:08.979085 28816 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:08.671398 28919 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.432s	user 0.001s	sys 0.000s
I20260924 14:30:08.209952 28870 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:08.903414 28868 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:08.652639 28740 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:08.609056 28896 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:08.756122 28865 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:08.903051 28595 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.828s	user 0.016s	sys 0.000s
I20260924 14:30:08.264335 28610 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:08.998931 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 305248us
W20260924 14:30:09.139832 27186 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 385) took 2750 ms (client timeout 2177 ms). Trace:
I20260924 14:30:09.137422 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 593) took 2093 ms. Trace:
W20260924 14:30:09.220623 27186 rpcz_store.cc:269] 0924 14:30:06.389331 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.393834 (+  4503us) service_pool.cc:227] Handling call
0924 14:30:09.139584 (+2745750us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b54003269d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:09.139704 (+   120us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1129117568}
I20260924 14:30:08.676347 28789 consensus_queue.cc:237] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.102129 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:09.221961 26297 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.062s (SENT)
I20260924 14:30:09.000643 27188 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:09.103950 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.223501 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:09.222276 26397 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:09.105540 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.145072 27192 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:09.220849 26399 rpcz_store.cc:276] 0924 14:30:07.044405 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:07.044562 (+   157us) service_pool.cc:227] Handling call
0924 14:30:07.091641 (+ 47079us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:07.110638 (+ 18997us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:07.110671 (+    33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:07.110712 (+    41us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:07.110743 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:09.137079 (+2026336us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b540023fd54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:09.137137 (+    58us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:09.137349 (+   212us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":43202816}
I20260924 14:30:08.550468 28833 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.560490 28898 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:09.228806 27123 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:09.245442 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:08.649775 28914 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:08.746776 28805 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:08.755127 28864 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:08.675601 28918 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:09.249725 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 281375us
I20260924 14:30:09.252473 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:09.262344 27189 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:09.253181 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:09.253975 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:09.001513 28728 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:09.264756 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.265218 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:09.265473 27190 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:09.265978 26327 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:30:09.204507 28889 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:30:09.219871 28807 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:09.145008 28902 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:09.293399 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.245924 28754 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:09.252280 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:09.139261 28926 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.998s	user 0.007s	sys 0.005s
I20260924 14:30:09.127672 28941 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:09.267230 27191 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:09.280808 27192 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:08.983274 28861 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.121210 28940 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:08.995097 28638 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.188169 28648 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:09.076844 28897 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:09.230244 28867 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.004s	sys 0.012s
I20260924 14:30:09.247357 28933 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:08.946211 28698 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:09.297030 26397 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:09.297955 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:09.299327 26249 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:09.256196 28870 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:09.255846 28830 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:09.299391 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.301751 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.304677 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.261083 28865 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260924 14:30:09.305569 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.005s	sys 0.020s
I20260924 14:30:09.307530 27190 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:30:09.279170 28918 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.250313 28936 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:09.279423 28805 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:09.313011 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.281021 28864 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:09.246829 28896 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:08.945909 28899 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:09.294298 26221 connection.cc:232] RPC call timeout handler was delayed by -1.2868989509997846s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
W20260924 14:30:09.289628 28856 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.964s	user 0.008s	sys 0.033s
I20260924 14:30:09.282338 28852 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:09.320125 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.293148 28807 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:09.294044 28868 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:09.299474 28901 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:09.259202 28833 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.300045 28529 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:09.260056 28898 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:09.245368 28931 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:08.810214 28911 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.328342 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:09.367204 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 145260us
W20260924 14:30:09.449069 26297 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.112s (SENT)
I20260924 14:30:09.390826 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1109) took 2424 ms. Trace:
I20260924 14:30:09.444931 26327 raft_consensus.cc:2396] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260924 14:30:09.411654 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 658) took 1526 ms. Trace:
I20260924 14:30:09.385926 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 966) took 2493 ms. Trace:
I20260924 14:30:09.455026 28868 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:09.455588 26398 rpcz_store.cc:276] 0924 14:30:06.892095 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.892227 (+   132us) service_pool.cc:227] Handling call
0924 14:30:06.911774 (+ 19547us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:06.912960 (+  1186us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:06.913002 (+    42us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:08.711059 (+1798057us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b2c0013ed68. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:08.723273 (+ 12214us) log.cc:842] Serialized 26 byte log entry
0924 14:30:09.246480 (+523207us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:09.246516 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:09.246589 (+    73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:09.385470 (+138881us) raft_consensus.cc:1671] finished
0924 14:30:09.385500 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:09.385839 (+   339us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":5439,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":3091712,"thread_start_us":406539,"threads_started":1,"wal-append.queue_time_us":626376}
I20260924 14:30:09.457369 28868 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.458020 28529 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:09.458508 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:09.460002 28529 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:09.453012 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.460711 28865 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:09.461110 28865 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:09.461166 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.461807 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 452) took 1202 ms. Trace:
I20260924 14:30:09.462342 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.453594 28807 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:09.385300 28928 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:09.388761 28816 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:09.462186 27191 rpcz_store.cc:276] 0924 14:30:08.259815 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:08.261482 (+  1667us) service_pool.cc:227] Handling call
0924 14:30:09.461627 (+1200145us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54003b79d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:09.461725 (+    98us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":760796032}
I20260924 14:30:09.470507 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:09.368500 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 383) took 2760 ms. Trace:
I20260924 14:30:09.472568 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:09.474404 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.475435 26249 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:09.464282 26257 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:09.482558 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.450639 26324 rpcz_store.cc:276] 0924 14:30:07.885277 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:07.887698 (+  2421us) service_pool.cc:227] Handling call
0924 14:30:07.888060 (+   362us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:08.939573 (+1051513us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:09.411254 (+471681us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b54007d46d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:09.411571 (+   317us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1490632960}
I20260924 14:30:09.401053 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.464373 26406 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:09.451608 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.464978 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:09.469089 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:09.465106 27189 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:09.484968 26252 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:09.463950 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.005s	sys 0.024s
I20260924 14:30:09.477608 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.485685 26401 raft_consensus.cc:2396] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260924 14:30:09.485867 26224 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 1.698s (SENT) [suppressed 13 similar messages]
I20260924 14:30:09.486363 26256 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:09.450443 28856 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.133s	user 0.008s	sys 0.036s
I20260924 14:30:09.471617 26325 rpcz_store.cc:276] 0924 14:30:06.570115 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.638202 (+ 68087us) service_pool.cc:227] Handling call
0924 14:30:06.650520 (+ 12318us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:06.657625 (+  7105us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:30:07.044278 (+386653us) log.cc:842] Serialized 14 byte log entry
0924 14:30:07.308327 (+264049us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:07.329247 (+ 20920us) log.cc:842] Serialized 26 byte log entry
0924 14:30:07.329474 (+   227us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:07.518467 (+188993us) log.cc:842] Serialized 14 byte log entry
0924 14:30:07.518666 (+   199us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:07.518758 (+    92us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:09.329953 (+1811195us) raft_consensus.cc:1671] finished
0924 14:30:09.329972 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:09.330122 (+   150us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":203939,"prepare.queue_time_us":1394870,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":461436,"raft.run_cpu_time_us":405,"raft.run_wall_time_us":15722,"thread_start_us":556003,"threads_started":2,"wal-append.queue_time_us":806843}
I20260924 14:30:09.324949 28885 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:09.449200 26253 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:09.384315 28610 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.312629 28754 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:09.320657 28941 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:09.324194 28940 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.312217 28926 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.594194 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.364948 28814 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:09.530342 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.449621 26250 rpcz_store.cc:276] 0924 14:30:06.941527 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:07.001640 (+ 60113us) service_pool.cc:227] Handling call
0924 14:30:07.063698 (+ 62058us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:07.067279 (+  3581us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:07.067324 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:07.067913 (+   589us) log.cc:842] Serialized 26 byte log entry
0924 14:30:07.271839 (+203926us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:07.333516 (+ 61677us) log.cc:842] Serialized 14 byte log entry
0924 14:30:07.333853 (+   337us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:07.334323 (+   470us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:09.365524 (+2031201us) raft_consensus.cc:1671] finished
0924 14:30:09.365556 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:09.365743 (+   187us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3104,"prepare.queue_time_us":479327,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":4340,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":72793,"thread_start_us":207022,"threads_started":2,"wal-append.queue_time_us":855352}
I20260924 14:30:09.592577 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.514213 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.623760 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:09.485810 26400 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.008s	sys 0.003s
I20260924 14:30:09.591833 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:09.515642 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50746.5us
I20260924 14:30:07.853384 26328 rpcz_store.cc:276] 0924 14:30:06.777034 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.777292 (+   258us) service_pool.cc:227] Handling call
0924 14:30:06.777657 (+   365us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:06.778084 (+   427us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:06.778113 (+    29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:30:06.778665 (+   552us) log.cc:842] Serialized 52 byte log entry
0924 14:30:07.238808 (+460143us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:07.734657 (+495849us) log.cc:842] Serialized 14 byte log entry
0924 14:30:07.832137 (+ 97480us) log.cc:842] Serialized 14 byte log entry
0924 14:30:07.832705 (+   568us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:07.832791 (+    86us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:07.832844 (+    53us) raft_consensus.cc:1671] finished
0924 14:30:07.832873 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:07.833033 (+   160us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":131313,"prepare.queue_time_us":569403,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":86826,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":14268,"spinlock_wait_cycles":20550528,"thread_start_us":858019,"threads_started":2,"wal-append.queue_time_us":950135}
I20260924 14:30:09.516655 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.516762 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:09.472132 28867 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.005s	sys 0.026s
W20260924 14:30:09.547732 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 98694.4us
I20260924 14:30:09.486724 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.463217 28852 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:09.629797 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:09.638975 26297 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.197s (SENT)
I20260924 14:30:09.631079 26256 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:09.639657 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:09.640544 26257 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260924 14:30:09.641680 26400 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.010s	sys 0.008s
I20260924 14:30:09.642012 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:09.642411 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 596) took 1883 ms. Trace:
I20260924 14:30:09.643811 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:09.544158 28686 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:09.592037 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:09.764063 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 172103us
I20260924 14:30:09.367156 28648 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:09.764977 28648 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:09.487098 26329 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:09.510283 27194 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:09.482093 26403 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:09.772161 28795 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:09.772745 28795 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.772948 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.410458 28808 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:09.774763 28795 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:09.775041 28808 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.777230 28808 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:09.382695 28869 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.621672 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.621086 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:09.778285 26249 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:09.625768 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:09.778811 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 157810us
I20260924 14:30:09.779330 28857 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:30:09.779506 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 153822us
I20260924 14:30:09.779976 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:09.478392 28729 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:09.780777 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:09.781038 28924 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:09.781551 28857 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:09.627061 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:09.781895 28857 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:09.782202 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 155249us
I20260924 14:30:09.782289 28857 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:09.782856 26369 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:09.379253 28895 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:09.780833 28729 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:09.784035 26369 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:09.627723 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:09.785768 28729 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.786113 28938 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:09.786629 28938 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:09.787786 28869 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:09.787467 28938 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.788450 28869 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:09.643123 26400 rpcz_store.cc:276] 0924 14:30:07.758599 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:07.758780 (+   181us) service_pool.cc:227] Handling call
0924 14:30:07.792872 (+ 34092us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:07.793132 (+   260us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:30:08.175235 (+382103us) log.cc:842] Serialized 14 byte log entry
0924 14:30:09.642078 (+1466843us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:09.642129 (+    51us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:09.642174 (+    45us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:09.642248 (+    74us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:09.642351 (+   103us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":876525,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":297429,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":1781473,"threads_started":3,"wal-append.queue_time_us":556447}
I20260924 14:30:09.648455 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:09.649171 28805 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:09.619388 28816 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:09.799368 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.800861 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:09.468899 28901 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:09.801165 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.801946 26255 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:09.802001 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.802655 26400 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:09.650256 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.620384 28920 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:09.640511 28852 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:09.659054 28760 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:09.637988 26406 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:09.660141 28902 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:09.660535 26254 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:09.664688 28899 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:09.666778 28898 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.668373 28928 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.824172 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.590240 28931 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:09.790694 28648 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.672345 28940 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:09.674070 28918 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:09.686630 28941 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:09.691954 26331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1005) took 2801 ms (client timeout 1691 ms). Trace:
W20260924 14:30:09.833431 26331 rpcz_store.cc:269] 0924 14:30:06.890335 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:06.910727 (+ 20392us) service_pool.cc:227] Handling call
0924 14:30:09.691753 (+2781026us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400237654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:09.691875 (+   122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":661207936}
I20260924 14:30:09.691164 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:09.834506 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 348639us
I20260924 14:30:09.694024 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 388) took 1194 ms. Trace:
I20260924 14:30:09.696146 28581 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:09.702430 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.713873 28610 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:09.716063 28886 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:09.728899 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 389) took 1228 ms. Trace:
I20260924 14:30:09.835405 26332 rpcz_store.cc:276] 0924 14:30:08.500064 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:08.560302 (+ 60238us) service_pool.cc:227] Handling call
0924 14:30:08.643485 (+ 83183us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:08.991714 (+348229us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:08.991754 (+    40us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:08.991809 (+    55us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:08.991854 (+    45us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:09.693771 (+701917us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540001ead4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:09.693826 (+    55us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:09.693962 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":800291072}
I20260924 14:30:09.836436 26330 rpcz_store.cc:276] 0924 14:30:08.500460 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:08.503537 (+  3077us) service_pool.cc:227] Handling call
0924 14:30:08.503915 (+   378us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:08.968074 (+464159us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:08.968123 (+    49us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:08.968185 (+    62us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:08.968229 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:09.728649 (+760420us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b5400498254. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:09.728695 (+    46us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:09.728818 (+   123us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1066904960}
I20260924 14:30:09.730741 28892 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:09.734673 28896 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:09.736493 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.649580 28754 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:09.740301 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.744364 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 660) took 1858 ms. Trace:
I20260924 14:30:09.844753 26323 rpcz_store.cc:276] 0924 14:30:07.886163 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:07.887610 (+  1447us) service_pool.cc:227] Handling call
0924 14:30:07.887960 (+   350us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:08.935193 (+1047233us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:09.744087 (+808894us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5400498c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:09.744297 (+   210us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":875614080}
I20260924 14:30:09.745658 27186 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:09.746376 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.847152 26404 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:09.747093 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.748660 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:09.849856 27190 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:09.694410 28833 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:09.751843 28864 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:09.759238 28865 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.865576 28865 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:09.730104 28936 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:09.702672 28698 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.622786 28885 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.838881 28581 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:09.838730 28886 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:09.868557 28885 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:09.868343 28581 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:09.764178 28853 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:09.868852 28886 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:09.869366 28853 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:09.869657 28581 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:09.869876 28853 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:09.870083 28886 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:09.832621 28807 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:09.873140 28857 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:09.875156 28857 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:09.790825 26328 raft_consensus.cc:2379] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:09.793059 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:09.884012 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 91009.2us
I20260924 14:30:09.813664 28760 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.468232 28870 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:09.390385 28883 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.797757 27194 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:09.537976 26326 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:09.799603 28917 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:09.657187 28640 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:30:09.803314 26253 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:09.813035 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.817864 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.816509 28920 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:09.920825 26406 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:09.922170 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.819598 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.923717 27189 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:09.820190 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 631) took 1572 ms. Trace:
I20260924 14:30:09.824324 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:09.924991 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:09.670511 28926 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:09.925540 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 101326us
I20260924 14:30:09.829196 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:09.933516 27191 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:09.468631 28933 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:09.802766 28901 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:09.812186 28794 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:30:09.814509 28838 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:09.812668 28856 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260924 14:30:09.859977 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:09.949084 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.875739 26398 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:30:09.878741 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.949995 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:09.879290 26224 connection.cc:232] RPC call timeout handler was delayed by -1.0669108209995102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2134s for a response
I20260924 14:30:09.884320 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.951689 26250 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:09.843150 28610 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:09.953276 27125 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:09.885084 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:09.956668 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 71679.8us
I20260924 14:30:09.794829 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.885360 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:09.886667 27181 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.081s	sys 0.093s
I20260924 14:30:09.887895 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:09.959668 27181 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.081s	sys 0.094s
I20260924 14:30:09.912797 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.924703 26254 rpcz_store.cc:276] 0924 14:30:08.247337 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:08.247543 (+   206us) service_pool.cc:227] Handling call
0924 14:30:09.819973 (+1572430us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54006d2ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:09.820119 (+   146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1678533376}
I20260924 14:30:09.936403 26327 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:09.902837 28807 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:09.842573 28892 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:09.934747 28648 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:09.963920 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 393) took 1447 ms. Trace:
I20260924 14:30:09.981737 26326 rpcz_store.cc:276] 0924 14:30:08.501773 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:08.946623 (+444850us) service_pool.cc:227] Handling call
0924 14:30:09.949171 (+1002548us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b540012dc54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:09.949292 (+   121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1356509952}
I20260924 14:30:09.919937 28917 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:09.966122 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.967206 27185 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:09.989816 26249 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:09.958424 26403 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:30:09.992293 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.993698 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:09.994992 26249 raft_consensus.cc:2396] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260924 14:30:09.959143 26397 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:09.972467 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.932250 28936 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:09.903429 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:09.996294 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.996609 27182 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:09.934834 28808 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:09.896026 28865 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260924 14:30:09.976190 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 453) took 1300 ms. Trace:
I20260924 14:30:09.947544 28794 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:10.001468 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.004235 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:10.005440 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:10.006564 27186 raft_consensus.cc:2396] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:10.007875 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:10.024286 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:10.025835 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.032068 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:10.038401 26406 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:10.040171 27126 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:10.040278 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.972288 28937 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:09.974980 28724 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:10.046782 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:10.048271 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.052821 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.056734 26398 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:09.974427 28816 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:10.058241 27126 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:10.001415 27181 rpcz_store.cc:276] 0924 14:30:08.675933 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:08.676076 (+   143us) service_pool.cc:227] Handling call
0924 14:30:08.696899 (+ 20823us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:08.725047 (+ 28148us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:08.725094 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:08.794375 (+ 69281us) log.cc:842] Serialized 26 byte log entry
0924 14:30:09.283673 (+489298us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:09.455821 (+172148us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b18003fdfc0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426764 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:30:09.960090 (+504269us) log.cc:842] Serialized 14 byte log entry
0924 14:30:09.960338 (+   248us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:09.960445 (+   107us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:09.960499 (+    54us) raft_consensus.cc:1671] finished
0924 14:30:09.975914 (+ 15415us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:09.976121 (+   207us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":431,"prepare.queue_time_us":145268,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":62077,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":16444,"spinlock_wait_cycles":5054336,"thread_start_us":987837,"threads_started":2,"wal-append.queue_time_us":581923}
I20260924 14:30:10.059662 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:09.981657 28648 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:10.000692 28917 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:10.066892 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:10.067199 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:09.954756 28838 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:10.068346 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:10.064143 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:10.069577 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:09.988996 28807 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:10.000922 28808 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:10.077225 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 419788us
I20260924 14:30:10.070753 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:10.071481 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:10.084888 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:10.085152 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:10.053941 28724 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:10.072932 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:10.062445 26401 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 4.
I20260924 14:30:10.074057 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.029095 28794 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:10.075184 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.075651 27194 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:10.081717 26297 leader_election.cc:398] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8.
I20260924 14:30:10.086971 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:10.093727 26297 leader_election.cc:401] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260924 14:30:10.080309 26327 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:10.087754 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.065125 28929 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:10.070482 28728 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:10.068813 27191 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
W20260924 14:30:10.090538 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 279803us
W20260924 14:30:10.071069 28947 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260924 14:30:10.095227 26403 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:10.023869 28936 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:10.066537 28857 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:10.074741 28816 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:10.117332 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:10.118683 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:10.119858 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:10.120220 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:10.120946 26257 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:10.081918 28640 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:10.070833 28648 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:10.076833 28833 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:10.081008 28808 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:10.115543 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.138630 26406 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:10.059684 28924 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260924 14:30:10.122478 26298 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:10.124590 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:10.128068 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:10.094483 28928 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:10.129699 26255 raft_consensus.cc:2379] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:10.134732 26403 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:10.098412 28724 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:10.136648 28895 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:10.215730 26328 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:10.218973 28640 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:10.223955 28895 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:10.224414 28895 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:10.216817 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.136853 28524 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:10.225706 28524 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:10.226109 28524 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:10.220029 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 74426.4us
I20260924 14:30:10.224409 27190 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:10.108347 28856 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:10.228816 28856 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:10.231779 26398 raft_consensus.cc:2396] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:10.231215 28856 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:10.232983 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48765us
I20260924 14:30:10.234125 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:10.233884 28853 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:10.234550 28524 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:10.237495 28524 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:10.240073 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:10.241078 26401 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:30:10.241894 28956 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260924 14:30:10.242341 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:10.243158 26401 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 7
I20260924 14:30:10.155157 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:10.108867 28728 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:10.245764 28728 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:10.245867 28524 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260924 14:30:10.246055 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 123658us
I20260924 14:30:10.251649 28724 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:10.254448 28724 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:10.262050 28838 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:10.275273 26398 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:10.278956 28856 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:10.279800 28856 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:10.280999 28856 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:10.283831 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 49753.3us
W20260924 14:30:10.284130 28958 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
W20260924 14:30:10.304785 26402 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.047s	sys 0.104s
W20260924 14:30:10.305363 26402 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.047s	sys 0.104s
I20260924 14:30:10.326998 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:10.313347 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:10.096220 28926 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.193s
W20260924 14:30:10.311623 26405 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.045s	sys 0.092s
W20260924 14:30:10.341081 26405 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.046s	sys 0.092s
I20260924 14:30:10.141947 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:10.342485 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.343420 26406 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:10.231122 28936 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:10.100044 28898 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8.
I20260924 14:30:10.625324 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:10.584396 26401 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:10.439254 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:10.527921 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:10.528332 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:10.629833 26331 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:10.630611 26295 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:10.630784 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:10.574083 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 663) took 1157 ms. Trace:
I20260924 14:30:10.631587 26401 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:10.439755 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:10.112154 28899 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:10.639931 28899 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:10.641983 28865 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:10.350503 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:10.642717 26326 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:10.631832 27192 rpcz_store.cc:276] 0924 14:30:09.416982 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:09.471047 (+ 54065us) service_pool.cc:227] Handling call
0924 14:30:09.514242 (+ 43195us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:09.514667 (+   425us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:09.514712 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:09.593341 (+ 78629us) log.cc:842] Serialized 26 byte log entry
0924 14:30:10.131587 (+538246us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:10.131625 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:10.131688 (+    63us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:10.573760 (+442072us) raft_consensus.cc:1671] finished
0924 14:30:10.573789 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:10.573998 (+   209us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":360,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":28202,"thread_start_us":484521,"threads_started":1,"wal-append.queue_time_us":819623}
W20260924 14:30:10.523782 28581 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.004s
I20260924 14:30:10.644757 26397 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:10.353474 28883 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:10.524082 28524 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:10.639328 28754 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:10.646098 28524 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:10.646162 28883 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election won for term 8
W20260924 14:30:10.647877 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 396772us
W20260924 14:30:10.501984 28852 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.689s	user 0.004s	sys 0.008s
I20260924 14:30:10.641728 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:10.643695 28686 consensus_meta.cc:287] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.874s	user 0.012s	sys 0.000s
I20260924 14:30:10.643889 28901 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:10.646162 28754 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:10.650966 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:10.647357 28883 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:10.353277 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 211431us
I20260924 14:30:10.711177 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:10.714510 26250 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:10.656889 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:10.658106 26323 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:10.352908 28807 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:10.654659 26326 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:10.588506 28728 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:10.568650 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 645) took 1277 ms. Trace:
I20260924 14:30:10.717420 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:10.709082 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 58191us
I20260924 14:30:10.113040 28698 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.544s
I20260924 14:30:10.724370 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:10.632949 26332 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:10.715636 26257 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:10.720762 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:10.668469 27192 raft_consensus.cc:1261] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:30:10.709120 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:10.721782 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:10.722240 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:10.722622 28885 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:10.117074 28864 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.551s
W20260924 14:30:10.731220 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 80393.1us
I20260924 14:30:10.731642 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:10.404336 28805 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:10.717840 26258 rpcz_store.cc:276] 0924 14:30:09.290988 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:09.291122 (+   134us) service_pool.cc:227] Handling call
0924 14:30:09.310271 (+ 19149us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:09.327710 (+ 17439us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:09.327748 (+    38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:09.371112 (+ 43364us) log.cc:842] Serialized 26 byte log entry
0924 14:30:09.606714 (+235602us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:09.606755 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:09.606843 (+    88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:10.568369 (+961526us) raft_consensus.cc:1671] finished
0924 14:30:10.568406 (+    37us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:10.568579 (+   173us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16925,"raft.queue_time_us":17086,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":21845,"thread_start_us":159514,"threads_started":1,"wal-append.queue_time_us":565942}
I20260924 14:30:10.729449 27194 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:10.679702 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:10.721470 26401 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:10.726675 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:10.738094 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:10.739521 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.739967 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:10.740528 26401 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:10.449601 28760 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:10.747030 28867 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:10.503525 28920 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:10.651494 28899 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:10.605036 28929 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:10.412294 28870 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:10.751101 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:10.611809 28892 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:10.524019 28853 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:10.129369 28857 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:10.755846 28234 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:10.721294 28686 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:10.596586 28610 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:10.761085 27191 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 7: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:10.656558 28838 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260924 14:30:10.773219 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:10.752415 26257 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:10.764990 26298 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:10.769162 26332 raft_consensus.cc:1261] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:30:10.770824 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.731030 28885 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:10.781528 26324 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:10.765710 26295 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:10.784191 26253 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.009s	sys 0.023s
W20260924 14:30:10.784839 26253 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.009s	sys 0.023s
I20260924 14:30:10.787535 26250 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:10.680131 28865 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260924 14:30:10.747864 28805 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:10.793157 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.756455 28859 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:10.789443 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:10.764787 28896 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.017s	sys 0.034s
I20260924 14:30:10.719305 28901 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:10.822506 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 56912.4us
I20260924 14:30:10.438804 28816 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:10.839634 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 67169.4us
I20260924 14:30:10.783512 28885 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:10.789351 28928 raft_consensus.cc:2741] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260924 14:30:10.764358 28892 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:10.831202 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:10.831406 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:10.836381 27182 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.836s	user 0.011s	sys 0.000s
W20260924 14:30:10.779811 28938 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.021s	sys 0.057s
I20260924 14:30:10.748241 28924 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260924 14:30:10.865695 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1034) took 1112 ms. Trace:
I20260924 14:30:10.866317 26398 rpcz_store.cc:276] 0924 14:30:09.753306 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.276378 (+523072us) service_pool.cc:227] Handling call
0924 14:30:10.276825 (+   447us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:10.277560 (+   735us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:10.277601 (+    41us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:10.278166 (+   565us) log.cc:842] Serialized 26 byte log entry
0924 14:30:10.769506 (+491340us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:10.769543 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:10.769617 (+    74us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:10.865394 (+ 95777us) raft_consensus.cc:1671] finished
0924 14:30:10.865424 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:10.865614 (+   190us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":102,"thread_start_us":490894,"threads_started":1,"wal-append.queue_time_us":585474}
I20260924 14:30:10.679286 28852 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:10.850636 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:10.796247 28805 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:11.134517 26332 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:11.159893 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:11.160763 26323 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:10.859292 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:10.861378 28901 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:10.854952 27182 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
W20260924 14:30:11.166723 27182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 398) took 1343 ms (client timeout 1772 ms). Trace:
W20260924 14:30:10.862666 28938 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.022s	sys 0.057s
I20260924 14:30:10.771011 28867 raft_consensus.cc:686] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:10.828931 26324 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:11.120682 28728 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:10.916272 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:10.839543 28686 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:10.772482 28234 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:10.858320 28895 consensus_meta.cc:287] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.003s
I20260924 14:30:10.904670 28859 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:11.040073 26251 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.012s	sys 0.028s
W20260924 14:30:11.227596 26251 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.013s	sys 0.028s
I20260924 14:30:11.027088 28524 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:10.903324 28883 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:11.038691 28933 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:11.123931 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 387507us
I20260924 14:30:10.830540 28838 raft_consensus.cc:3032] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Stepping down as leader of term 7
W20260924 14:30:11.148553 26328 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.010s	sys 0.036s
W20260924 14:30:11.167098 27182 rpcz_store.cc:269] 0924 14:30:09.823105 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:09.877859 (+ 54754us) service_pool.cc:227] Handling call
0924 14:30:11.166647 (+1288788us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:11.170733 28964 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:10.900594 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:11.221583 26399 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.017s	sys 0.032s
I20260924 14:30:11.162575 28892 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:11.171697 28857 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:11.198487 28852 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:11.235584 26326 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:11.439561 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:11.439944 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 511634us
I20260924 14:30:11.440568 28814 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260924 14:30:11.318213 27188 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.020s	sys 0.029s
I20260924 14:30:11.235967 28938 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:11.441910 27188 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.020s	sys 0.030s
I20260924 14:30:11.442138 28814 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:30:11.235392 28859 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:11.443542 28814 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:30:11.443929 28814 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:11.442054 28938 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:11.444017 28964 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:11.444991 28814 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:11.445778 28859 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:11.446182 28814 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
W20260924 14:30:11.446386 26399 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.021s	sys 0.032s
I20260924 14:30:11.447329 26329 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260924 14:30:11.447731 26328 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.013s	sys 0.045s
W20260924 14:30:11.345219 28967 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.387s	user 0.001s	sys 0.001s
I20260924 14:30:11.449313 28892 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:11.204807 28686 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:11.451725 28686 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:11.452641 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:11.453830 28838 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:11.455114 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:11.450510 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 599) took 2535 ms. Trace:
I20260924 14:30:11.454856 28838 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:11.458966 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:11.450510 26401 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:11.276965 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:11.470098 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:11.464468 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:11.471392 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:11.472798 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:11.475520 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:11.455861 26402 rpcz_store.cc:276] 0924 14:30:08.914992 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:08.915136 (+   144us) service_pool.cc:227] Handling call
0924 14:30:08.915676 (+   540us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:08.916140 (+   464us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:30:08.939605 (+ 23465us) log.cc:842] Serialized 14 byte log entry
0924 14:30:09.685484 (+745879us) spinlock_profiling.cc:243] Waited 9.95 ms on lock 0x7b18001e4fc0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd557b 00007f09b935888a
0924 14:30:10.308543 (+623059us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:10.333253 (+ 24710us) log.cc:842] Serialized 26 byte log entry
0924 14:30:10.333398 (+   145us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:10.333425 (+    27us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:10.333490 (+    65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:11.450192 (+1116702us) raft_consensus.cc:1671] finished
0924 14:30:11.450294 (+   102us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:11.450448 (+   154us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":23247,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":7510,"raft.queue_time_us":82607,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":28707,"spinlock_wait_cycles":22895872,"thread_start_us":620058,"threads_started":1,"wal-append.queue_time_us":1877886}
I20260924 14:30:11.467231 26224 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:11.488370 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:11.488539 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:11.489212 26401 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:11.489903 26224 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:11.490121 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:11.491209 26401 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:30:11.492723 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:11.457729 28816 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.009s	sys 0.012s
I20260924 14:30:11.477587 26332 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:11.486892 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:11.489069 26402 raft_consensus.cc:2362] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6. Re-sending same reply.
I20260924 14:30:11.498471 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:11.428820 28234 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:11.502584 27123 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.012s	sys 0.012s
I20260924 14:30:11.504770 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:11.508648 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 623) took 1265 ms. Trace:
I20260924 14:30:11.511121 26402 rpcz_store.cc:276] 0924 14:30:10.243380 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:11.508001 (+1264621us) service_pool.cc:227] Handling call
0924 14:30:11.508245 (+   244us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:11.508417 (+   172us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:11.508436 (+    19us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:11.508470 (+    34us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:11.508496 (+    26us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:11.508538 (+    42us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:11.508603 (+    65us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:11.512107 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:11.514230 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 611699us
W20260924 14:30:11.514945 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 648838us
W20260924 14:30:11.477842 28529 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.002s	sys 0.017s
W20260924 14:30:11.517022 27123 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.012s	sys 0.012s
W20260924 14:30:11.517369 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 704959us
I20260924 14:30:11.519130 26401 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
W20260924 14:30:11.490300 28610 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.011s	sys 0.012s
W20260924 14:30:11.499133 28816 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.009s	sys 0.012s
I20260924 14:30:11.487470 28859 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:11.478650 28857 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:11.229074 28895 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:11.502467 28924 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:11.509927 28838 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:11.522415 28529 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.002s	sys 0.018s
I20260924 14:30:11.501360 28578 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:11.657577 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:11.665052 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 146733us
I20260924 14:30:11.665287 27191 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:30:11.525084 28610 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.011s	sys 0.012s
I20260924 14:30:11.392439 28867 consensus_queue.cc:237] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:11.682009 26325 raft_consensus.cc:2379] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:11.533612 28857 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:11.683573 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 164782us
I20260924 14:30:11.688045 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:11.664004 28859 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:11.692898 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:11.696756 27181 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260924 14:30:11.662322 28816 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:11.687498 28924 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:11.672993 28578 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:11.703883 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:11.705360 28648 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.477s	user 0.010s	sys 0.006s
I20260924 14:30:11.705819 28909 consensus_queue.cc:1046] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:30:11.686667 28808 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.382s	user 0.012s	sys 0.008s
W20260924 14:30:11.706694 28794 consensus_meta.cc:287] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.594s	user 0.013s	sys 0.000s
W20260924 14:30:11.709930 28917 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.628s	user 0.008s	sys 0.004s
I20260924 14:30:11.733867 27125 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:11.673774 28728 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:11.678161 28982 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:11.688647 28857 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:11.793350 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:11.797135 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1010) took 1489 ms. Trace:
I20260924 14:30:11.797598 26328 rpcz_store.cc:276] 0924 14:30:10.307719 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.307859 (+   140us) service_pool.cc:227] Handling call
0924 14:30:10.308210 (+   351us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:10.308571 (+   361us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:10.308627 (+    56us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:10.311423 (+  2796us) log.cc:842] Serialized 26 byte log entry
0924 14:30:11.448343 (+1136920us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:11.448398 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:11.448489 (+    91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:11.796893 (+348404us) raft_consensus.cc:1671] finished
0924 14:30:11.796917 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:11.797075 (+   158us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":289,"raft.queue_time_us":599,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":27965,"thread_start_us":1004214,"threads_started":1,"wal-append.queue_time_us":1230392}
W20260924 14:30:11.750128 28896 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.832s	user 0.081s	sys 0.100s
I20260924 14:30:11.801647 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:11.802363 26254 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
W20260924 14:30:11.804436 28896 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.082s	sys 0.101s
I20260924 14:30:11.768343 28933 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:11.752177 28865 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:11.828191 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:11.731534 28859 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:11.790786 28875 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:11.834925 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:11.768069 28808 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:11.751209 28924 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:11.849402 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 621) took 1811 ms. Trace:
W20260924 14:30:11.846877 28805 consensus_meta.cc:287] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.676s	user 0.009s	sys 0.002s
W20260924 14:30:11.846707 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 348282us
I20260924 14:30:11.861308 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:11.751719 28648 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:11.768502 28917 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:11.767817 28794 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:11.846475 28885 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.971s	user 0.013s	sys 0.000s
I20260924 14:30:11.848232 26328 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:11.853500 28816 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:11.858464 26399 rpcz_store.cc:276] 0924 14:30:10.037617 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.602880 (+565263us) service_pool.cc:227] Handling call
0924 14:30:10.606269 (+  3389us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:10.612222 (+  5953us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:10.612273 (+    51us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:10.612915 (+   642us) log.cc:842] Serialized 26 byte log entry
0924 14:30:11.447005 (+834090us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:11.447061 (+    56us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:11.447158 (+    97us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:11.849191 (+402033us) raft_consensus.cc:1671] finished
0924 14:30:11.849212 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:11.849340 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2511,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":12778,"thread_start_us":767506,"threads_started":1,"wal-append.queue_time_us":901892}
I20260924 14:30:11.875056 28895 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:11.878738 28979 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:11.887833 28979 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:11.888231 28988 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:30:11.888324 28979 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:11.889786 28985 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260924 14:30:11.891251 28981 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260924 14:30:11.891819 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:11.893392 27189 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:11.878229 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:11.878954 27186 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:11.884418 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:11.784448 28728 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:11.877936 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:11.900039 27185 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:11.897234 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:11.904222 26404 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:11.904886 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:11.905706 26330 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: []
W20260924 14:30:11.906016 26297 connection.cc:232] RPC call timeout handler was delayed by -2.1625335409999025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
W20260924 14:30:11.906596 26297 connection.cc:232] RPC call timeout handler was delayed by -2.0068335409996507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
I20260924 14:30:11.906982 26400 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:11.907873 28896 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:11.907915 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:11.909840 28899 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.141s	user 0.013s	sys 0.000s
I20260924 14:30:11.910346 28899 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:11.913017 28814 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:11.914084 28814 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:11.914359 28578 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:11.914453 28814 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:11.900210 28816 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:11.916550 28979 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:11.917500 28816 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:11.913295 28917 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:11.922752 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1024) took 1152 ms. Trace:
I20260924 14:30:11.920797 26295 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:11.920867 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:11.917865 28686 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:11.928763 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:11.922587 26328 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:11.857468 28859 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:11.923231 26331 rpcz_store.cc:276] 0924 14:30:10.769967 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.771656 (+  1689us) service_pool.cc:227] Handling call
0924 14:30:10.796502 (+ 24846us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:10.848825 (+ 52323us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:10.848872 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:10.849427 (+   555us) log.cc:842] Serialized 26 byte log entry
0924 14:30:11.159574 (+310147us) spinlock_profiling.cc:243] Waited 899 us on lock 0x7b180032e120. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:30:11.499182 (+339608us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:11.499264 (+    82us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:11.499412 (+   148us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:11.894625 (+395213us) raft_consensus.cc:1671] finished
0924 14:30:11.894648 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:11.922690 (+ 28042us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8983,"raft.queue_time_us":12688,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":13063,"spinlock_wait_cycles":2066688,"thread_start_us":339221,"threads_started":1,"wal-append.queue_time_us":885839}
I20260924 14:30:11.853484 28865 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:11.886664 28885 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:11.863196 28805 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:11.938593 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:11.907006 26297 connection.cc:232] RPC call timeout handler was delayed by -1.9054322450001564s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260924 14:30:11.940692 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:11.947850 26297 connection.cc:232] RPC call timeout handler was delayed by -1.8838322450001215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
W20260924 14:30:11.948417 26297 connection.cc:232] RPC call timeout handler was delayed by -1.5631335409998428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25920000000000004s for a response
W20260924 14:30:11.949050 26297 connection.cc:232] RPC call timeout handler was delayed by -1.4797322449994681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20650000000000002s for a response
W20260924 14:30:11.949488 26297 connection.cc:232] RPC call timeout handler was delayed by -1.2333667829989281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260924 14:30:11.952535 26297 connection.cc:232] RPC call timeout handler was delayed by -1.0065667829994709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260924 14:30:11.944000 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 248922us
W20260924 14:30:12.022158 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 66416.5us
W20260924 14:30:12.022204 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 80055.2us
I20260924 14:30:12.025899 26329 raft_consensus.cc:2379] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:12.026942 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.027174 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:11.922813 28895 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:12.028689 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:11.930184 28896 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:12.031512 26250 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
W20260924 14:30:12.033504 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 254302us
I20260924 14:30:12.034228 26323 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:12.034915 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:12.027762 26252 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:11.930215 28524 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:30:12.039623 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.039031 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.047155 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:12.048133 26399 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:12.048527 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:12.049501 26331 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
W20260924 14:30:12.052171 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 208453us
I20260924 14:30:12.040480 28896 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:12.060534 27125 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:11.924926 28814 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:12.064204 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.065173 26254 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:12.067330 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.071347 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:12.088102 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.040284 28895 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:12.103094 26255 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260924 14:30:12.103770 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 262696us
I20260924 14:30:12.123790 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:12.124661 26331 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:12.109896 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:12.129325 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52936.3us
I20260924 14:30:12.128470 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.130664 26254 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:12.063616 28896 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:12.133037 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:12.142474 27191 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:12.136982 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:12.059576 28794 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:12.103729 28924 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:12.126180 28807 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.152s	user 0.007s	sys 0.032s
I20260924 14:30:12.078069 28979 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:12.207026 28937 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 2.146s	user 0.014s	sys 0.010s
I20260924 14:30:12.204550 28999 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:12.069715 28933 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:12.205729 28895 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:12.208575 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.254200 26250 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:12.215344 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.249706 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:12.251777 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 989) took 1535 ms. Trace:
I20260924 14:30:12.133924 28814 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:12.252421 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:12.259860 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.256036 26401 rpcz_store.cc:276] 0924 14:30:10.716432 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:11.520481 (+804049us) service_pool.cc:227] Handling call
0924 14:30:11.521040 (+   559us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:11.521237 (+   197us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:30:11.937510 (+416273us) log.cc:842] Serialized 14 byte log entry
0924 14:30:12.250790 (+313280us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:12.250876 (+    86us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:12.251175 (+   299us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:12.251321 (+   146us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:12.251655 (+   334us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":580451,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":6220,"thread_start_us":593065,"threads_started":2}
I20260924 14:30:12.264726 27126 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:12.264896 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:12.265480 26401 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:12.267856 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:12.157200 28980 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
W20260924 14:30:12.207594 28807 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.007s	sys 0.033s
I20260924 14:30:12.259311 26331 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:12.155176 28808 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:12.254138 28937 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:12.281597 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.283156 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.286072 26255 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:12.289939 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:12.291908 26401 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:12.250859 28899 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:12.295583 26331 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:12.293754 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:12.280733 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 325601us
I20260924 14:30:12.296244 28794 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:12.259908 28728 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:12.298311 28754 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:12.258051 28917 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:12.314805 27191 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:12.251283 28648 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:12.317973 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:12.325275 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:12.319357 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:12.321784 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:12.326040 27192 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:12.326303 26399 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:12.327293 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:12.327965 26399 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:12.320387 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:12.323014 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:12.328895 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:12.329700 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:12.259253 28524 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:12.330471 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:12.331110 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:12.325403 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:12.331643 28524 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:12.332697 26331 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:12.332609 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:12.333920 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:12.334070 28524 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:12.338477 28998 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.198s	user 0.000s	sys 0.000s
I20260924 14:30:12.258730 28999 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:12.335114 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:12.344476 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:12.345840 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 215657us
I20260924 14:30:12.344637 26331 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:12.334892 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:12.350214 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:12.369392 27192 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:12.350194 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:12.343191 26401 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
W20260924 14:30:12.376116 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 56011.9us
I20260924 14:30:12.383713 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:12.385618 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:12.385901 28610 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:12.383850 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:12.386514 28895 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:12.386637 26297 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:12.386968 28610 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:12.387131 28895 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:12.387368 28610 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:12.383630 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:12.387944 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:12.387981 28610 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:12.388134 26401 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:12.390564 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:12.388876 26295 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:12.344205 28885 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:12.349027 28899 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:12.321614 28686 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:12.345523 28728 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:12.376643 26331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00498335d0b847f299d347155be5c437"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:12.400202 26331 raft_consensus.cc:595] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:12.374281 28613 raft_consensus.cc:2741] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260924 14:30:12.377893 28865 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:12.389165 28648 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:12.414844 26328 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:12.341428 28794 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:12.423308 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 295355us
I20260924 14:30:12.345367 28867 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260924 14:30:12.396677 28859 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:12.423843 26328 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260924 14:30:12.345681 28917 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:12.389564 29006 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:12.425752 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 677) took 1707 ms. Trace:
I20260924 14:30:12.435132 26399 raft_consensus.cc:2396] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260924 14:30:12.435794 26401 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:12.429405 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:12.420288 28899 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:12.415071 28686 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:12.421938 28885 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:12.388211 28895 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:12.436903 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:12.438438 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:12.435421 27188 rpcz_store.cc:276] 0924 14:30:10.718442 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.718580 (+   138us) service_pool.cc:227] Handling call
0924 14:30:10.718883 (+   303us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:10.733904 (+ 15021us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:10.733931 (+    27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:10.734435 (+   504us) log.cc:842] Serialized 26 byte log entry
0924 14:30:11.442460 (+708025us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:11.442500 (+    40us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:11.442567 (+    67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:12.425486 (+982919us) raft_consensus.cc:1671] finished
0924 14:30:12.425518 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:12.425675 (+   157us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":969,"raft.queue_time_us":12426,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":9600,"thread_start_us":695836,"threads_started":1,"wal-append.queue_time_us":1020450}
I20260924 14:30:12.420661 28805 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:12.424743 28728 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260924 14:30:12.442389 26371 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.534s (SENT)
I20260924 14:30:12.377332 28933 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:12.440832 28991 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
I20260924 14:30:12.447356 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:12.449337 28524 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260924 14:30:12.451361 29013 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260924 14:30:12.452244 28857 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.000s
I20260924 14:30:12.444952 28928 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:12.443393 28899 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:12.453186 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1065) took 1445 ms. Trace:
I20260924 14:30:12.446406 26399 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:12.456754 26297 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:12.447127 28728 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:12.463956 26398 rpcz_store.cc:276] 0924 14:30:11.008349 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:11.666123 (+657774us) service_pool.cc:227] Handling call
0924 14:30:11.682678 (+ 16555us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:11.946736 (+264058us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:11.946770 (+    34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:12.026715 (+ 79945us) log.cc:842] Serialized 26 byte log entry
0924 14:30:12.142316 (+115601us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:12.142352 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:12.142418 (+    66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:12.452955 (+310537us) raft_consensus.cc:1671] finished
0924 14:30:12.452978 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:12.453126 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":167063,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":6450,"thread_start_us":359218,"threads_started":2,"wal-append.queue_time_us":305817}
I20260924 14:30:12.448140 29018 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:30:12.463958 28857 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:12.471266 28971 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.498s	user 0.007s	sys 0.006s
I20260924 14:30:12.471088 28524 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:12.471917 28928 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:12.472456 28928 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:12.473820 26257 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:12.474658 28794 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:12.475250 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1119) took 2474 ms. Trace:
I20260924 14:30:12.475646 26253 rpcz_store.cc:276] 0924 14:30:10.024617 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.059747 (+ 35130us) service_pool.cc:227] Handling call
0924 14:30:10.079597 (+ 19850us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:10.079917 (+   320us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:10.079944 (+    27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:10.080383 (+   439us) log.cc:842] Serialized 26 byte log entry
0924 14:30:10.265385 (+185002us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b180042ee00. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:30:10.785356 (+519971us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:10.785409 (+    53us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:10.785499 (+    90us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:12.468711 (+1683212us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:12.475055 (+  6344us) raft_consensus.cc:1671] finished
0924 14:30:12.475074 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:12.475195 (+   121us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13752,"raft.queue_time_us":668,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7420,"spinlock_wait_cycles":7115008,"thread_start_us":519731,"threads_started":1,"wal-append.queue_time_us":887069}
I20260924 14:30:12.478446 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:12.480023 28648 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:30:12.480785 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 647) took 3163 ms. Trace:
I20260924 14:30:12.481168 26251 rpcz_store.cc:276] 0924 14:30:09.317207 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:09.317506 (+   299us) service_pool.cc:227] Handling call
0924 14:30:09.317896 (+   390us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:09.941427 (+623531us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:30:10.309229 (+367802us) log.cc:842] Serialized 14 byte log entry
0924 14:30:11.236026 (+926797us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:11.236090 (+    64us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:11.236158 (+    68us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:12.480564 (+1244406us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b54004c9d54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:12.480612 (+    48us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:12.480730 (+   118us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":682057,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":8047,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":30296,"spinlock_wait_cycles":1441805440,"thread_start_us":1286023,"threads_started":2,"wal-append.queue_time_us":1462307}
I20260924 14:30:12.484148 28865 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:12.455317 28885 raft_consensus.cc:686] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:12.466516 28899 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:12.469264 28728 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:12.489097 26400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1041) took 2675 ms (client timeout 1793 ms). Trace:
W20260924 14:30:12.489521 26400 rpcz_store.cc:269] 0924 14:30:09.813472 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:09.831911 (+ 18439us) service_pool.cc:227] Handling call
0924 14:30:12.488844 (+2656933us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b5400217854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:12.488999 (+   155us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4487830400}
I20260924 14:30:12.490829 26402 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:12.490973 28999 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:12.491055 28885 consensus_queue.cc:237] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:12.492794 28986 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.617s	user 0.001s	sys 0.010s
W20260924 14:30:12.492940 28984 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.635s	user 0.002s	sys 0.008s
I20260924 14:30:12.494108 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1006) took 2340 ms. Trace:
I20260924 14:30:12.494446 26327 rpcz_store.cc:276] 0924 14:30:10.154232 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.315073 (+160841us) service_pool.cc:227] Handling call
0924 14:30:10.625410 (+310337us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:10.625967 (+   557us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:10.626010 (+    43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:10.634399 (+  8389us) log.cc:842] Serialized 26 byte log entry
0924 14:30:10.959819 (+325420us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b180042b6e0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:30:11.345235 (+385416us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:11.345274 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:11.345351 (+    77us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:12.492163 (+1146812us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:12.493896 (+  1733us) raft_consensus.cc:1671] finished
0924 14:30:12.493922 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:12.494054 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":55257,"raft.queue_time_us":121,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":25362,"spinlock_wait_cycles":101731072,"thread_start_us":385178,"threads_started":1,"wal-append.queue_time_us":1208080}
I20260924 14:30:12.486116 28917 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:12.474592 28928 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:12.454468 29016 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260924 14:30:12.532848 28875 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.003s	sys 0.004s
W20260924 14:30:12.538120 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 149316us
W20260924 14:30:12.535516 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 66454.1us
I20260924 14:30:12.533411 29006 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:12.559528 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:12.536773 28875 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.003s	sys 0.004s
I20260924 14:30:12.562652 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:12.569166 27188 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260924 14:30:12.609717 26369 connection.cc:232] RPC call timeout handler was delayed by -1.2810404690008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
W20260924 14:30:12.610483 26369 connection.cc:232] RPC call timeout handler was delayed by -1.1793404690006355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260924 14:30:12.614570 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61828.5us
I20260924 14:30:12.562484 28875 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:12.573683 28794 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election won for term 9
W20260924 14:30:12.622043 26369 connection.cc:232] RPC call timeout handler was delayed by -1.1631404690006093s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260924 14:30:12.627084 26369 connection.cc:232] RPC call timeout handler was delayed by -1.1181404690005365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18400000000000002s for a response
W20260924 14:30:12.627780 26369 connection.cc:232] RPC call timeout handler was delayed by -1.0812404690004769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260924 14:30:12.628247 26369 connection.cc:232] RPC call timeout handler was delayed by -1.0632404690004478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
W20260924 14:30:12.628625 26369 connection.cc:232] RPC call timeout handler was delayed by -1.0092404690003605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
W20260924 14:30:12.614099 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 56408.9us
I20260924 14:30:12.633049 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:12.636296 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 72975us
W20260924 14:30:12.668627 26224 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 1.997s (SENT)
I20260924 14:30:12.669160 26224 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:12.607874 28754 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260924 14:30:12.622280 28857 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:12.625991 28994 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:12.569038 29016 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260924 14:30:12.630980 28917 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:12.671451 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 140021us
I20260924 14:30:12.679041 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:12.677147 27123 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:12.713567 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:12.672363 28999 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:12.718678 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:12.723467 26224 leader_election.cc:334] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 2.053s (SENT)
I20260924 14:30:12.724113 26224 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:12.725621 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:12.728935 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:12.740906 28754 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:12.742493 28754 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:12.748994 28896 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
I20260924 14:30:12.749524 28896 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:12.752198 29045 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:12.752736 29045 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:12.679486 29028 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260924 14:30:12.671799 29006 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:12.699693 28883 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260924 14:30:12.718801 28794 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:12.760998 29032 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260924 14:30:12.688933 28994 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:12.745569 28999 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:12.892176 28857 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:12.891953 28640 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:12.733052 28875 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:12.733646 29005 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:12.689908 28917 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:12.802287 26371 leader_election.cc:334] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.761s (SENT)
W20260924 14:30:12.923179 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 120955us
W20260924 14:30:12.924180 26371 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.873s (SENT)
I20260924 14:30:12.924280 28917 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:12.924679 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:12.926821 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 241734us
I20260924 14:30:12.927774 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:12.803953 28814 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:12.894444 29016 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:12.965101 26397 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:30:12.745199 28754 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:12.971230 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 43511.1us
I20260924 14:30:13.014735 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1014) took 2665 ms. Trace:
I20260924 14:30:13.062739 26330 rpcz_store.cc:276] 0924 14:30:10.349404 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.349618 (+   214us) service_pool.cc:227] Handling call
0924 14:30:10.438440 (+ 88822us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:11.906324 (+1467884us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:13.014454 (+1108130us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540012dc54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:13.014672 (+   218us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3364114688}
W20260924 14:30:13.076138 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 52499.2us
I20260924 14:30:13.072791 28895 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:13.077804 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 54893.3us
I20260924 14:30:13.083343 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:12.941777 28917 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:13.104142 26224 leader_election.cc:334] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 2.434s (SENT)
I20260924 14:30:13.108335 26224 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:12.943637 28896 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:12.975811 28640 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:12.928620 28857 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:13.123703 28980 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260924 14:30:13.113837 28938 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:13.108177 28895 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:13.135519 28794 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:13.113109 28999 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:13.076937 28937 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:13.125118 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:13.167632 27123 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:12.970940 28814 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:13.170296 28896 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:13.173264 28578 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:13.118628 28917 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:13.153146 28640 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:13.234791 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:13.241456 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:13.244793 28968 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 1.844s	user 0.013s	sys 0.018s
I20260924 14:30:13.247891 29031 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:13.249753 29024 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:13.250634 29024 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:13.254103 26398 raft_consensus.cc:3032] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:13.261139 26398 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:13.255106 29024 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:13.260268 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.263247 26400 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:13.243295 27123 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:13.268246 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 992) took 2428 ms. Trace:
I20260924 14:30:13.268700 26402 rpcz_store.cc:276] 0924 14:30:10.840371 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:11.656729 (+816358us) service_pool.cc:227] Handling call
0924 14:30:12.490584 (+833855us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:12.491244 (+   660us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:13.268065 (+776821us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b5400217850. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:13.268189 (+   124us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1902579968}
W20260924 14:30:13.275341 26369 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.022s	sys 0.071s
W20260924 14:30:13.275756 26369 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.022s	sys 0.071s
W20260924 14:30:13.276031 27186 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 401) took 3384 ms (client timeout 2484 ms). Trace:
W20260924 14:30:13.276284 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 547418us
I20260924 14:30:13.277045 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:13.277669 26403 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
W20260924 14:30:13.267483 26404 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1045) took 3422 ms (client timeout 1541 ms). Trace:
I20260924 14:30:13.278633 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:13.276588 27186 rpcz_store.cc:269] 0924 14:30:09.891689 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.071543 (+179854us) service_pool.cc:227] Handling call
0924 14:30:13.275768 (+3204225us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540000c654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:13.275936 (+   168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3756416896}
W20260924 14:30:13.279106 26404 rpcz_store.cc:269] 0924 14:30:09.844981 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:09.848065 (+  3084us) service_pool.cc:227] Handling call
0924 14:30:13.267193 (+3419128us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54001e50d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:13.267384 (+   191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4471118592}
I20260924 14:30:13.268108 26398 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:13.285276 26398 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:13.289851 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:13.233623 28909 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
W20260924 14:30:13.267640 28610 consensus_meta.cc:287] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.879s	user 0.002s	sys 0.011s
I20260924 14:30:13.302234 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.302903 26328 raft_consensus.cc:2362] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8. Re-sending same reply.
I20260924 14:30:13.304813 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.287314 28795 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:13.290126 29031 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:13.305541 26328 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:13.265048 29049 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:13.267768 29029 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:13.286927 29048 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:13.310212 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:13.299201 28728 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.811s	user 0.007s	sys 0.005s
I20260924 14:30:13.296098 29050 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:30:13.316814 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 129423us
I20260924 14:30:13.299484 29016 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:13.320093 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:13.301471 28909 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:13.324316 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.327203 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.330941 27194 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260924 14:30:13.319204 27125 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.007s	sys 0.015s
W20260924 14:30:13.337523 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.007s	sys 0.016s
I20260924 14:30:13.340068 28870 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:13.340746 28917 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:13.342000 28857 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:13.342022 28578 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:13.342701 28578 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:13.343119 28578 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:13.343817 28814 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:13.315907 29048 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:13.312271 28795 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:13.325237 29016 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:13.346680 29048 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:13.346903 29016 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:13.348163 28795 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:13.350235 29058 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:13.318651 28728 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:13.353670 28728 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:13.353943 28977 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.515s	user 0.013s	sys 0.044s
I20260924 14:30:13.355033 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:13.355845 26329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 399) took 1973 ms (client timeout 1644 ms). Trace:
W20260924 14:30:13.356258 26329 rpcz_store.cc:269] 0924 14:30:11.382231 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:11.465191 (+ 82960us) service_pool.cc:227] Handling call
0924 14:30:13.355635 (+1890444us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b54002765d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:13.355764 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1170596608}
I20260924 14:30:13.342877 28895 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:13.346357 28965 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 2.359s	user 0.016s	sys 0.031s
I20260924 14:30:13.327425 28896 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.014s
W20260924 14:30:13.357380 26324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 664) took 4109 ms (client timeout 2108 ms). Trace:
I20260924 14:30:13.298106 28610 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:13.362185 26324 rpcz_store.cc:269] 0924 14:30:09.248137 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:09.627457 (+379320us) service_pool.cc:227] Handling call
0924 14:30:13.357227 (+3729770us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b54007d3cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:13.357318 (+    91us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2289362304}
W20260924 14:30:13.337981 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 696513us
I20260924 14:30:13.364069 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1026) took 4069 ms. Trace:
I20260924 14:30:13.369287 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:13.370219 27192 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:13.375034 26295 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:13.379666 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:13.364511 28899 consensus_meta.cc:287] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.876s	user 0.010s	sys 0.002s
I20260924 14:30:13.366122 26405 rpcz_store.cc:276] 0924 14:30:09.294654 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:09.311521 (+ 16867us) service_pool.cc:227] Handling call
0924 14:30:09.311840 (+   319us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:09.329871 (+ 18031us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:09.329901 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:09.608847 (+278946us) log.cc:842] Serialized 26 byte log entry
0924 14:30:10.341618 (+732771us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:10.341654 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:10.341715 (+    61us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:13.327630 (+2985915us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:13.358902 (+ 31272us) raft_consensus.cc:1671] finished
0924 14:30:13.358922 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:13.364005 (+  5083us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15880,"raft.queue_time_us":9031,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":31759,"spinlock_wait_cycles":1077075328,"thread_start_us":690379,"threads_started":1,"wal-append.queue_time_us":1219264}
I20260924 14:30:13.384240 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:13.383482 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.385035 29057 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:13.385524 29057 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:13.383654 26257 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:13.385844 29057 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:13.363112 28975 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.471s	user 0.009s	sys 0.011s
W20260924 14:30:13.388731 28975 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.496s	user 0.010s	sys 0.011s
I20260924 14:30:13.388902 28909 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:13.391676 29063 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:13.392910 28640 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:13.392432 29063 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:13.395323 29063 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:13.378841 28754 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260924 14:30:13.377413 28814 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:13.340823 28870 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:13.367957 28917 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260924 14:30:13.399820 28610 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:13.400286 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1115) took 3415 ms. Trace:
I20260924 14:30:13.417045 26256 rpcz_store.cc:276] 0924 14:30:09.985194 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:09.985333 (+   139us) service_pool.cc:227] Handling call
0924 14:30:10.117274 (+131941us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:30:10.117466 (+   192us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:11.198938 (+1081472us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:11.198984 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:11.461217 (+262233us) log.cc:842] Serialized 26 byte log entry
0924 14:30:11.843199 (+381982us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:12.053351 (+210152us) log.cc:842] Serialized 14 byte log entry
0924 14:30:12.053644 (+   293us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:13.399903 (+1346259us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b54002882d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:13.399976 (+    73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:13.400036 (+    60us) raft_consensus.cc:1671] finished
0924 14:30:13.400053 (+    17us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:13.400212 (+   159us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4119,"prepare.queue_time_us":590207,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":50054,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":12060,"spinlock_wait_cycles":1571201792,"thread_start_us":934598,"threads_started":3,"wal-append.queue_time_us":562565}
I20260924 14:30:13.404326 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:13.405256 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:13.406000 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:13.387997 28899 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:13.433517 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 113451us
I20260924 14:30:13.438053 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.445675 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:13.450589 27123 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:13.451746 26405 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:13.459957 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.459815 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.460739 26326 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:13.460705 26328 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:13.463369 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:13.463694 26371 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.007s (SENT)
I20260924 14:30:13.464226 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:13.469349 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.470129 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.473392 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.480573 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.455726 28754 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:13.478549 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.421480 28814 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:13.490476 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.471329 28794 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:13.504662 28899 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:13.506218 28578 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:13.508634 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:13.509305 26398 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:13.509651 28814 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:13.476502 29052 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:13.494227 26405 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:13.488391 28754 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:13.534461 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:13.535696 26405 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:30:13.548810 26325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1019) took 2989 ms (client timeout 1881 ms). Trace:
I20260924 14:30:13.548450 29016 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:13.549232 26325 rpcz_store.cc:269] 0924 14:30:10.559858 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.634266 (+ 74408us) service_pool.cc:227] Handling call
0924 14:30:13.548534 (+2914268us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54007cf6d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:13.548724 (+   190us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2161918208}
I20260924 14:30:13.550992 29016 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:13.553455 26332 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:13.556053 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:13.557006 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:13.558914 29031 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:13.532958 26398 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:13.603739 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:13.604619 26329 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:13.604279 29045 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:13.605532 29045 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:13.600554 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.570601 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.579320 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.565534 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 684) took 2040 ms. Trace:
I20260924 14:30:13.567924 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.573843 26298 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:13.607839 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:13.594161 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:13.609334 27192 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:13.609455 26328 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:13.610885 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:13.608144 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:13.593586 26371 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.740s (SENT)
I20260924 14:30:13.611851 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:13.613103 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.613732 26405 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:30:13.613572 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:13.614279 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.614591 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:13.617615 26371 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.140s (SENT)
I20260924 14:30:13.618062 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:13.607019 26399 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:13.623364 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:13.624089 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:13.608176 27182 rpcz_store.cc:276] 0924 14:30:11.524870 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:11.525746 (+   876us) service_pool.cc:227] Handling call
0924 14:30:11.526023 (+   277us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:11.771739 (+245716us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:30:12.072260 (+300521us) log.cc:842] Serialized 14 byte log entry
0924 14:30:12.208401 (+136141us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:12.208451 (+    50us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:12.208501 (+    50us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:13.565299 (+1356798us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b540031f954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:13.565345 (+    46us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:13.565475 (+   130us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":51443,"prepare.queue_time_us":531349,"raft.queue_time_us":163671,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":44005,"spinlock_wait_cycles":571277312,"thread_start_us":299769,"threads_started":2,"wal-append.queue_time_us":264600,"wal-append.run_cpu_time_us":1054,"wal-append.run_wall_time_us":1002009}
I20260924 14:30:13.630272 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.632688 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:13.634030 26399 raft_consensus.cc:2396] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260924 14:30:13.634316 26405 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:30:13.568997 29046 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
I20260924 14:30:13.559139 29007 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:13.638924 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:13.636381 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:13.572693 28976 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 1.687s	user 0.023s	sys 0.065s
I20260924 14:30:13.559289 29031 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:13.638861 27192 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260924 14:30:13.635888 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:13.655148 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:13.655539 26326 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:13.648731 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.657807 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.657914 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:13.659575 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:13.566792 28578 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:13.563213 28870 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:13.571725 28923 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:13.670753 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.630241 29041 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:13.652364 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:13.630028 29016 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:13.709112 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:13.664690 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:13.621479 28857 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:13.637617 28999 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:30:13.708438 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 59416.8us
W20260924 14:30:13.720155 26371 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.795s (SENT)
I20260924 14:30:13.639593 28865 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:13.718513 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:13.639846 29007 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:13.720669 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:13.661490 29031 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:13.654065 29073 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:13.660662 28895 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:13.621805 29048 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:13.575759 28754 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:13.712296 29061 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:13.708437 29041 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:13.712626 28923 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:13.750394 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 84376.5us
I20260924 14:30:13.751153 27191 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260924 14:30:13.720337 28870 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:13.751739 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1040) took 3996 ms. Trace:
I20260924 14:30:13.734524 29007 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:13.728130 28857 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:13.739435 28938 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:30:13.763664 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 144270us
I20260924 14:30:13.737222 29016 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:13.718750 28937 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260924 14:30:13.760208 26406 rpcz_store.cc:276] 0924 14:30:09.755551 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.344185 (+588634us) service_pool.cc:227] Handling call
0924 14:30:10.462520 (+118335us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:10.570402 (+107882us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:10.570445 (+    43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:10.601505 (+ 31060us) log.cc:842] Serialized 26 byte log entry
0924 14:30:10.987965 (+386460us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:10.988022 (+    57us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:10.988129 (+   107us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:13.572060 (+2583931us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:13.742507 (+170447us) raft_consensus.cc:1671] finished
0924 14:30:13.742538 (+    31us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:13.751643 (+  9105us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":259,"raft.queue_time_us":107341,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":17495,"spinlock_wait_cycles":36033408,"thread_start_us":333525,"threads_started":1,"wal-append.queue_time_us":1242238}
I20260924 14:30:13.769182 26402 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
W20260924 14:30:13.767035 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 143729us
W20260924 14:30:13.770859 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 290380us
I20260924 14:30:13.771435 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:13.780323 28794 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:13.781044 26224 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:13.781738 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:13.784045 28794 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:13.788396 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 402) took 2298 ms. Trace:
I20260924 14:30:13.790659 26332 rpcz_store.cc:276] 0924 14:30:11.490153 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:11.524604 (+ 34451us) service_pool.cc:227] Handling call
0924 14:30:13.553248 (+2028644us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:13.553957 (+   709us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:13.788107 (+234150us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54007cf6d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:13.788312 (+   205us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4663459584}
I20260924 14:30:13.793447 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 682) took 2016 ms. Trace:
I20260924 14:30:13.794041 26258 rpcz_store.cc:276] 0924 14:30:11.776681 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:11.776839 (+   158us) service_pool.cc:227] Handling call
0924 14:30:11.800652 (+ 23813us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:11.863570 (+ 62918us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:11.876276 (+ 12706us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:11.876321 (+    45us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:11.876359 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:13.793134 (+1916775us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b5400715f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:13.793182 (+    48us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:13.793365 (+   183us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":144198016}
I20260924 14:30:13.759871 29041 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:13.767246 28754 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:13.766306 28870 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:13.832696 26297 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.909s (SENT)
W20260924 14:30:13.829030 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 69906.9us
W20260924 14:30:13.829612 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46157.5us
I20260924 14:30:13.834677 26332 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260924 14:30:13.727051 29057 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:13.832696 26371 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.147s (SENT)
I20260924 14:30:13.840157 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:13.835574 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 56286.8us
I20260924 14:30:13.759341 29050 consensus_queue.cc:1046] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.071s
W20260924 14:30:13.828899 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 52094us
I20260924 14:30:13.835024 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:13.856834 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:13.801048 29061 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:13.859107 26332 raft_consensus.cc:2379] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:13.769255 28857 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:13.865454 26371 leader_election.cc:334] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.840s (SENT)
I20260924 14:30:13.871949 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:13.858711 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 395) took 2653 ms. Trace:
I20260924 14:30:13.872346 26404 pending_rounds.cc:76] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 2
I20260924 14:30:13.873152 26404 pending_rounds.cc:97] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260924 14:30:13.872583 26323 rpcz_store.cc:276] 0924 14:30:11.206692 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:11.206856 (+   164us) service_pool.cc:227] Handling call
0924 14:30:11.441054 (+234198us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:12.039313 (+598259us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:13.858440 (+1819127us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54004100d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:13.858643 (+   203us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1363598080}
I20260924 14:30:13.873785 26404 raft_consensus.cc:2866] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:13.828473 28964 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:13.836431 28870 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:13.850687 28794 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260924 14:30:13.878777 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 43844.6us
I20260924 14:30:13.870440 26332 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260924 14:30:13.769626 28917 consensus_queue.cc:1159] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e09fbebe29a49d291d23f9d395fe335" responder_term: 7 status { last_received { term: 7 index: 4 } last_committed_idx: 3 last_received_current_leader { term: 7 index: 4 } }
I20260924 14:30:13.863149 29050 raft_consensus.cc:3032] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Stepping down as leader of term 6
W20260924 14:30:13.873363 29075 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260924 14:30:13.921064 26371 leader_election.cc:334] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.152s (SENT)
I20260924 14:30:13.930511 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:13.864825 29087 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:13.838975 28869 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
W20260924 14:30:13.940908 27189 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 471) took 2053 ms (client timeout 2014 ms). Trace:
I20260924 14:30:13.940935 27192 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260924 14:30:13.942860 29016 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:13.938519 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:13.945530 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32930 (request call id 24) took 2258 ms. Trace:
W20260924 14:30:13.948298 29078 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
W20260924 14:30:13.948690 29079 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260924 14:30:13.952805 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 104011us
I20260924 14:30:13.877960 29073 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:13.968724 26331 rpcz_store.cc:276] 0924 14:30:11.687499 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:12.376141 (+688642us) service_pool.cc:227] Handling call
0924 14:30:13.945172 (+1569031us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b540067b4a0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92e3dc0 00007f09b93489c2 00007f09be29281a 00007f09b7985d04 00007f09b7985c92 00007f09b7985bc1 00007f09b7985b3c 00007f09b7984df2 00007f09b777894c 00007f09b7777d86
0924 14:30:13.945443 (+   271us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4060032}
W20260924 14:30:13.965667 27189 rpcz_store.cc:269] 0924 14:30:11.886923 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:11.894540 (+  7617us) service_pool.cc:227] Handling call
0924 14:30:13.940512 (+2045972us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b540014ffd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:13.940681 (+   169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3550474112}
I20260924 14:30:13.907219 29050 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:13.953758 29040 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:13.966598 29087 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:14.001821 26328 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:14.003979 26329 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: [5.4-6.5]
I20260924 14:30:13.848053 28923 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:14.010892 26371 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.533s (SENT)
I20260924 14:30:14.011569 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:14.002060 29087 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:30:14.020128 29072 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.516s	user 0.003s	sys 0.001s
I20260924 14:30:14.014477 29016 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:13.974887 29083 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260924 14:30:14.017238 29048 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:14.006541 29061 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:14.038579 29083 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:14.040231 29083 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:14.022797 29050 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:14.041622 29050 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:14.041210 29083 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:14.042979 29083 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:30:14.066000 29055 log.cc:924] Time spent T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.732s	user 0.000s	sys 0.003s
W20260924 14:30:14.068122 29082 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
W20260924 14:30:14.071379 29056 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.763s	user 0.000s	sys 0.004s
W20260924 14:30:13.936121 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 155114us
I20260924 14:30:14.114213 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:30:14.124102 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 153908us
I20260924 14:30:14.124547 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1082) took 1412 ms. Trace:
I20260924 14:30:14.125300 26397 rpcz_store.cc:276] 0924 14:30:12.711462 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:12.711640 (+   178us) service_pool.cc:227] Handling call
0924 14:30:12.711994 (+   354us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:12.965766 (+253772us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:14.123732 (+1157966us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54001e46d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:14.124166 (+   434us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":531329792}
I20260924 14:30:14.133801 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:14.136749 28640 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:14.138046 28640 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:14.132627 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.141392 28899 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:14.142537 28899 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:14.144246 28964 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:14.146660 28964 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:14.163542 26400 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:14.164410 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 702) took 1360 ms. Trace:
I20260924 14:30:14.165040 27188 rpcz_store.cc:276] 0924 14:30:12.804362 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:12.804562 (+   200us) service_pool.cc:227] Handling call
0924 14:30:12.805072 (+   510us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:12.805314 (+   242us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:12.805368 (+    54us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:12.805946 (+   578us) log.cc:842] Serialized 26 byte log entry
0924 14:30:13.179713 (+373767us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:13.179828 (+   115us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:13.180004 (+   176us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:14.069131 (+889127us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:14.163696 (+ 94565us) raft_consensus.cc:1671] finished
0924 14:30:14.163733 (+    37us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:14.164177 (+   444us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":64328064,"thread_start_us":216584,"threads_started":1,"wal-append.queue_time_us":325995}
I20260924 14:30:14.164515 28964 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:14.171406 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:14.178220 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.143412 28640 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:14.186878 26258 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:14.202601 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 637) took 1994 ms. Trace:
I20260924 14:30:14.202903 26257 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260924 14:30:14.203223 26401 rpcz_store.cc:276] 0924 14:30:12.208045 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:12.401023 (+192978us) service_pool.cc:227] Handling call
0924 14:30:12.435635 (+ 34612us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:12.437092 (+  1457us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:14.200190 (+1763098us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b5400477050. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:14.202364 (+  2174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":44987776}
I20260924 14:30:14.204581 28857 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:14.209507 29057 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:14.210608 29057 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:14.211138 26324 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:14.213071 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.213449 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:14.214167 26252 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:30:14.214447 26329 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:14.215713 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 102075us
I20260924 14:30:14.222919 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:14.223191 26255 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:14.473397 28728 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260924 14:30:14.471817 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:14.272468 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:14.449365 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:14.395352 29107 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.101s	user 0.000s	sys 0.002s
I20260924 14:30:14.263338 29057 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:14.497802 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 259076us
W20260924 14:30:14.323673 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 190414us
I20260924 14:30:14.427218 26329 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:14.229638 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.310932 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 979) took 4684 ms. Trace:
I20260924 14:30:14.523283 26327 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:14.524580 26327 raft_consensus.cc:595] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:14.525079 26327 raft_consensus.cc:603] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:14.527221 28899 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:14.528043 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 380483us
I20260924 14:30:14.529320 26251 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
W20260924 14:30:14.541146 28754 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.699s	user 0.008s	sys 0.012s
I20260924 14:30:14.549494 26324 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:14.300144 29053 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:14.339279 28857 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260924 14:30:14.532343 29007 consensus_meta.cc:287] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.754s	user 0.013s	sys 0.010s
W20260924 14:30:14.528591 28870 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.005s
I20260924 14:30:14.516587 26403 rpcz_store.cc:276] 0924 14:30:09.626651 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:10.250682 (+624031us) service_pool.cc:227] Handling call
0924 14:30:13.277421 (+3026739us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:13.287936 (+ 10515us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:14.292742 (+1004806us) spinlock_profiling.cc:243] Waited 3.03 s on lock 0x7b54001e50d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:14.310646 (+ 17904us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6960086784}
W20260924 14:30:14.539098 29031 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.787s	user 0.016s	sys 0.001s
I20260924 14:30:14.311476 28923 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:14.580611 29083 consensus_meta.cc:287] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.007s
I20260924 14:30:14.580856 26249 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:14.581482 28870 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:14.583253 29031 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:14.586447 29031 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:14.587353 29050 consensus_meta.cc:287] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.505s	user 0.005s	sys 0.006s
I20260924 14:30:14.587821 27182 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:14.584118 28870 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:14.589138 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 222318us
I20260924 14:30:14.403434 29085 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:14.602697 26258 raft_consensus.cc:2396] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260924 14:30:14.605993 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:14.611225 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 228361us
I20260924 14:30:14.593797 29007 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:14.607648 28754 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:14.605623 28857 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:14.620550 29081 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:14.620609 28917 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:30:14.621997 29041 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.728s	user 0.014s	sys 0.000s
I20260924 14:30:14.628751 26249 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:14.618091 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:14.638206 26397 raft_consensus.cc:2396] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:14.614346 29057 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:14.637809 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.627772 28857 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:14.637102 28754 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:14.657665 27190 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260924 14:30:14.647069 28995 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:14.668759 29007 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:14.664793 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.651633 29057 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:14.674454 29057 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:14.680389 26249 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:14.648068 29081 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:14.634027 29041 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:14.703497 29036 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:14.693321 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.651538 29005 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260924 14:30:14.711552 28640 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:14.704350 29036 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:14.646740 29085 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:14.697849 28995 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:14.693233 26371 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.796s (SENT) [suppressed 56 similar messages]
I20260924 14:30:14.731480 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.693569 28754 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:14.736977 26325 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:14.737190 26251 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:14.739897 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.740811 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.741381 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:14.741750 26256 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:14.742741 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.742650 28640 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:14.743240 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.751084 26251 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:14.751165 29041 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:14.752101 28640 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:14.745369 29083 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:14.745821 28591 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:14.748205 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:14.742725 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:14.748703 29081 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:14.749310 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1037) took 1023 ms. Trace:
I20260924 14:30:14.750073 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:14.768160 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 44783.9us
I20260924 14:30:14.770342 26253 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:14.759745 28591 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:14.776377 28870 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:14.777554 28870 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:14.777160 28591 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:14.777989 28870 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:14.778951 29036 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:14.777554 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1038) took 1160 ms. Trace:
I20260924 14:30:14.782756 26325 rpcz_store.cc:276] 0924 14:30:13.617407 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:13.637568 (+ 20161us) service_pool.cc:227] Handling call
0924 14:30:14.777308 (+1139740us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54005baed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:14.777471 (+   163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2426622848}
I20260924 14:30:14.796167 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:14.729382 29061 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.150s
I20260924 14:30:14.803634 27190 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:14.810698 27192 raft_consensus.cc:2396] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260924 14:30:14.779090 28870 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:14.770998 29083 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:14.764320 26330 rpcz_store.cc:276] 0924 14:30:13.617068 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:13.654113 (+ 37045us) service_pool.cc:227] Handling call
0924 14:30:13.654474 (+   361us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:13.654734 (+   260us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:30:14.463834 (+809100us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b180041abe0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b8fdd82c 00007f09b64089dc 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6423155 00007f09bcd23167 00007f09b9358807 00007f09b9367f0f
0924 14:30:14.464078 (+   244us) log.cc:842] Serialized 14 byte log entry
0924 14:30:14.575033 (+110955us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:14.597025 (+ 21992us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:14.608559 (+ 11534us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:14.619635 (+ 11076us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:14.640602 (+ 20967us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":53463,"prepare.queue_time_us":209335,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":61226,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":2087,"spinlock_wait_cycles":49660800,"thread_start_us":238562,"threads_started":2,"wal-append.queue_time_us":42670}
I20260924 14:30:14.826094 26397 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:14.767931 28865 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.169s
I20260924 14:30:14.844568 29095 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:14.845191 26326 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:14.845217 26325 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:14.845588 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:14.849825 26325 raft_consensus.cc:595] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:14.860471 29109 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.206s	user 0.000s	sys 0.002s
I20260924 14:30:14.864142 28591 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:14.846439 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:14.868978 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:14.870301 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:14.876345 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 56342.3us
I20260924 14:30:14.882722 26253 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
W20260924 14:30:14.865845 29045 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.245s	user 0.016s	sys 0.004s
I20260924 14:30:14.846459 29041 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:14.899596 27192 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:14.864461 29085 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:14.865926 28994 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:14.872239 28995 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:14.903409 27188 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:14.890007 28640 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:14.892808 29057 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:14.928782 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 703) took 1582 ms. Trace:
I20260924 14:30:14.929380 27186 rpcz_store.cc:276] 0924 14:30:13.312509 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:13.312634 (+   125us) service_pool.cc:227] Handling call
0924 14:30:13.325127 (+ 12493us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:13.336465 (+ 11338us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:13.336492 (+    27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:14.205319 (+868827us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b2c001c06e8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:14.206157 (+   838us) log.cc:842] Serialized 26 byte log entry
0924 14:30:14.319485 (+113328us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:14.341870 (+ 22385us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:14.348903 (+  7033us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:14.872733 (+523830us) raft_consensus.cc:1671] finished
0924 14:30:14.872751 (+    18us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:14.894422 (+ 21671us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10955,"raft.queue_time_us":11064,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":52561280,"thread_start_us":60296,"threads_started":1,"wal-append.queue_time_us":386961}
I20260924 14:30:14.931613 29108 consensus_queue.cc:1046] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260924 14:30:14.932224 29087 consensus_meta.cc:287] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.851s	user 0.012s	sys 0.006s
I20260924 14:30:14.932802 29085 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:14.892864 29031 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
W20260924 14:30:14.948379 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 54916.8us
I20260924 14:30:14.919962 28870 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:14.910728 28640 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:14.913625 29041 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:14.933655 29085 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:14.969657 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:14.970661 27194 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:14.899574 29095 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:14.972170 28640 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:14.973635 28640 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:14.974323 29095 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:14.972182 29121 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:14.976171 29121 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:14.976574 29121 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:14.976939 29041 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:14.975615 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53193.5us
I20260924 14:30:14.972246 28865 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:14.956652 28994 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:14.989871 26324 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:14.997941 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:14.932838 29087 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:14.990373 29050 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:15.013886 26324 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:14.986377 28865 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.021430 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:15.022930 29083 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:15.023670 29050 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.024232 28857 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:15.026108 29050 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:15.027256 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:15.014628 28994 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.033156 26330 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:15.035035 27188 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:15.035437 26257 raft_consensus.cc:2354] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4.
I20260924 14:30:15.038890 28994 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:15.039534 26295 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:15.039853 29006 raft_consensus.cc:3032] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:15.040298 29006 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:15.040442 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 72447.1us
I20260924 14:30:15.041313 29006 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:15.041926 26295 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:15.042691 29006 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:15.043370 29061 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:15.044956 28867 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:15.045243 29061 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:15.045421 28867 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:15.045644 29061 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:15.045818 28867 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:14.995167 28995 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260924 14:30:15.052951 28865 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:15.054418 29053 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:15.056068 26401 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:15.052918 28857 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:15.061257 29085 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:15.004679 29045 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:15.065968 29045 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.096890 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:15.065634 29053 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260924 14:30:15.096143 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:15.136274 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:15.141085 29041 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:15.123996 29045 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:15.143356 28795 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:15.144784 29087 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:15.118259 29007 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:15.146654 29007 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.165513 28754 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:15.183353 28865 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:15.184135 28865 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.186213 28865 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:15.191756 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:15.193985 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 348611us
I20260924 14:30:15.111850 28994 consensus_queue.cc:1046] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260924 14:30:15.207671 26330 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260924 14:30:15.182389 29007 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:15.224488 26324 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:15.192966 29041 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:30:15.246863 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47445.9us
W20260924 14:30:15.240993 29056 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260924 14:30:15.232156 29005 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:15.255703 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:15.264300 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 183942us
I20260924 14:30:15.264870 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:15.265377 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:15.241653 28754 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:15.276932 26249 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:15.297122 27192 raft_consensus.cc:2396] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260924 14:30:15.278282 28994 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260924 14:30:15.333109 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 43799.6us
I20260924 14:30:15.345739 26249 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:15.346983 26401 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:15.325565 28994 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:15.292661 28754 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.394182 28754 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:15.351763 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:15.352074 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:15.395498 28640 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:15.351963 26079 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 6, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 6 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:15.396166 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:15.329167 29050 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:15.397186 29050 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:15.401989 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:15.402872 26406 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:15.403053 28896 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:15.403555 28896 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:15.397758 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:15.359251 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:15.404484 28896 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.336651 28895 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:15.406602 26400 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260924 14:30:15.407076 26400 pending_rounds.cc:76] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 6
I20260924 14:30:15.360203 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:15.359174 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:15.408455 26399 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:15.407703 26079 catalog_manager.cc:5495] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 6, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:15.410718 26249 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260924 14:30:15.410257 26079 catalog_manager.cc:5495] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 8, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 8 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:15.411128 26249 pending_rounds.cc:76] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 4
I20260924 14:30:15.411504 26249 pending_rounds.cc:97] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260924 14:30:15.361212 29045 raft_consensus.cc:3032] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:15.411777 26249 raft_consensus.cc:2866] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:15.412369 26402 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:15.363180 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:15.365440 28867 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:15.366178 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:15.376677 29061 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.287016 29041 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:15.412026 29045 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:15.386948 29065 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:15.358623 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:15.392042 28994 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:15.291553 29005 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.396166 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:15.411860 26079 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:15.413293 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:15.416270 29036 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:15.417109 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:15.417985 29083 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:30:15.358588 29109 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.066s	user 0.000s	sys 0.002s
I20260924 14:30:15.407341 26400 pending_rounds.cc:97] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260924 14:30:15.361950 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:15.333782 28865 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:15.449373 27194 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:15.450691 26403 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:15.450929 26397 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:15.396060 28640 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:15.433473 28896 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:15.480535 26400 raft_consensus.cc:2866] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:15.429093 29053 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:15.426802 29050 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:15.509382 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 96156.7us
I20260924 14:30:15.477468 29005 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:15.480976 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48647.1us
W20260924 14:30:15.500404 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 83358.9us
I20260924 14:30:15.464572 29065 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:15.406719 28895 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:15.480512 29083 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
W20260924 14:30:15.525658 29118 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.150s	user 0.002s	sys 0.000s
I20260924 14:30:15.515005 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:15.515690 29053 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.516114 26079 catalog_manager.cc:5495] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:15.532650 28640 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:15.513625 28795 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:15.533538 28795 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:15.533931 28795 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:15.508822 29036 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:15.554448 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:15.497640 28865 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.521595 29083 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:15.563329 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:15.570094 28852 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:15.576997 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:15.578460 28795 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:15.580765 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 140899us
I20260924 14:30:15.517660 29081 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260924 14:30:15.583856 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:15.556226 29036 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:15.594022 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:15.604830 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 1.0067e+06us
I20260924 14:30:15.608855 26397 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:30:15.606957 29077 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:15.590230 29031 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260924 14:30:15.630002 27126 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:15.631539 26079 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:15.632535 26403 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:15.564429 29045 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:15.649487 26371 leader_election.cc:334] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.996s (SENT)
I20260924 14:30:15.629611 29095 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:15.672627 28865 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:15.646893 29065 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.674587 29065 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:15.653985 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:15.655270 29061 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:15.659875 26077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55200 (request call id 127) took 15313 ms (client timeout 15000 ms). Trace:
I20260924 14:30:15.660346 29053 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:15.660648 29036 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.661751 28754 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:15.663761 26079 catalog_manager.cc:5495] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 6, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 6 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
W20260924 14:30:15.666963 29114 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.290s	user 0.000s	sys 0.004s
I20260924 14:30:15.633105 29077 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:15.643756 29050 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:15.738514 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48318.3us
W20260924 14:30:15.751641 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 193138us
I20260924 14:30:15.619544 29083 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.686944 29041 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:15.710800 29121 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.734s	user 0.015s	sys 0.000s
I20260924 14:30:15.713702 26079 catalog_manager.cc:5495] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 8, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 8 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:15.713037 29053 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:15.772181 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:15.710474 29077 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:15.748420 29006 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.705s	user 0.005s	sys 0.011s
I20260924 14:30:15.711668 28754 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:15.679432 28895 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:15.750901 29045 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:15.805447 26079 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 9, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 9 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:15.804183 26403 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:15.752604 29108 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:15.748613 29050 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.817580 29052 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:15.810134 28795 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:15.819983 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 468182us
I20260924 14:30:15.832141 26295 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:15.833921 28883 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:15.835593 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:15.834496 28883 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:15.838514 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:15.838831 26330 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:15.839361 27186 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:15.839551 26330 raft_consensus.cc:595] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:15.840034 26330 raft_consensus.cc:603] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:15.839600 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:15.841239 27188 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:15.843071 26330 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:15.843855 26330 raft_consensus.cc:595] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:15.844251 26330 raft_consensus.cc:603] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:15.837641 28883 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:15.846066 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:30:15.850375 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:30:15.851771 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:15.860118 29122 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:15.860917 29122 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:15.863788 29122 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:15.850373 27186 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:15.866519 26295 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:15.867606 28895 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:15.868213 28895 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:15.817780 29083 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:15.827129 26079 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:15.815033 29053 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:15.809041 29121 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:15.884472 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 720) took 1365 ms. Trace:
I20260924 14:30:15.891719 28867 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:15.892431 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:15.891832 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 79009us
I20260924 14:30:15.893096 27186 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:15.894280 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:15.897557 29095 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:15.900719 27186 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:15.901991 26295 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:15.902956 28867 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:15.903499 28867 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:15.903631 29121 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:15.904644 28867 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:15.905790 26079 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 3, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 3 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:15.908769 26079 catalog_manager.cc:5495] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 8, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 8 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:15.886770 26255 rpcz_store.cc:276] 0924 14:30:14.512420 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:14.580295 (+ 67875us) service_pool.cc:227] Handling call
0924 14:30:14.580820 (+   525us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:14.581390 (+   570us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:14.581415 (+    25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:14.581947 (+   532us) log.cc:842] Serialized 26 byte log entry
0924 14:30:14.773791 (+191844us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:14.773844 (+    53us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:14.773931 (+    87us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:15.608028 (+834097us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:15.870415 (+262387us) raft_consensus.cc:1671] finished
0924 14:30:15.870434 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:15.878012 (+  7578us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1346,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":279,"spinlock_wait_cycles":106958720,"thread_start_us":178302,"threads_started":1,"wal-append.queue_time_us":794047}
I20260924 14:30:15.912588 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:15.912791 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:15.913103 29024 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.006s	sys 0.006s
I20260924 14:30:15.914150 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:15.914585 29050 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:15.915488 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:15.916802 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:15.918268 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:15.918998 28754 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.926271 26254 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:15.923527 26252 raft_consensus.cc:2362] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6. Re-sending same reply.
I20260924 14:30:15.934370 26258 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:15.936501 26250 raft_consensus.cc:2379] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:15.939337 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:15.940973 26079 catalog_manager.cc:5495] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 2, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 2 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:15.944090 26079 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 7, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 7 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:15.947032 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:15.948566 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:15.949599 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:15.973742 29133 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260924 14:30:15.974277 26253 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:15.951048 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:15.952611 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:15.976212 26257 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:15.954057 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:15.970011 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 728) took 1162 ms. Trace:
W20260924 14:30:15.925189 29024 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.007s	sys 0.007s
I20260924 14:30:15.921871 29053 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:15.878119 29108 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:15.979985 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.017s	sys 0.056s
W20260924 14:30:15.990221 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.017s	sys 0.056s
I20260924 14:30:15.869377 28895 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:15.993988 26252 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:15.995242 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:15.999783 29108 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:15.997586 26252 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:16.013396 26257 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:16.014746 26253 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:15.991137 26079 catalog_manager.cc:5495] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 5, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
W20260924 14:30:15.926251 29007 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.013s	sys 0.025s
I20260924 14:30:15.977272 26251 rpcz_store.cc:276] 0924 14:30:14.807590 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:14.818881 (+ 11291us) service_pool.cc:227] Handling call
0924 14:30:14.835860 (+ 16979us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:15.077237 (+241377us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:15.077284 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:15.092359 (+ 15075us) log.cc:842] Serialized 26 byte log entry
0924 14:30:15.537761 (+445402us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:15.537815 (+    54us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:15.539068 (+  1253us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:15.969746 (+430678us) raft_consensus.cc:1671] finished
0924 14:30:15.969767 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:15.969929 (+   162us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":66434,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":18346,"thread_start_us":589997,"threads_started":2,"wal-append.queue_time_us":658572}
W20260924 14:30:15.684077 26077 rpcz_store.cc:269] 0924 14:30:00.346096 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:00.348262 (+  2166us) service_pool.cc:227] Handling call
0924 14:30:15.659623 (+15311361us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:30:08.246496 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:30:08.247022 (+   526us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:30:08.247057 (+    35us) write_op.cc:435] Acquired schema lock
0924 14:30:08.247070 (+    13us) tablet.cc:660] Decoding operations
0924 14:30:08.645650 (+398580us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:30:08.645734 (+    84us) write_op.cc:641] Partition lock acquired for write op
0924 14:30:08.645773 (+    39us) tablet.cc:683] Acquiring locks for 15 operations
0924 14:30:08.647322 (+  1549us) tablet.cc:699] Row locks acquired
0924 14:30:08.647347 (+    25us) write_op.cc:260] PREPARE: finished
0924 14:30:08.704574 (+ 57227us) write_op.cc:270] Start()
0924 14:30:08.725522 (+ 20948us) write_op.cc:276] Timestamp: P: 1790260208698832 usec, L: 0
0924 14:30:08.739989 (+ 14467us) op_driver.cc:331] REPLICATION: starting
0924 14:30:09.076814 (+336825us) log.cc:842] Serialized 6014 byte log entry
0924 14:30:10.641412 (+1564598us) op_driver.cc:444] REPLICATION: finished
0924 14:30:11.874417 (+1233005us) write_op.cc:301] APPLY: starting
0924 14:30:11.876143 (+  1726us) tablet.cc:1365] starting BulkCheckPresence
0924 14:30:11.889717 (+ 13574us) tablet.cc:1368] finished BulkCheckPresence
0924 14:30:11.889880 (+   163us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:30:11.891363 (+  1483us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:30:11.891646 (+   283us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0924 14:30:11.891692 (+    46us) write_op.cc:312] APPLY: finished
0924 14:30:11.892040 (+   348us) log.cc:842] Serialized 106 byte log entry
0924 14:30:13.297770 (+1405730us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b64085fe 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb
0924 14:30:13.389285 (+ 91515us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:13.389447 (+   162us) write_op.cc:454] Released schema lock
0924 14:30:13.389503 (+    56us) write_op.cc:341] FINISH: Updating metrics
0924 14:30:13.389889 (+   386us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:13.389917 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":441148,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":1213496,"apply.run_cpu_time_us":24594,"apply.run_wall_time_us":1531508,"num_ops":15,"prepare.queue_time_us":1158080,"prepare.run_cpu_time_us":41500,"prepare.run_wall_time_us":1078967,"raft.queue_time_us":990782,"raft.run_cpu_time_us":103884,"raft.run_wall_time_us":487421,"replication_time_us":1899187,"spinlock_wait_cycles":29788544,"thread_start_us":3311686,"threads_started":5,"wal-append.queue_time_us":1984973,"wal-append.run_cpu_time_us":46658,"wal-append.run_wall_time_us":2623579}]]}
I20260924 14:30:16.040691 29053 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:16.019762 29050 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:16.045657 29050 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.047781 29050 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:16.060246 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 698350us
W20260924 14:30:16.021265 29007 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.013s	sys 0.026s
W20260924 14:30:16.080426 28994 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.581s	user 0.007s	sys 0.003s
I20260924 14:30:16.092588 26079 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 7, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 7 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:16.094214 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.097888 26255 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:16.098765 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:16.107468 26251 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:16.096207 26079 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 7, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 7 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:16.148082 26079 catalog_manager.cc:5495] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 5, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 5 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
W20260924 14:30:16.177927 26249 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.017s	sys 0.025s
W20260924 14:30:16.178463 26249 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.017s	sys 0.025s
I20260924 14:30:16.091728 28994 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:16.193743 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 564293us
I20260924 14:30:16.194135 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:16.184966 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.024s	sys 0.020s
W20260924 14:30:16.202512 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.024s	sys 0.020s
W20260924 14:30:16.203042 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 573292us
I20260924 14:30:16.204779 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:16.205580 26405 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:16.201834 26255 raft_consensus.cc:2362] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8. Re-sending same reply.
I20260924 14:30:16.216356 26398 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:16.217644 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.217978 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.218855 27188 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 2.
I20260924 14:30:16.217710 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.229272 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:16.231844 26371 leader_election.cc:334] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.037s (SENT)
I20260924 14:30:16.235771 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.239493 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:16.234586 29065 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.559s	user 0.014s	sys 0.000s
I20260924 14:30:16.256337 29121 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:16.261287 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.262295 26224 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:16.263329 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.267357 27182 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:16.271327 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:16.272024 26253 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:16.273993 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 252021us
I20260924 14:30:16.277117 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:16.284633 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57087.8us
I20260924 14:30:16.280779 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.287817 26398 raft_consensus.cc:2396] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:16.288733 28870 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:30:16.289479 28870 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:16.290151 29143 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260924 14:30:16.290663 28591 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:16.281322 28754 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:16.290462 28870 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:16.291196 28591 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:16.286441 29050 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:30:16.291419 28754 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:16.278806 29148 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:16.292125 29144 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:16.302627 29144 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.303702 29151 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:16.304209 29151 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.290505 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:16.296056 29050 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:16.305466 29050 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:16.306416 29151 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:16.293390 28591 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:16.295047 29036 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260924 14:30:16.309392 29151 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:16.309866 29151 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:16.310245 29151 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:16.304862 29144 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:16.310259 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:16.311499 26254 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:16.311499 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:16.312568 26256 raft_consensus.cc:2362] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8. Re-sending same reply.
I20260924 14:30:16.313445 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.313606 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.314314 27192 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:16.314651 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.316061 28994 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:16.260347 29077 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.317015 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.318132 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.318343 29077 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:16.318403 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:16.318902 26257 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:16.319128 26324 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:16.319708 26254 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:16.319823 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.320549 26257 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:16.320590 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.321316 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:16.321856 26257 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:16.302651 28754 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:16.323909 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.325119 27186 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:16.325280 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.326418 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.326972 26252 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:16.329375 26257 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:16.329938 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.330693 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.333370 29122 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:16.333880 29122 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:16.321808 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.336555 28883 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:16.337958 28883 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:16.338634 28883 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:16.335088 29122 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:16.286996 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:16.340198 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.341552 27186 raft_consensus.cc:2396] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:16.342326 29122 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:16.340304 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 53398.2us
I20260924 14:30:16.345291 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:16.338006 26371 connection.cc:232] RPC call timeout handler was delayed by -1.7675511569996161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
I20260924 14:30:16.346618 26253 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:16.340224 29121 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:16.353928 29121 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:16.355499 26403 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260924 14:30:16.356040 26403 pending_rounds.cc:76] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 4
I20260924 14:30:16.355324 29121 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:16.356324 26403 pending_rounds.cc:97] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260924 14:30:16.356947 26403 raft_consensus.cc:2866] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:30:16.361851 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 81708.2us
I20260924 14:30:16.363797 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:16.352176 26371 connection.cc:232] RPC call timeout handler was delayed by -1.6945031149989518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
I20260924 14:30:16.371069 26324 raft_consensus.cc:2396] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:16.381552 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:16.382798 26251 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:16.387665 26257 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:16.389231 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.390128 26406 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:16.395967 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:16.396286 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:16.396625 27194 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:16.396915 26398 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:16.383698 28754 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:16.400010 28754 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:16.400187 29144 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:16.400414 28754 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:16.400678 29144 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:16.400593 27125 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:16.401234 29144 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:16.402197 29150 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:16.403836 29007 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:16.360606 28883 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.407905 28883 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:16.408726 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.409452 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:16.410671 29083 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:16.411099 27186 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:16.412765 29139 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260924 14:30:16.414196 26371 connection.cc:232] RPC call timeout handler was delayed by -1.4601592270000765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2484s for a response
I20260924 14:30:16.414945 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.415755 26327 raft_consensus.cc:2379] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:16.416213 26329 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260924 14:30:16.416509 29148 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:16.417035 29148 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.412863 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:16.416440 26371 connection.cc:232] RPC call timeout handler was delayed by -1.392949803999727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260924 14:30:16.419250 26371 connection.cc:232] RPC call timeout handler was delayed by -1.2813498040004561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
I20260924 14:30:16.423301 27194 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:16.408814 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:16.427834 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.446008 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.443228 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:16.449093 26324 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:16.439308 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:16.451704 26371 connection.cc:232] RPC call timeout handler was delayed by -1.0572498040000937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
I20260924 14:30:16.454842 28896 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:16.455509 28896 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:16.455811 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:16.457034 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.457181 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.457760 28896 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:16.457952 26406 raft_consensus.cc:2396] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260924 14:30:16.458422 28896 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:16.459393 29045 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:16.459755 29045 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:16.460325 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.460634 29045 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:16.461567 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.462149 29045 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:16.462221 26331 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:30:16.463788 29150 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:16.464236 29150 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:16.464670 29150 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:16.465037 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:16.467985 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.468782 26400 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:16.411242 29083 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:16.444090 26405 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:16.447028 26327 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:16.464035 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:16.457892 26323 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:16.478055 27194 raft_consensus.cc:2354] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:30:16.463261 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.464684 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.479617 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.491294 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:16.495821 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:16.498030 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.483131 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.501695 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.506965 26397 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260924 14:30:16.507596 26397 pending_rounds.cc:76] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 5
I20260924 14:30:16.507889 26397 pending_rounds.cc:97] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260924 14:30:16.508180 26397 raft_consensus.cc:2866] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:16.509955 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.510596 26326 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:16.510654 26323 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:16.464145 29077 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:16.516842 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.517593 27194 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:16.525547 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.525888 26324 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:16.527876 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.528612 26331 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:16.530553 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.531637 26328 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:16.532131 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.535008 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.534250 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:16.535660 26323 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:16.537437 27188 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260924 14:30:16.538010 26326 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:16.538172 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.532683 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.544880 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:16.545615 26331 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:16.520848 29053 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:16.548360 26401 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:16.533948 28591 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260924 14:30:16.519590 28865 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:16.552815 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:16.553145 27182 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:16.554270 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:16.555634 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.547156 26398 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:16.564854 26406 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:16.563966 26221 connection.cc:232] RPC call timeout handler was delayed by -3.0237640110008215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260924 14:30:16.565383 26221 connection.cc:232] RPC call timeout handler was delayed by -2.9895640110007662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
W20260924 14:30:16.565716 26221 connection.cc:232] RPC call timeout handler was delayed by -2.8698640110005726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
I20260924 14:30:16.539958 28896 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:16.566497 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:16.529328 29077 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:16.619735 26221 connection.cc:232] RPC call timeout handler was delayed by -2.5170640110000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
W20260924 14:30:16.621047 26221 connection.cc:232] RPC call timeout handler was delayed by -1.9509640109999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2761s for a response
I20260924 14:30:16.621078 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.642742 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:16.640378 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 73987.5us
I20260924 14:30:16.646446 28728 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:16.646842 29077 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:16.647094 28728 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:16.647524 29077 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:16.648072 29077 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:16.648200 28728 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.579806 29148 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:16.564456 29053 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:16.651738 29053 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:16.657172 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.654753 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.658406 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:16.658701 26401 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:16.659087 26397 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:16.650125 29077 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:16.559747 28865 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.564476 29119 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:16.660360 29119 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:16.661538 28865 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:16.662262 29119 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:16.551338 29083 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:16.663054 26254 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:16.658047 26404 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:16.664235 26399 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:16.664805 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:16.665238 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.665431 26404 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:16.665658 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.666386 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.663112 29151 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.668383 29151 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:16.665766 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.669538 27188 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:16.671115 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:16.671940 28865 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:16.656128 27192 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:16.648100 27123 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:16.678617 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:16.674439 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.680673 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.679980 26258 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:16.678491 26404 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:16.681461 26401 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:16.685101 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.681656 26397 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:16.672612 28865 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:16.575052 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:16.691051 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:16.693543 26253 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:16.695103 27125 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:16.688632 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 45969.2us
I20260924 14:30:16.698246 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:16.703912 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.708985 26401 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:16.719756 26297 connection.cc:232] RPC call timeout handler was delayed by -1.210421046999727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.323s for a response
W20260924 14:30:16.720338 26297 connection.cc:232] RPC call timeout handler was delayed by -1.0952210469995407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33580000000000004s for a response
I20260924 14:30:16.682205 29065 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:16.688354 28870 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:16.698801 28865 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:16.701068 29036 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:16.688383 28994 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:16.644346 28896 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:16.706357 29007 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:16.761605 26397 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
W20260924 14:30:16.774915 26297 connection.cc:232] RPC call timeout handler was delayed by -1.0004123919998165s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
I20260924 14:30:16.768925 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.777920 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:16.742905 28870 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:16.794222 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:16.753664 29036 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:16.760280 29007 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:16.767055 28994 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:16.810745 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 319665us
W20260924 14:30:16.811981 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 105283us
I20260924 14:30:16.815212 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.817706 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:16.822465 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:16.801352 29036 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:16.800616 28870 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:16.819101 26326 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:16.805860 29121 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:16.832937 27188 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:30:16.813297 29007 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:16.849119 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:16.850873 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:16.853015 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:16.855301 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 100627us
I20260924 14:30:16.861435 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:16.868485 26371 leader_election.cc:334] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.513s (SENT)
I20260924 14:30:16.872665 29148 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:30:16.873181 29166 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260924 14:30:16.883052 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:16.858407 29179 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:16.857697 29045 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:16.891189 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:16.905256 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 77179.7us
I20260924 14:30:16.835785 29150 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.905581 26329 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:16.867177 29185 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:30:16.914114 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50918.1us
I20260924 14:30:16.902678 29121 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:16.908735 29146 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
W20260924 14:30:16.925644 29050 consensus_meta.cc:287] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.620s	user 0.007s	sys 0.004s
I20260924 14:30:16.926132 29050 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:16.926756 29150 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:16.915938 29170 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:16.950995 26331 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
W20260924 14:30:16.955425 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 71774.8us
W20260924 14:30:16.935379 29144 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.534s	user 0.007s	sys 0.003s
I20260924 14:30:16.959431 29189 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:16.959509 29146 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:16.964323 29189 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.965281 29146 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:16.966763 29189 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:16.963744 29186 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:16.971689 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.972471 27192 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:16.973272 29036 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:16.976135 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:16.977066 29121 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:16.976495 29050 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:16.977665 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:16.978463 29121 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:16.979176 28896 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:16.982726 26295 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:16.990545 29036 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:16.991264 29170 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:16.991886 29170 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:16.991328 28754 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.590s	user 0.012s	sys 0.000s
I20260924 14:30:16.995750 28754 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:16.998687 29175 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.002s
I20260924 14:30:16.967962 29186 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:16.979228 29050 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:30:16.989420 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:17.010918 26371 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.126s (SENT)
I20260924 14:30:17.016497 26398 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:17.018314 28754 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:17.018535 29186 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:16.971235 29144 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.023751 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:16.987789 29189 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:17.051033 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:17.020325 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 155232us
I20260924 14:30:17.059962 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:17.060837 26398 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260924 14:30:17.062326 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 140970us
W20260924 14:30:17.036163 29181 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.096s	user 0.002s	sys 0.000s
I20260924 14:30:17.031710 29120 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:17.036218 29179 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:17.020828 28994 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:17.061827 29189 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:17.067188 26399 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:17.071662 26401 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:17.083871 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:17.064584 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 192084us
I20260924 14:30:17.064813 26404 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 2. Current term is 3. Ops: []
W20260924 14:30:17.067214 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 160516us
I20260924 14:30:17.081476 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:17.087926 26406 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 8
I20260924 14:30:17.025278 29007 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:17.119625 29007 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:17.120409 27188 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:17.040299 28870 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:17.122637 26249 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:17.123659 28870 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:17.091322 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:17.125073 26402 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:17.091575 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:17.091847 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106252us
I20260924 14:30:17.126039 26252 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:17.128084 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.129381 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.130169 29122 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:17.130503 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:17.130774 29122 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:17.132851 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.089600 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:17.090787 29053 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:17.085800 29189 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:17.140618 28870 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260924 14:30:17.098827 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 87976us
I20260924 14:30:17.141108 28870 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:17.141480 28870 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:17.143568 29139 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:17.144918 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:17.091923 29045 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260924 14:30:17.155753 26406 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
W20260924 14:30:17.141686 26371 leader_election.cc:334] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.174s (SENT)
I20260924 14:30:17.098934 29193 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:17.099723 28896 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:17.168579 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.037681 29170 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.103579 29121 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:30:17.088901 29201 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:17.112950 29119 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:17.172292 29170 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:17.115461 29144 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:17.117220 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:17.173388 27182 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:17.091575 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:17.131337 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:17.176285 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44998.8us
I20260924 14:30:17.134487 26252 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:17.178148 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:17.164189 26371 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.484s (SENT)
I20260924 14:30:17.165434 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:17.166870 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 44191.9us
I20260924 14:30:17.167681 28867 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:17.169796 26298 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:17.168823 28896 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:17.168205 29193 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:17.113004 29120 consensus_queue.cc:1046] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:17.156781 28865 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.177008 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:17.182614 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.184439 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:17.186124 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50432.9us
I20260924 14:30:17.187633 26257 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:17.197571 29120 raft_consensus.cc:3032] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:30:17.198033 29120 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:17.200446 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.201826 26397 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:17.183414 29119 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:17.180585 29196 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:17.211758 29196 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:17.212165 29196 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:17.213488 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:17.213734 28896 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:17.213793 26257 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:17.208153 29121 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:17.207855 29193 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:17.191485 29170 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:17.216147 29193 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:17.208866 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44727.9us
I20260924 14:30:17.219969 29196 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.222223 29196 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:17.215428 28865 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:17.225041 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:17.168936 28870 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:17.226989 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.225452 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.244002 26252 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:17.245051 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.246423 29185 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:17.247138 26400 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:17.247364 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:17.204208 29122 raft_consensus.cc:2741] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260924 14:30:17.247970 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46326us
I20260924 14:30:17.243378 28895 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:17.257277 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:17.258683 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:17.258841 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 93536.5us
I20260924 14:30:17.259594 27194 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:17.260334 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.260391 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.261619 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.187732 29189 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.261183 29121 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:17.244109 28867 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:17.291271 26257 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:17.291477 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:17.297971 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:17.305320 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:17.320168 29209 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260924 14:30:17.311196 28867 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:17.322845 28867 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:17.323838 29208 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260924 14:30:17.300810 29122 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:17.337404 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.313231 29007 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:17.313709 29121 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:17.343056 27125 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:17.343811 29200 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260924 14:30:17.362239 29200 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:17.362620 29200 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:17.354969 29052 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:17.363480 29200 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:17.364434 29052 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:17.364998 29200 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:17.365655 29052 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:17.355019 29007 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:17.368548 29007 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:17.370126 26402 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:30:17.372256 29150 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:30:17.373478 29183 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:17.373912 29121 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:17.340090 29031 consensus_queue.cc:1046] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260924 14:30:17.378232 29031 raft_consensus.cc:3032] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:17.378564 29031 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:17.379369 29031 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.380501 29031 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:17.251837 29120 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:17.384207 29120 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:17.388193 29189 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:17.394572 26252 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:30:17.443159 27192 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260924 14:30:17.443629 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 111078us
W20260924 14:30:17.452495 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 190209us
I20260924 14:30:17.457106 29122 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:17.462536 26221 leader_election.cc:398] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:30:17.462944 26221 leader_election.cc:401] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260924 14:30:17.472119 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.477294 28870 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:17.480705 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.460207 29119 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:17.457800 29122 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:17.482475 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 221229us
I20260924 14:30:17.474089 29121 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:30:17.458112 29146 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260924 14:30:17.464619 29144 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260924 14:30:17.488054 28870 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:17.516296 29217 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:17.518232 29217 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:17.518648 29217 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:17.520651 29119 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:17.527952 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.530535 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:17.531220 26402 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:17.535310 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:17.537529 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:17.538337 26404 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:17.539542 26329 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260924 14:30:17.540454 26326 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260924 14:30:17.542639 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:17.543339 26328 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:17.543934 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:17.549969 26330 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:17.548280 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:17.537480 29189 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:30:17.556083 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 64925us
I20260924 14:30:17.556015 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.563098 26404 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:17.599025 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:17.603076 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 95936.2us
I20260924 14:30:17.594159 29200 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:17.605894 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 125253us
I20260924 14:30:17.597795 29218 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:17.620980 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:17.621773 27194 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:17.557029 28865 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260924 14:30:17.536640 29217 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.630904 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:17.633713 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 77796.2us
I20260924 14:30:17.635211 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:17.634086 29121 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260924 14:30:17.637496 29214 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:17.637881 29225 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:17.638643 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.638814 29225 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:17.640117 29214 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:17.640383 29217 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:17.615792 29200 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.635252 28867 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:17.661877 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:17.662357 27194 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:17.651163 29196 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:17.680706 29200 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:17.681347 29224 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:17.681800 29224 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:17.682842 29224 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:17.692389 29219 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260924 14:30:17.700547 26256 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:17.704733 29191 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260924 14:30:17.705230 29226 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
I20260924 14:30:17.706295 29191 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:17.706655 29191 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:17.707893 29191 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:17.709327 29191 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:17.716153 29203 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:17.716730 29203 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:17.717639 29203 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:17.718958 29203 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:17.725997 29218 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:30:17.732247 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 69211.7us
I20260924 14:30:17.733214 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.734408 26297 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:17.735807 29199 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:30:17.736487 29221 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260924 14:30:17.736539 29199 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:17.663545 29122 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:17.755724 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:17.767203 26400 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:17.727346 29196 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:17.770229 29196 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.780937 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:17.784186 26328 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:17.786420 29146 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:17.789179 27192 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:17.789865 27192 raft_consensus.cc:595] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:17.799640 29199 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:17.800307 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.802728 29215 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:17.802047 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.805173 29189 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:17.825670 26399 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:17.827214 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.828428 29215 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:17.828693 26328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0756687d99e643c094531124c9438376"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:17.829136 29224 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:17.830150 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.830344 29196 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:17.831055 29196 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:17.831506 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.832317 29215 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:17.832141 29196 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.833266 29215 raft_consensus.cc:2741] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260924 14:30:17.837311 29119 raft_consensus.cc:3032] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:17.830153 26328 raft_consensus.cc:595] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:17.838341 26328 raft_consensus.cc:603] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:17.837782 29119 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:17.839380 29119 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:17.840749 29119 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:17.842773 29215 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:17.848824 26297 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:17.849886 26328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:17.850939 26328 raft_consensus.cc:595] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:17.852465 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 217555us
I20260924 14:30:17.856328 29217 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:17.858786 29217 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:17.859227 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.860302 29217 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:17.861555 29119 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:17.862175 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.862862 26249 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:17.864594 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.865206 26249 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:30:17.866690 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.867430 26249 raft_consensus.cc:2354] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8.
I20260924 14:30:17.868243 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.868911 26250 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:17.870682 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.866585 26078 catalog_manager.cc:5495] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 5 to 6, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:17.871266 26250 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:17.872637 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:17.877138 26078 catalog_manager.cc:5495] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 3 to 5, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:17.880126 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:17.880776 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.859788 26328 raft_consensus.cc:603] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:17.882766 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:17.883183 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.883565 26330 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:17.883834 26256 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:17.885753 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:17.888412 26330 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:17.889070 26330 raft_consensus.cc:595] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:17.889401 26330 raft_consensus.cc:603] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:17.894017 26256 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:30:17.895573 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.901813 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.905652 26249 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:17.902448 26331 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:17.906700 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.903416 26332 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:17.907546 26255 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:17.908865 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.908941 26405 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:17.909646 26255 raft_consensus.cc:2396] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:17.910480 29215 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:17.911029 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:17.911813 26255 raft_consensus.cc:2362] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5. Re-sending same reply.
I20260924 14:30:17.903687 26330 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:30:17.913239 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:17.913285 26330 raft_consensus.cc:595] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:17.903793 26250 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:17.915553 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:30:17.916814 29151 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:17.915086 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:17.917174 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:17.914327 26252 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:17.924686 26330 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:17.918262 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:17.918640 26078 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 3 to 4, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 4 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:17.919701 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:30:17.926769 26371 leader_election.cc:398] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8.
I20260924 14:30:17.927309 26371 leader_election.cc:401] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:30:17.927872 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:17.919701 26249 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:30:17.921482 26079 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 139) took 4508 ms. Trace:
I20260924 14:30:17.881699 29217 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:17.922603 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:17.937965 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:17.939874 29031 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:17.940428 28896 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8.
I20260924 14:30:17.941571 26078 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 4 to 5, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:17.943849 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:17.944164 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.945461 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.946419 29144 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:17.946817 29144 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:17.947196 29144 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:17.934906 29151 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:17.949699 29151 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:17.950713 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:17.951781 26405 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:17.953655 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:17.955340 28896 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:17.955901 28896 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:17.956300 28896 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:17.957248 27192 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:17.957530 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.958484 29119 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:17.959192 29119 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:30:17.962445 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:17.963799 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.964704 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.965045 29119 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:17.966645 29121 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:17.967144 29121 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:17.968328 29193 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:17.968048 29121 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:17.971933 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:17.973562 26330 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:17.974624 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:17.974658 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:17.977734 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:17.963179 26331 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:17.987043 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:17.987924 26399 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:17.987123 26332 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:17.986990 26255 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:17.987372 26323 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:17.989902 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.001129 26323 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:18.004437 29119 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:18.013317 26326 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:17.986907 29224 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:18.016978 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:18.021335 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:18.022193 26329 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:18.022028 26323 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:18.010102 29119 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:18.028817 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:18.036568 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:18.037318 26255 raft_consensus.cc:2362] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8. Re-sending same reply.
I20260924 14:30:17.934065 26079 rpcz_store.cc:276] 0924 14:30:13.413243 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:13.413401 (+   158us) service_pool.cc:227] Handling call
0924 14:30:17.921403 (+4508002us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:30:16.433472 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:30:16.434152 (+   680us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:30:16.434216 (+    64us) write_op.cc:435] Acquired schema lock
0924 14:30:16.434242 (+    26us) tablet.cc:660] Decoding operations
0924 14:30:16.436195 (+  1953us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:30:16.436349 (+   154us) write_op.cc:641] Partition lock acquired for write op
0924 14:30:16.436388 (+    39us) tablet.cc:683] Acquiring locks for 18 operations
0924 14:30:16.835030 (+398642us) tablet.cc:699] Row locks acquired
0924 14:30:16.858846 (+ 23816us) write_op.cc:260] PREPARE: finished
0924 14:30:16.878949 (+ 20103us) write_op.cc:270] Start()
0924 14:30:16.879221 (+   272us) write_op.cc:276] Timestamp: P: 1790260216878817 usec, L: 0
0924 14:30:16.879277 (+    56us) op_driver.cc:331] REPLICATION: starting
0924 14:30:16.970978 (+ 91701us) log.cc:842] Serialized 7191 byte log entry
0924 14:30:17.247033 (+276055us) op_driver.cc:444] REPLICATION: finished
0924 14:30:17.473123 (+226090us) write_op.cc:301] APPLY: starting
0924 14:30:17.488350 (+ 15227us) tablet.cc:1365] starting BulkCheckPresence
0924 14:30:17.516583 (+ 28233us) tablet.cc:1368] finished BulkCheckPresence
0924 14:30:17.516707 (+   124us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:30:17.518696 (+  1989us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:30:17.519082 (+   386us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0924 14:30:17.519125 (+    43us) write_op.cc:312] APPLY: finished
0924 14:30:17.555884 (+ 36759us) log.cc:842] Serialized 124 byte log entry
0924 14:30:17.598107 (+ 42223us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:17.606603 (+  8496us) write_op.cc:454] Released schema lock
0924 14:30:17.630297 (+ 23694us) write_op.cc:341] FINISH: Updating metrics
0924 14:30:17.654975 (+ 24678us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:17.660599 (+  5624us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":923494,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":214733,"apply.run_cpu_time_us":6003,"apply.run_wall_time_us":203843,"num_ops":18,"prepare.queue_time_us":107348,"prepare.run_cpu_time_us":28531,"prepare.run_wall_time_us":734278,"raft.queue_time_us":96173,"raft.run_cpu_time_us":47775,"raft.run_wall_time_us":88293,"replication_time_us":367657,"thread_start_us":303254,"threads_started":4,"wal-append.queue_time_us":161185}]]}
I20260924 14:30:18.041828 26324 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:18.042580 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:18.043576 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:18.045603 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:18.020108 29193 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:18.052460 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:18.048801 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:18.049660 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:18.050338 26250 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:18.051067 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:18.055128 26326 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:18.055397 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:18.055822 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:18.056443 26327 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:18.058642 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:18.053367 26329 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:18.059990 26399 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:18.058987 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:18.064208 26324 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:18.065349 26332 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:30:18.068032 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 96180us
I20260924 14:30:18.068753 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.069720 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:18.014649 29233 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260924 14:30:18.047847 29217 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:18.070885 29246 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260924 14:30:18.063949 28865 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:18.067516 29213 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:18.155397 27192 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:18.083297 29193 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260924 14:30:18.157279 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 161367us
I20260924 14:30:18.085083 29151 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:18.156824 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 87142.9us
I20260924 14:30:18.161185 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:18.161808 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 103232us
I20260924 14:30:18.168783 29036 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:18.168598 29139 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260924 14:30:18.071157 29150 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260924 14:30:18.153534 28865 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:18.160811 29193 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:18.173738 26325 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:18.180028 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:18.095827 29119 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:18.192564 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:18.178558 28865 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:18.177340 29251 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:30:18.164654 29151 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:18.181407 29215 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:18.166842 29203 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260924 14:30:18.183539 29193 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:18.201748 28883 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:30:18.206136 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:18.215538 26325 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:18.220079 26327 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:18.184571 29185 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:18.169462 29036 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:18.184526 29065 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:18.250375 26255 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:18.213493 29186 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:18.244611 29144 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:18.299645 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 100531us
I20260924 14:30:18.266577 29065 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:18.309638 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:18.322016 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 160901us
I20260924 14:30:18.323253 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:18.316610 29052 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260924 14:30:18.332019 29244 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
W20260924 14:30:18.331432 29242 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260924 14:30:18.351082 26327 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260924 14:30:18.355213 29248 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260924 14:30:18.379029 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 72320.4us
I20260924 14:30:18.380352 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:18.423651 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 100489us
I20260924 14:30:18.426388 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.361928 29150 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
I20260924 14:30:18.362705 29224 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260924 14:30:18.414664 29260 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:18.421687 29036 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:18.434998 29250 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.159s	user 0.001s	sys 0.001s
I20260924 14:30:18.422739 29151 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:18.447785 29254 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260924 14:30:18.433226 29262 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:18.456682 29256 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:18.441946 28896 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:18.461045 29256 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:18.461462 29256 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:18.462520 28896 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:18.463100 28896 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:18.442672 29265 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260924 14:30:18.465560 29252 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260924 14:30:18.466043 29262 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:18.468822 29262 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:18.469412 29262 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:18.478847 26258 raft_consensus.cc:1203] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Deduplicated request from leader. Original: 6.5->[8.6-8.6]   Dedup: 8.6->[]
I20260924 14:30:18.479538 26258 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260924 14:30:18.507266 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 126979us
I20260924 14:30:18.454909 29144 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:18.454514 29065 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:18.509770 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.516386 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:18.509410 29258 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260924 14:30:18.526692 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:18.536325 29267 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260924 14:30:18.486639 29257 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260924 14:30:18.540971 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.541710 29271 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:18.541970 29268 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:18.542052 29272 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:18.542435 29186 raft_consensus.cc:979] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:18.542654 29272 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:18.542954 29268 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:18.545609 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:18.524999 29144 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260924 14:30:18.549510 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 337481us
I20260924 14:30:18.552524 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:18.542964 29186 raft_consensus.cc:1067] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:18.555698 26249 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260924 14:30:18.556032 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:18.556213 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:18.556143 26249 pending_rounds.cc:76] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 3
I20260924 14:30:18.557468 26249 pending_rounds.cc:97] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260924 14:30:18.557914 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:18.558053 26249 raft_consensus.cc:2866] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:18.560266 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:18.561017 26249 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 3: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:18.546368 29257 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:30:18.562212 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:18.566349 26327 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:18.557224 26405 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:18.568011 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:18.570185 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 143921us
I20260924 14:30:18.571406 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:18.505479 29193 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:18.550091 29271 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:18.581440 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:18.582079 26256 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:18.599193 26331 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260924 14:30:18.603040 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 287860us
I20260924 14:30:18.550979 28865 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:18.582572 29144 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:18.616964 27192 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:18.594187 29274 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:18.606729 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.624810 27186 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:30:18.596321 29271 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:18.580376 29199 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260924 14:30:18.638816 26254 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:49618
I20260924 14:30:18.645999 26254 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:18.646405 26254 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:18.623353 29144 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:18.650281 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51468.8us
I20260924 14:30:18.654995 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:18.546221 29272 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:18.660816 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.666560 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:18.668044 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 69314.9us
I20260924 14:30:18.637820 29119 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260924 14:30:18.658195 29144 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:18.674778 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.678172 27182 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:18.691776 29280 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:18.608446 29268 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:18.676231 28896 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:18.692728 29280 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:18.663389 29276 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:30:18.712739 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52062.8us
I20260924 14:30:18.718137 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:18.719189 29184 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260924 14:30:18.696980 29213 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
W20260924 14:30:18.732990 29253 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.458s	user 0.003s	sys 0.000s
I20260924 14:30:18.736248 27182 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260924 14:30:18.738022 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 63361.7us
I20260924 14:30:18.738281 29191 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:30:18.739045 29191 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:18.707852 29193 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:18.708995 29271 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:18.710384 29270 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260924 14:30:18.701249 29225 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:18.669876 29203 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260924 14:30:18.662909 29257 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260924 14:30:18.712664 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:18.739105 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:18.762315 26406 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:18.705648 29256 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:18.749461 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 68811.7us
I20260924 14:30:18.753273 26254 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:18.754062 26252 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:18.764555 27126 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:18.764891 26405 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:18.766139 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:18.768069 26406 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:18.769071 26406 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:18.772127 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:18.776520 26402 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:18.779029 26254 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:18.780465 27192 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:18.783546 26398 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:30:18.784371 29213 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:18.785914 29213 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:18.786332 29213 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:18.786576 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:18.787861 28865 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:18.789454 26254 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:18.789594 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:18.790205 29285 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:18.790867 29280 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:18.787359 29213 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:18.793629 29213 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:18.749604 29274 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:18.799654 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.802299 26327 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
W20260924 14:30:18.814172 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 49695.4us
W20260924 14:30:18.814546 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 75558.1us
I20260924 14:30:18.815943 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.818835 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:18.774966 29139 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:18.771550 28896 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:18.829131 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:18.829372 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:18.750093 29186 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260924 14:30:18.832463 27192 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:18.835790 26327 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:18.851917 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:18.855525 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:18.816715 29274 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260924 14:30:18.884382 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 59643.3us
I20260924 14:30:18.886473 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:18.887312 26331 raft_consensus.cc:2362] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12. Re-sending same reply.
I20260924 14:30:18.888729 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:18.830116 29193 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:18.761519 29225 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:18.891672 26402 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:18.777468 29144 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:18.833539 29184 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:18.827872 29065 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:18.895941 27125 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:18.833781 28895 consensus_queue.cc:1046] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260924 14:30:18.886412 27192 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:18.924448 29289 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260924 14:30:18.925518 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:18.926149 29295 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:30:18.927770 29288 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:18.819947 29224 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260924 14:30:18.836563 29189 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260924 14:30:18.933586 26252 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:18.877818 28865 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260924 14:30:18.928628 29065 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:18.837919 29191 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:18.955407 29065 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:18.917389 29184 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:18.958226 29184 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:18.833881 29139 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:18.960630 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 64798.9us
I20260924 14:30:18.986538 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:19.019165 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 119912us
I20260924 14:30:19.035876 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:19.039710 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:19.049700 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:19.025156 29256 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:19.043007 29289 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:18.965799 28896 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:18.941632 29288 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:19.103482 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 130718us
W20260924 14:30:19.105217 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 51812.7us
I20260924 14:30:19.046366 29295 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:19.117583 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:19.104311 29281 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260924 14:30:19.051061 29189 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260924 14:30:19.135390 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:19.167794 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:19.168753 26398 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:19.123728 29224 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:19.096033 29282 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.276s
I20260924 14:30:19.175733 26298 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:19.193133 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:19.099198 29289 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:19.199663 29288 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:19.171162 29225 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:19.137593 29144 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:19.198319 29139 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:19.199816 29245 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.038s	user 0.005s	sys 0.000s
I20260924 14:30:19.195614 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:19.194872 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:19.259791 26325 raft_consensus.cc:3032] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:19.260138 26325 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:19.282145 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 161878us
I20260924 14:30:19.260106 26331 raft_consensus.cc:2396] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:19.195547 29186 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:19.130010 29277 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.272s
I20260924 14:30:19.343246 29274 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:19.344365 29304 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260924 14:30:19.329389 26325 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:19.344379 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:19.353453 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
W20260924 14:30:19.355538 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 183714us
I20260924 14:30:19.361527 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:19.369727 26398 raft_consensus.cc:2396] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:19.358011 27188 raft_consensus.cc:595] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:19.376013 26325 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:19.382607 29225 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:19.367609 29274 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:19.401849 29189 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:19.404256 29318 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260924 14:30:19.418159 27192 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:19.418831 27192 raft_consensus.cc:595] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:19.421245 29300 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:19.421859 29300 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:19.423168 29294 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.404s	user 0.001s	sys 0.000s
I20260924 14:30:19.415205 29225 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:19.425104 29301 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260924 14:30:19.425753 29225 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:19.426088 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 752) took 1060 ms. Trace:
I20260924 14:30:19.426227 29309 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:19.426756 29309 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:19.426527 26397 rpcz_store.cc:276] 0924 14:30:18.366059 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.368059 (+  2000us) service_pool.cc:227] Handling call
0924 14:30:18.368548 (+   489us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:18.430339 (+ 61791us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:18.430385 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:18.709055 (+278670us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b2c0013ed68. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:18.709528 (+   473us) log.cc:842] Serialized 26 byte log entry
0924 14:30:18.877110 (+167582us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:18.877154 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:18.877236 (+    82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:19.422462 (+545226us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:19.425878 (+  3416us) raft_consensus.cc:1671] finished
0924 14:30:19.425900 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:19.426027 (+   127us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":85229,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":6704,"spinlock_wait_cycles":230235648,"thread_start_us":228439,"threads_started":2,"wal-append.queue_time_us":239309}
I20260924 14:30:19.428735 29309 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:19.430032 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 234850us
W20260924 14:30:19.423990 29306 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260924 14:30:19.432945 26401 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:19.435417 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:19.436074 27188 raft_consensus.cc:595] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:19.440589 26325 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:19.445539 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:19.387672 29224 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:19.450487 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:19.433741 29189 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:19.448253 29277 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:19.468185 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:19.469370 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 334271us
I20260924 14:30:19.451247 29225 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:19.476573 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:19.478948 27188 raft_consensus.cc:595] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:19.482676 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:19.430092 29065 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:19.490147 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:19.463709 29268 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:19.490885 26403 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
W20260924 14:30:19.495962 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 329007us
I20260924 14:30:19.504341 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:19.505247 26249 raft_consensus.cc:2396] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260924 14:30:19.486702 29251 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.206s	user 0.007s	sys 0.015s
I20260924 14:30:19.505934 29300 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:19.506395 29321 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:19.494134 29302 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:19.503893 29320 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:19.514266 29251 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.235s	user 0.008s	sys 0.016s
I20260924 14:30:19.531013 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:19.482606 29225 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:19.477697 29277 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:19.517911 29289 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:19.547372 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:19.501257 29268 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:19.532625 29251 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:19.557389 27191 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260924 14:30:19.558952 29300 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:19.559597 29300 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:19.561512 29300 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:19.507735 29224 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:19.561990 29300 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260924 14:30:19.576612 29213 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.782s	user 0.014s	sys 0.000s
I20260924 14:30:19.581298 29144 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:19.582692 29274 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:19.589915 29300 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:19.457576 29216 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.133s
I20260924 14:30:19.591653 29189 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:19.605365 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:19.608387 29300 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:19.615921 29225 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:19.601223 29065 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:19.605577 29184 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.647s	user 0.012s	sys 0.004s
I20260924 14:30:19.524436 29321 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:19.608130 29322 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260924 14:30:19.533402 29320 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:19.639586 29330 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260924 14:30:19.628508 29189 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:19.641270 29189 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:19.647902 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:19.633950 29184 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:19.651079 27191 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:19.651774 29184 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:19.655123 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:19.606248 29144 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:19.619232 29251 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:19.660055 29274 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260924 14:30:19.668435 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1176) took 1638 ms. Trace:
I20260924 14:30:19.668870 26258 rpcz_store.cc:276] 0924 14:30:18.029708 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.029832 (+   124us) service_pool.cc:227] Handling call
0924 14:30:18.478724 (+448892us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:18.485489 (+  6765us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:19.668221 (+1182732us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54000c1cd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:19.668371 (+   150us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1031655808}
I20260924 14:30:19.670368 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:19.670864 29189 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:19.705138 29268 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:19.704631 29225 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:19.703992 29338 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:19.722290 29277 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:19.733321 29338 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:19.744220 29320 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:19.745744 29338 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:19.747786 28867 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260924 14:30:19.728739 29225 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:19.750876 29225 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:19.771040 27190 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260924 14:30:19.767480 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 267359us
I20260924 14:30:19.774748 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:19.782392 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:19.783869 29286 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.001s	sys 0.016s
I20260924 14:30:19.769155 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:19.753221 29302 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:19.752285 29144 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:19.753115 29321 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:19.788923 29332 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260924 14:30:19.788844 29348 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:19.797287 27125 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:19.789526 29286 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.001s	sys 0.016s
I20260924 14:30:19.795699 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:19.813301 26323 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:19.814215 29335 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:19.825999 28883 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:19.826265 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:19.826930 28883 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:19.829223 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:19.829514 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:19.856490 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:19.829486 28883 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:19.812749 29144 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260924 14:30:19.868316 26398 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:19.882546 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46447.4us
I20260924 14:30:19.876514 29260 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:19.897776 29298 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.802s	user 0.004s	sys 0.000s
I20260924 14:30:19.899320 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1190) took 1132 ms. Trace:
I20260924 14:30:19.899976 29329 raft_consensus.cc:979] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:19.900724 29329 raft_consensus.cc:1067] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:19.899857 26256 rpcz_store.cc:276] 0924 14:30:18.767248 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.787709 (+ 20461us) service_pool.cc:227] Handling call
0924 14:30:18.788049 (+   340us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:18.798846 (+ 10797us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:18.798876 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:18.799254 (+   378us) log.cc:842] Serialized 26 byte log entry
0924 14:30:18.917788 (+118534us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:18.917829 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:18.917908 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:19.897151 (+979243us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:19.898885 (+  1734us) raft_consensus.cc:1671] finished
0924 14:30:19.898921 (+    36us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:19.899240 (+   319us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10384,"raft.queue_time_us":14794,"spinlock_wait_cycles":45253248,"thread_start_us":116565,"threads_started":1,"wal-append.queue_time_us":218014}
I20260924 14:30:19.902974 28883 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260924 14:30:19.904759 29217 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:19.905670 29217 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:19.906020 29217 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:19.908957 29278 raft_consensus.cc:979] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:19.909499 29278 raft_consensus.cc:1067] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:19.918411 26256 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
I20260924 14:30:19.919041 26256 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:19.919373 26256 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:19.923346 26399 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:35444
I20260924 14:30:19.924021 26399 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:19.924433 26399 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:19.924881 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:19.906877 29217 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:19.941720 29266 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:30:19.941754 28883 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:19.959050 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1184) took 1640 ms. Trace:
I20260924 14:30:19.959198 26256 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:19.967170 26255 rpcz_store.cc:276] 0924 14:30:18.319017 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.319352 (+   335us) service_pool.cc:227] Handling call
0924 14:30:18.319729 (+   377us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:18.322745 (+  3016us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:18.322785 (+    40us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:30:19.649797 (+1327012us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b2c001c3658. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:19.753617 (+103820us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b2c001c3658. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:19.789618 (+ 36001us) log.cc:842] Serialized 52 byte log entry
0924 14:30:19.880028 (+ 90410us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:19.880074 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:19.880219 (+   145us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:19.958832 (+ 78613us) raft_consensus.cc:1671] finished
0924 14:30:19.958852 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:19.958991 (+   139us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3357,"raft.queue_time_us":175,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":7031,"spinlock_wait_cycles":12822656,"thread_start_us":82474,"threads_started":1,"wal-append.queue_time_us":146768}
I20260924 14:30:19.983192 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:20.000547 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 469638us
I20260924 14:30:20.005151 26256 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:20.006891 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:19.980217 26399 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:20.009910 26399 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:20.011206 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 221985us
I20260924 14:30:20.014930 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.016975 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:19.923872 29260 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.019193 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.019815 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.020985 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:20.021555 27181 raft_consensus.cc:3032] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:20.021826 27181 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:20.015532 29321 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.024304 27181 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:20.024561 29321 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.025743 29272 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:20.005915 29333 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:20.028311 29333 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:20.029384 26400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 753) took 1481 ms (client timeout 1889 ms). Trace:
W20260924 14:30:20.029906 26400 rpcz_store.cc:269] 0924 14:30:18.548403 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.548562 (+   159us) service_pool.cc:227] Handling call
0924 14:30:20.029166 (+1480604us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54001e41d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:20.029321 (+   155us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2839414528}
I20260924 14:30:20.007666 29052 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:20.031510 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1183) took 1791 ms. Trace:
I20260924 14:30:20.031913 26257 rpcz_store.cc:276] 0924 14:30:18.240576 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.240780 (+   204us) service_pool.cc:227] Handling call
0924 14:30:18.270832 (+ 30052us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:18.271278 (+   446us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:18.271317 (+    39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:18.284044 (+ 12727us) log.cc:842] Serialized 26 byte log entry
0924 14:30:18.425332 (+141288us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:18.436513 (+ 11181us) log.cc:842] Serialized 14 byte log entry
0924 14:30:18.436731 (+   218us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:18.436824 (+    93us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:19.819739 (+1382915us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b180036e0e0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:30:20.015371 (+195632us) raft_consensus.cc:1671] finished
0924 14:30:20.031257 (+ 15886us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:20.031405 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":10954,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":3432,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":14630,"spinlock_wait_cycles":11516672,"thread_start_us":138985,"threads_started":1,"wal-append.queue_time_us":167134}
I20260924 14:30:20.033531 29052 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.034067 29052 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:20.036510 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.039098 29317 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.039469 29344 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.039772 29317 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.040086 29344 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.041850 29317 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:20.042215 29344 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:20.042337 29317 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:20.043010 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:20.006359 29330 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260924 14:30:20.019191 26256 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:30:20.019910 27185 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:20.031067 29333 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:20.043845 26252 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:20.056602 27126 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:20.057703 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:20.058652 26252 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:20.006491 29224 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.049371 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:20.060169 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:20.060606 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.026958 29321 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:20.053402 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:20.060765 27192 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:30:20.046336 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:20.061513 27192 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:20.061574 26323 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:30:20.061947 26399 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:20.067418 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:20.063834 27192 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:20.070376 29337 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:20.071213 29304 consensus_queue.cc:1046] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260924 14:30:20.073982 29297 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.038s	user 0.002s	sys 0.000s
I20260924 14:30:20.076874 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:20.077301 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:20.102195 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 782) took 1355 ms. Trace:
I20260924 14:30:20.042851 29300 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:20.104041 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:20.104408 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.105417 26406 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:20.102818 27186 rpcz_store.cc:276] 0924 14:30:18.746633 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.748158 (+  1525us) service_pool.cc:227] Handling call
0924 14:30:18.748456 (+   298us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:18.748799 (+   343us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:18.748826 (+    27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:18.749298 (+   472us) log.cc:842] Serialized 26 byte log entry
0924 14:30:18.834806 (+ 85508us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:18.834841 (+    35us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:18.834906 (+    65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:20.073089 (+1238183us) spinlock_profiling.cc:243] Waited 66.8 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:20.076596 (+  3507us) raft_consensus.cc:1671] finished
0924 14:30:20.076624 (+    28us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:20.102101 (+ 25477us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":116479,"raft.queue_time_us":14326,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":12053,"spinlock_wait_cycles":153610496,"thread_start_us":33559,"threads_started":1,"wal-append.queue_time_us":208317}
I20260924 14:30:20.125882 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:20.129267 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:20.129572 29196 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.131112 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:20.112769 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:20.137531 29344 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.143312 29344 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.142508 29284 raft_consensus.cc:979] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:20.042873 29317 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.144785 29284 raft_consensus.cc:1067] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:20.145659 29344 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:20.146135 29317 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:20.146977 26249 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
I20260924 14:30:20.142661 29196 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.147578 26249 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:20.147975 26249 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:20.150286 29196 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:20.150642 26298 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 2.105s (SENT) [suppressed 17 similar messages]
I20260924 14:30:20.142704 29300 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:20.152293 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.154562 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.154986 26406 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:30:20.157541 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:20.158218 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1141) took 1440 ms. Trace:
I20260924 14:30:20.158872 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:20.159260 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 489) took 1305 ms. Trace:
I20260924 14:30:20.160542 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:20.159807 26327 rpcz_store.cc:276] 0924 14:30:18.853760 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.857252 (+  3492us) service_pool.cc:227] Handling call
0924 14:30:18.887372 (+ 30120us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:18.897985 (+ 10613us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:18.898031 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:18.917354 (+ 19323us) log.cc:842] Serialized 26 byte log entry
0924 14:30:19.027610 (+110256us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:19.280129 (+252519us) log.cc:842] Serialized 14 byte log entry
0924 14:30:19.348552 (+ 68423us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:19.348655 (+   103us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:20.155916 (+807261us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b1800263920. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:30:20.158946 (+  3030us) raft_consensus.cc:1671] finished
0924 14:30:20.159059 (+   113us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:20.159182 (+   123us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":120358,"prepare.queue_time_us":66590,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":4819,"raft.queue_time_us":46886,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":22695,"spinlock_wait_cycles":26214784,"thread_start_us":283660,"threads_started":2,"wal-append.queue_time_us":180841}
I20260924 14:30:20.158587 26404 rpcz_store.cc:276] 0924 14:30:18.717561 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.717698 (+   137us) service_pool.cc:227] Handling call
0924 14:30:20.158046 (+1440348us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b5400336654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:20.158162 (+   116us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1776947456}
I20260924 14:30:20.162478 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:20.165380 26252 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:20.162308 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:20.166273 26403 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:20.168677 26404 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:30:20.171698 27181 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:20.173980 26403 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:20.177948 29320 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.178522 29320 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:20.141497 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 74170.6us
I20260924 14:30:20.180528 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:20.181494 29065 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:20.181849 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:20.182994 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:20.185642 29321 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:20.188057 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1154) took 1424 ms. Trace:
I20260924 14:30:20.188530 26402 rpcz_store.cc:276] 0924 14:30:18.763388 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.777310 (+ 13922us) service_pool.cc:227] Handling call
0924 14:30:18.813754 (+ 36444us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:18.904925 (+ 91171us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:20.187827 (+1282902us) spinlock_profiling.cc:243] Waited 77.6 ms on lock 0x7b540036bad4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:20.188001 (+   174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":178478848}
I20260924 14:30:20.063910 29260 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:20.059963 29224 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.145761 26255 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:30:20.197297 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:20.171819 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 772) took 1598 ms. Trace:
I20260924 14:30:20.144305 28895 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:20.205827 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:20.141973 29233 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:20.212761 29233 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:20.213287 29233 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:20.217523 29224 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:20.218228 29290 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.218812 29290 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:20.220970 29290 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:20.204617 29317 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:20.223577 29317 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.223970 29317 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:20.228466 27181 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:20.229067 26252 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:20.235811 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:20.201951 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:20.245433 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 43607.6us
I20260924 14:30:20.246804 26224 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:20.250927 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.200171 27194 rpcz_store.cc:276] 0924 14:30:18.573573 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.573745 (+   172us) service_pool.cc:227] Handling call
0924 14:30:18.574083 (+   338us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:18.665951 (+ 91868us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:18.665984 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:19.917649 (+1251665us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b2c001c06e8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:19.918064 (+   415us) log.cc:842] Serialized 26 byte log entry
0924 14:30:19.985590 (+ 67526us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:19.985642 (+    52us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:19.985786 (+   144us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:20.171493 (+185707us) raft_consensus.cc:1671] finished
0924 14:30:20.171513 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:20.171758 (+   245us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":83735,"raft.queue_time_us":77590,"raft.run_cpu_time_us":706,"raft.run_wall_time_us":10568,"spinlock_wait_cycles":5591808,"thread_start_us":127760,"threads_started":2,"wal-append.queue_time_us":248884}
I20260924 14:30:20.265255 26399 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:30:20.245353 26249 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.270707 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:20.271713 27188 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:20.275028 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.208520 28895 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:20.294292 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:20.292995 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:20.296365 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:20.296233 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:20.296725 26224 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:20.328522 29196 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:20.346151 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 195525us
I20260924 14:30:20.273356 29274 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:20.297214 26250 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:20.299687 29233 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.301144 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:20.364041 27181 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:20.364454 27181 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:20.365353 27181 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.366627 27181 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:20.276337 29224 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.272361 29300 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:20.385391 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:20.386010 26331 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:20.279487 29144 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.305851 27186 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:20.312597 29320 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:20.318270 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:20.400022 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 81831.8us
I20260924 14:30:20.261850 29260 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:20.293185 26249 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:20.320869 27192 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:20.323668 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1142) took 1605 ms. Trace:
I20260924 14:30:20.213327 29331 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.336647 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:20.354530 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:20.414342 26323 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:20.411926 26405 rpcz_store.cc:276] 0924 14:30:18.717874 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.745944 (+ 28070us) service_pool.cc:227] Handling call
0924 14:30:18.747015 (+  1071us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:18.765302 (+ 18287us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:20.323352 (+1558050us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b5400335754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:20.323601 (+   249us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":40704896}
W20260924 14:30:20.403715 29356 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260924 14:30:20.358711 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:20.352887 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 56255.7us
I20260924 14:30:20.420845 26224 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:20.350538 29196 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.303807 29344 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:20.327935 29185 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.372262 27189 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:20.321838 29337 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260924 14:30:20.387562 26327 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:20.404017 29144 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:20.437103 29337 raft_consensus.cc:979] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:20.437660 29337 raft_consensus.cc:1067] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:20.437855 29317 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.439406 29119 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:20.439934 29317 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:20.441926 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:20.443192 26401 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:20.445443 29144 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:20.446527 29311 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.188s	user 0.004s	sys 0.000s
I20260924 14:30:20.416325 29275 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260924 14:30:20.448873 29275 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:20.449252 29275 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:20.410679 29212 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.452160 29052 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.453423 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.455523 29309 raft_consensus.cc:979] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:20.456202 29309 raft_consensus.cc:1067] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:20.458765 26255 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:30:20.345543 28895 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.460068 26258 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260924 14:30:20.460219 26398 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:35444
I20260924 14:30:20.461004 26398 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:20.461115 29233 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:20.461414 26398 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:20.461745 29351 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:20.463037 29196 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:20.465219 29353 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:20.465525 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 783) took 1568 ms. Trace:
I20260924 14:30:20.466486 27185 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:30:20.466006 27182 rpcz_store.cc:276] 0924 14:30:18.896896 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.897136 (+   240us) service_pool.cc:227] Handling call
0924 14:30:18.897492 (+   356us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:18.949505 (+ 52013us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:18.949560 (+    55us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:30:18.960528 (+ 10968us) log.cc:842] Serialized 52 byte log entry
0924 14:30:19.043505 (+ 82977us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:19.368613 (+325108us) log.cc:842] Serialized 14 byte log entry
0924 14:30:19.373426 (+  4813us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:19.373533 (+   107us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:20.445723 (+1072190us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:20.465179 (+ 19456us) raft_consensus.cc:1671] finished
0924 14:30:20.465307 (+   128us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:20.465459 (+   152us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":64776,"prepare.queue_time_us":358756,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":59726,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":18423,"spinlock_wait_cycles":51255552,"thread_start_us":341304,"threads_started":2,"wal-append.queue_time_us":161985}
I20260924 14:30:20.469656 27194 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:30:20.473656 26255 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
I20260924 14:30:20.474315 26255 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:20.474689 26255 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:20.469842 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:20.405598 29260 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.398082 27186 raft_consensus.cc:595] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
W20260924 14:30:20.362267 29280 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.558s	user 0.002s	sys 0.019s
I20260924 14:30:20.292762 29295 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.351557 29348 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:20.408944 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 43855.3us
I20260924 14:30:20.523672 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:20.365453 29290 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:30:20.276010 29321 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:20.384255 29300 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.396479 29274 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:20.422273 26224 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:20.367762 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:20.547063 26328 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:20.465708 29353 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:20.519556 29196 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.450187 29275 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.514993 26398 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:20.398504 29224 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.525007 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:20.467700 29272 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
W20260924 14:30:20.518712 29280 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.724s	user 0.002s	sys 0.019s
I20260924 14:30:20.429021 29286 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:20.451612 29212 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.511606 29119 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:20.418668 29331 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.549520 27185 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
W20260924 14:30:20.549974 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 127783us
I20260924 14:30:20.638731 26224 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:20.397218 27181 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:20.508023 27192 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:30:20.556955 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:20.640341 26330 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:20.519876 29260 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:20.513283 27186 raft_consensus.cc:603] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:20.574352 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:20.581188 26328 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:20.602973 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:20.645579 26325 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:20.498440 29351 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260924 14:30:20.532209 29295 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:20.621480 26398 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:20.626417 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.545997 29300 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:20.464488 29185 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.556425 29317 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:20.548149 29290 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:20.535522 26255 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.634433 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:20.555980 29292 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.566223 29139 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:20.625221 29311 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260924 14:30:20.654156 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 773) took 1183 ms. Trace:
W20260924 14:30:20.605818 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 80929.9us
I20260924 14:30:20.534520 29348 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:20.597339 29196 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.631093 29275 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:20.635526 29119 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:20.654469 29233 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:20.658439 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:20.627295 29344 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:20.664158 29286 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:20.692747 27194 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:20.741503 27194 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:20.703925 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 51870.4us
I20260924 14:30:20.706308 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:30:20.709259 26401 rpcz_store.cc:276] 0924 14:30:19.470402 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:19.470610 (+   208us) service_pool.cc:227] Handling call
0924 14:30:20.653948 (+1183338us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54006e78d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:20.654091 (+   143us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2207364864}
W20260924 14:30:20.721100 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 167705us
I20260924 14:30:20.700107 26255 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:20.729022 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:20.752643 27186 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:30:20.753041 27186 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:20.737550 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 79219.1us
I20260924 14:30:20.759323 29347 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:20.760071 29347 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.762015 26325 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
I20260924 14:30:20.762784 29347 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:20.738626 29119 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:30:20.683235 29361 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260924 14:30:20.700227 29292 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:20.771833 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 861) took 1271 ms. Trace:
I20260924 14:30:20.772396 26254 rpcz_store.cc:276] 0924 14:30:19.500747 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:19.509430 (+  8683us) service_pool.cc:227] Handling call
0924 14:30:19.509837 (+   407us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:20.016160 (+506323us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:20.016193 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:20.077622 (+ 61429us) log.cc:842] Serialized 26 byte log entry
0924 14:30:20.350077 (+272455us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:20.350111 (+    34us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:20.350179 (+    68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:20.771540 (+421361us) raft_consensus.cc:1671] finished
0924 14:30:20.771573 (+    33us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:20.771747 (+   174us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":48362,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"thread_start_us":688756,"threads_started":2,"wal-append.queue_time_us":294573}
I20260924 14:30:20.776739 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:20.715265 29274 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:20.784072 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:20.785181 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:20.787357 29292 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:20.675377 27188 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:20.753973 27186 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:20.700955 29363 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260924 14:30:20.701418 29359 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260924 14:30:20.742440 27194 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:20.709158 29217 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:20.827553 27186 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 5
I20260924 14:30:20.742699 29213 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:20.729882 29272 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:20.693095 29052 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:20.742969 29344 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:20.701344 29185 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:20.749686 29317 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:20.792157 29256 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:20.862780 27194 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:20.865857 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 166213us
I20260924 14:30:20.697440 29144 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.151s
I20260924 14:30:20.904517 28895 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:20.712510 29333 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.152s
I20260924 14:30:20.702445 29139 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:20.864359 29344 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:20.932638 29295 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:20.938320 27186 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:20.702100 29290 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:20.893244 29317 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:20.862646 29277 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
W20260924 14:30:20.951516 29369 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260924 14:30:20.953361 29364 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260924 14:30:20.934361 29276 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:20.876086 29272 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.960544 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:20.894438 29256 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:20.963912 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:20.967463 26255 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:20.966074 29347 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:20.971912 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:20.976254 27126 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:20.980911 29321 raft_consensus.cc:2741] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260924 14:30:20.986327 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:20.990717 26255 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
W20260924 14:30:20.985208 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 192515us
W20260924 14:30:20.995016 29362 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260924 14:30:20.996745 29333 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:20.997385 29333 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:20.998898 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 149327us
I20260924 14:30:21.000033 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:20.973518 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 149814us
I20260924 14:30:20.975049 27194 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:21.003228 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:20.974737 29212 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:21.007944 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:20.984896 29373 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:21.017788 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:21.012209 29372 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260924 14:30:21.016098 29288 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:20.965006 29276 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:21.013022 29370 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260924 14:30:21.004668 29196 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:20.993467 29375 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:21.064990 29375 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:21.066809 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:30:21.067741 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:21.006078 29317 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:21.069590 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:20.992834 29347 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:21.019877 29321 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:21.071732 29321 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:21.072849 29371 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.255s	user 0.000s	sys 0.000s
I20260924 14:30:21.073843 29321 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:21.074369 29321 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:21.074810 29321 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:21.021837 29271 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:21.057967 26330 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:21.019907 28867 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:21.060056 29300 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:21.060081 29333 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:21.022512 29331 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:21.067925 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 51175.8us
I20260924 14:30:21.099435 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.076081 29144 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:21.017854 29373 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:21.055786 29146 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:21.102955 29146 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:21.103219 29373 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:21.105182 29146 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:21.023258 29288 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:21.105834 29288 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:21.106786 29144 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:21.107798 29144 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:21.080313 29318 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:21.071074 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:21.112452 29276 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:21.071334 29347 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:21.081681 29119 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:21.082738 29272 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:21.082862 29274 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:21.085023 29365 log.cc:924] Time spent T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Append to log took a long time: real 0.271s	user 0.000s	sys 0.003s
I20260924 14:30:21.084640 29260 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:21.085176 29256 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:21.086323 29139 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:21.024480 29189 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:21.067126 29375 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:21.151786 29224 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:21.140973 29274 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:21.152874 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.153667 26332 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:21.154256 29274 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:21.156495 29260 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:21.159175 26399 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:21.023133 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:21.090387 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:21.166150 26325 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:21.092988 29185 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:21.100835 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.171913 26328 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:21.105216 29353 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:21.143903 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:21.165598 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 142537us
I20260924 14:30:21.117532 29373 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:21.117810 29331 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:21.136310 29383 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:21.191058 29256 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:21.191568 29383 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:21.191757 29256 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:21.193953 29383 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:21.194669 29256 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:21.195568 29119 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:21.196867 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.198531 29321 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:21.201884 29366 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260924 14:30:21.204628 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:21.205614 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:21.208276 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:21.209367 26397 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:21.210935 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:21.214561 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:21.215708 26324 raft_consensus.cc:2396] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:21.141489 28895 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:21.218524 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:21.233011 26325 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:21.237501 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:21.235491 29189 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:21.241344 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:21.256762 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 384495us
I20260924 14:30:21.253867 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:21.254509 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:21.260766 26326 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:21.256122 26332 raft_consensus.cc:2379] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:21.263413 26325 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 3. Current term is 5. Ops: []
I20260924 14:30:21.265344 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.235880 28867 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:21.272377 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 846) took 2519 ms. Trace:
I20260924 14:30:21.270527 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.188920 29185 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:21.288102 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:21.289263 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:21.293727 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:21.305162 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:21.296047 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 288175us
I20260924 14:30:21.296882 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:21.264170 29368 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:21.300064 26251 rpcz_store.cc:276] 0924 14:30:18.745667 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.746827 (+  1160us) service_pool.cc:227] Handling call
0924 14:30:20.317337 (+1570510us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:30:20.317513 (+   176us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:20.497203 (+179690us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:20.497236 (+    33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:20.497276 (+    40us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:20.497309 (+    33us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:21.264846 (+767537us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400716454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:21.264916 (+    70us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:21.265084 (+   168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":423291520}
I20260924 14:30:21.301662 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:21.224151 29344 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:21.299837 28867 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:21.326978 28867 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:21.328040 29300 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:21.332038 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 802) took 1142 ms. Trace:
I20260924 14:30:21.332312 26331 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:21.332549 26403 rpcz_store.cc:276] 0924 14:30:20.190067 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:20.190172 (+   105us) service_pool.cc:227] Handling call
0924 14:30:20.190491 (+   319us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:20.191195 (+   704us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:30:20.191931 (+   736us) log.cc:842] Serialized 14 byte log entry
0924 14:30:20.603814 (+411883us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:20.604442 (+   628us) log.cc:842] Serialized 26 byte log entry
0924 14:30:20.604645 (+   203us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:20.604691 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:20.604793 (+   102us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:21.201317 (+596524us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:21.331693 (+130376us) raft_consensus.cc:1671] finished
0924 14:30:21.331819 (+   126us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:21.331970 (+   151us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":247,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":569,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":12391,"spinlock_wait_cycles":23985536,"thread_start_us":409838,"threads_started":1,"wal-append.queue_time_us":499687}
W20260924 14:30:21.336014 29380 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260924 14:30:21.338662 29288 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:21.341001 29288 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:21.343691 29275 consensus_meta.cc:287] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.605s	user 0.008s	sys 0.004s
I20260924 14:30:21.345669 26330 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:21.290616 29292 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260924 14:30:21.348183 29292 raft_consensus.cc:3032] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:21.348531 29292 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:21.294232 29139 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:21.350497 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:21.349326 29292 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:21.351758 29292 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:21.311347 29368 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:21.353658 29368 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:21.315165 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:21.313265 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.306035 27181 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:21.300336 26326 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:21.236904 28895 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:21.308079 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:21.363116 26397 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:21.373087 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:21.373452 27126 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:21.374109 26324 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:21.317232 29217 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:21.378193 26257 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:21.379958 26326 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260924 14:30:21.383602 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 242434us
I20260924 14:30:21.385886 26326 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:21.326308 29185 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:21.387177 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:21.394869 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:21.405077 29233 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:21.409935 29233 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:21.410166 26251 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:21.412456 26323 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:21.413645 26324 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
W20260924 14:30:21.417183 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 114162us
I20260924 14:30:21.421578 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:21.422408 26397 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:21.426541 26326 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:21.427134 26324 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:30:21.427582 26327 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:21.429163 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:21.427062 26298 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:21.384800 29108 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:21.436353 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.395018 29300 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:21.449280 26326 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260924 14:30:21.455582 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 216243us
I20260924 14:30:21.385151 29217 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:21.431093 29277 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:21.497251 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:21.499953 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 108612us
I20260924 14:30:21.520915 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:21.438934 29375 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:21.441635 29196 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:21.441668 29331 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:21.526741 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 176973us
I20260924 14:30:21.536641 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.446748 29108 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:21.542411 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:21.545881 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:21.547133 26397 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:21.443941 29328 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260924 14:30:21.552608 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:21.524516 29361 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260924 14:30:21.560348 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:21.516296 29300 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:21.557507 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:21.558933 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:21.563722 26402 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:21.523488 29196 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:21.562295 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:21.566684 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:21.567368 26405 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:21.554204 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.564900 26404 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:21.448814 29233 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:21.549512 29290 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:21.577569 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.580909 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:21.581262 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:21.582146 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:21.584164 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:21.584719 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.585954 27188 raft_consensus.cc:2379] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:21.588109 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:21.587729 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:21.545315 29331 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:21.764510 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:21.685848 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 107573us
I20260924 14:30:21.650243 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:21.765148 26255 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:21.595948 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:21.769085 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:21.770856 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:21.771978 26397 raft_consensus.cc:2396] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:21.773736 26224 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:21.655575 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:21.764447 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:21.777417 26254 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:21.607368 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:21.570557 29196 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:21.650489 26251 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260924 14:30:21.782074 27188 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:21.782232 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:21.839789 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 58176.5us
I20260924 14:30:21.835312 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:21.786993 27185 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:21.835547 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 485) took 2983 ms. Trace:
I20260924 14:30:21.784247 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.839649 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.846254 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:21.847153 26324 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:21.841136 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.845227 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:21.783703 26401 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:30:21.847165 26326 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:21.874779 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.579434 29376 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:21.845844 26329 rpcz_store.cc:276] 0924 14:30:18.852543 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.852661 (+   118us) service_pool.cc:227] Handling call
0924 14:30:18.852988 (+   327us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:19.019752 (+166764us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:30:19.034304 (+ 14552us) log.cc:842] Serialized 14 byte log entry
0924 14:30:19.180897 (+146593us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:20.420214 (+1239317us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b2c001e1fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:20.427435 (+  7221us) log.cc:842] Serialized 26 byte log entry
0924 14:30:20.596648 (+169213us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:20.597039 (+   391us) log.cc:842] Serialized 14 byte log entry
0924 14:30:20.738353 (+141314us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:21.835164 (+1096811us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400237654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:21.835225 (+    61us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:21.835278 (+    53us) raft_consensus.cc:1671] finished
0924 14:30:21.835290 (+    12us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:21.835439 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":113079,"prepare.queue_time_us":18188,"prepare.run_cpu_time_us":1144,"prepare.run_wall_time_us":4577,"raft.queue_time_us":90261,"raft.run_cpu_time_us":413,"raft.run_wall_time_us":43990,"spinlock_wait_cycles":392483968,"thread_start_us":364921,"threads_started":3,"wal-append.queue_time_us":458122,"wal-append.run_cpu_time_us":1534,"wal-append.run_wall_time_us":1143784}
I20260924 14:30:21.909530 26249 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:30:21.875767 26330 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:21.913203 26256 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
W20260924 14:30:21.774838 29370 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260924 14:30:21.920634 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 75481.1us
W20260924 14:30:21.921912 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 68798.3us
I20260924 14:30:21.922612 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:21.702752 29259 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:21.664523 29216 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:21.843940 29388 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.389s	user 0.000s	sys 0.001s
W20260924 14:30:21.838622 29387 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260924 14:30:21.927147 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:21.973943 26330 raft_consensus.cc:2379] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:21.932687 29212 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:21.974550 29274 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:21.975383 29212 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:21.976122 29144 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:21.976789 29144 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:21.977643 29212 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:21.934661 29217 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:21.590622 28895 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:21.904490 29366 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.205s	user 0.000s	sys 0.003s
I20260924 14:30:21.609163 29344 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:21.583714 29290 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:21.886071 29383 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:21.958221 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:21.960667 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:21.962245 29404 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
I20260924 14:30:21.955565 29259 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:21.988957 29398 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
I20260924 14:30:21.990557 29259 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:21.964183 29216 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:21.991243 29233 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:21.992030 29233 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:21.992509 29233 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:21.993311 29216 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:21.968096 28867 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.640s	user 0.005s	sys 0.008s
W20260924 14:30:21.969460 29392 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.300s	user 0.000s	sys 0.000s
I20260924 14:30:21.698562 29108 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:21.994894 28867 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:22.000311 26331 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
W20260924 14:30:22.001113 29403 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260924 14:30:22.006606 28867 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:21.749739 29300 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:21.956183 29260 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260924 14:30:22.008569 29224 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:22.011425 29331 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:21.908179 29393 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:21.886571 29376 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:21.901793 29186 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:21.839584 29351 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:22.038419 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:21.949270 29382 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:22.042822 27188 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:30:22.043035 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:22.047916 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 90606.2us
I20260924 14:30:22.051548 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:22.052186 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:22.055032 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:22.016423 29233 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:22.038040 29376 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:21.986927 29290 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:22.115258 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:22.116909 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:22.129515 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:22.134362 27185 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:22.049208 29382 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:22.146253 27123 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:22.150592 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:22.097455 28883 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:22.197292 29144 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:22.039947 29186 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:22.201992 29344 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:22.217906 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 71772.2us
I20260924 14:30:22.142315 29262 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:22.113910 29373 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:22.044224 29351 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:22.151351 29382 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:22.150525 29065 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:21.972259 29353 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:21.974741 29386 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260924 14:30:22.243366 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 461148us
I20260924 14:30:22.183341 29280 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:22.264078 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:21.931746 29271 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:22.252624 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:22.232313 29146 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:22.267197 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:22.271817 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:22.269177 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.022s	sys 0.026s
W20260924 14:30:22.272487 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.022s	sys 0.026s
I20260924 14:30:22.240422 29262 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:22.274681 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 350743us
I20260924 14:30:22.275350 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:22.282195 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 324077us
I20260924 14:30:22.283174 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:22.298405 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:22.302385 27194 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.014s	sys 0.009s
W20260924 14:30:22.302798 27194 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.014s	sys 0.009s
I20260924 14:30:22.306267 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:22.307372 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:22.313735 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:22.245683 29344 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:22.316854 27181 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.017s	sys 0.030s
W20260924 14:30:22.317451 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 58737.9us
I20260924 14:30:22.248370 29373 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:22.296610 29309 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:22.291819 29217 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:22.303243 29280 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:22.342525 29262 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:22.342962 29353 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260924 14:30:22.353864 26222 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.012s	sys 0.032s
W20260924 14:30:22.357686 26222 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.012s	sys 0.041s
I20260924 14:30:22.345770 29409 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:22.346769 29280 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:22.346048 29333 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:30:22.349953 29367 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:22.350560 29386 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:22.351470 29274 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:22.340466 29217 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:22.371922 29348 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:22.378386 29344 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:22.395722 26331 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:22.365762 29280 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:22.403735 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:22.390185 29386 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:22.382400 29353 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:22.393882 29224 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:22.397596 29416 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:22.397451 29262 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:22.323546 27181 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.018s	sys 0.030s
I20260924 14:30:22.428076 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 496) took 1192 ms. Trace:
I20260924 14:30:22.428659 26325 rpcz_store.cc:276] 0924 14:30:21.236120 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:21.243812 (+  7692us) service_pool.cc:227] Handling call
0924 14:30:21.244193 (+   381us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:21.264494 (+ 20301us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:22.427746 (+1163252us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b54005d8c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:22.427993 (+   247us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":34734336}
I20260924 14:30:22.406607 29280 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:22.401749 29412 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:22.405563 29186 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:22.409174 29351 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:22.414390 29386 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:22.413246 29317 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:22.394865 29409 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260924 14:30:22.398293 29274 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:22.436234 29036 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:22.464468 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:22.425792 29224 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:22.472820 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 197576us
I20260924 14:30:22.423113 29353 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:22.478964 26258 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260924 14:30:22.480379 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 205224us
I20260924 14:30:22.480690 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 905) took 1188 ms. Trace:
I20260924 14:30:22.481105 26257 rpcz_store.cc:276] 0924 14:30:21.292390 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:21.292541 (+   151us) service_pool.cc:227] Handling call
0924 14:30:21.314278 (+ 21737us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:21.378786 (+ 64508us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:22.480455 (+1101669us) spinlock_profiling.cc:243] Waited 59.9 ms on lock 0x7b5400288f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:22.480629 (+   174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":137879424}
I20260924 14:30:22.484143 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:22.487576 26253 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:22.467514 29373 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:22.490419 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:22.494248 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:22.466957 29036 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:22.496737 26328 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:22.497886 27188 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:22.500483 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:22.495517 29347 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:22.508210 29347 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:22.569437 29351 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:30:22.570806 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61360.3us
I20260924 14:30:22.570972 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:22.571807 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:22.572391 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:22.578931 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:22.574076 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:22.463882 29317 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:22.580238 26330 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260924 14:30:22.580978 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:22.581367 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:22.489462 29292 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.137s	user 0.015s	sys 0.000s
I20260924 14:30:22.581902 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:22.583077 26329 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:22.585526 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:22.455902 29412 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:22.589797 26256 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:22.590322 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:22.589103 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:22.593092 27188 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260924 14:30:22.594466 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:22.595077 27194 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
W20260924 14:30:22.595904 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 246076us
I20260924 14:30:22.596390 26328 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
W20260924 14:30:22.580442 29393 consensus_meta.cc:287] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.001s
I20260924 14:30:22.483014 29411 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.101s
I20260924 14:30:22.601120 26221 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:22.616822 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 309525us
I20260924 14:30:22.590871 29271 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:22.622186 26369 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:22.628654 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:22.574889 29309 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260924 14:30:22.631505 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:22.632239 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:22.632850 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:22.631551 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:22.634678 27185 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:30:22.569013 29418 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260924 14:30:22.642313 26297 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:22.598897 29351 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
W20260924 14:30:22.633401 29368 consensus_meta.cc:287] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.279s	user 0.011s	sys 0.003s
I20260924 14:30:22.584379 29419 consensus_queue.cc:1046] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260924 14:30:22.606776 29413 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:22.630529 29144 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:22.608970 29393 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:22.625450 29271 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:22.623812 29217 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:22.657756 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 470) took 1106 ms. Trace:
I20260924 14:30:22.662293 29333 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:22.646230 29275 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:22.654484 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:22.638365 29377 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:22.634101 28895 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:22.631685 29424 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:22.644869 29300 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:22.643781 29348 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:22.678431 29292 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:22.644102 29425 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:22.681591 26251 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:22.681797 26255 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:22.676693 27181 rpcz_store.cc:276] 0924 14:30:21.551099 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:21.551358 (+   259us) service_pool.cc:227] Handling call
0924 14:30:21.562999 (+ 11641us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:21.563211 (+   212us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:30:22.433428 (+870217us) log.cc:842] Serialized 14 byte log entry
0924 14:30:22.657354 (+223926us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:22.657414 (+    60us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:22.657465 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:22.657559 (+    94us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:22.657692 (+   133us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":102144,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"thread_start_us":1087327,"threads_started":2}
I20260924 14:30:22.630529 29379 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260924 14:30:22.686959 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:22.657699 29005 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:22.614754 29036 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:22.675660 29144 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:22.675052 29309 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:22.713155 29274 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:22.717837 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 43887.1us
W20260924 14:30:22.663982 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 433015us
I20260924 14:30:22.658660 29368 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:22.725935 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 72172.6us
W20260924 14:30:22.741508 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 55062.2us
I20260924 14:30:22.757752 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:22.716483 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 115877us
I20260924 14:30:22.746436 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:22.747084 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:22.725219 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 52384.7us
I20260924 14:30:22.758638 26324 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:22.759763 26328 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260924 14:30:22.760336 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:22.710197 29383 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:22.751194 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:22.757851 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:22.764398 26326 raft_consensus.cc:2362] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7. Re-sending same reply.
I20260924 14:30:22.733449 29224 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:22.767503 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:22.731161 29353 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:22.750826 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:22.733893 29425 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:22.684330 29333 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:22.745458 29144 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:22.793462 27126 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:22.776142 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:22.803661 26405 raft_consensus.cc:2396] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:22.744362 29292 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:22.682263 29428 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260924 14:30:22.780519 26323 raft_consensus.cc:2354] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8.
I20260924 14:30:22.724170 29300 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:22.748936 29418 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:22.724896 29348 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:22.765421 29036 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:22.763346 29274 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:22.781090 29196 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.869s	user 0.007s	sys 0.014s
W20260924 14:30:22.854002 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 90477.2us
I20260924 14:30:22.768855 29347 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:22.800400 29317 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:22.850287 29233 consensus_meta.cc:287] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.002s
I20260924 14:30:22.802083 29224 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:22.795284 29353 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:22.780280 29432 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:22.720018 29424 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:22.857018 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 54322.3us
I20260924 14:30:22.754552 29309 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:22.794925 29425 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:22.742358 29271 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:22.936177 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:22.845788 29139 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:22.941100 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 163515us
I20260924 14:30:22.849048 29412 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:22.862717 29401 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.085s	user 0.004s	sys 0.003s
I20260924 14:30:22.862969 29435 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:22.963541 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:22.940963 29287 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:22.978708 27185 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:22.853231 29393 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
W20260924 14:30:22.974545 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 204966us
I20260924 14:30:22.971637 26332 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:22.951960 29353 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:22.953130 29437 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:22.905486 29274 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:22.973685 29224 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:22.862047 29418 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:23.000972 29290 consensus_meta.cc:287] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.855s	user 0.010s	sys 0.002s
I20260924 14:30:23.018778 29418 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:22.990269 29271 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:22.979749 29317 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260924 14:30:22.945416 29233 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:22.753283 29429 consensus_queue.cc:1159] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "45ab97b763804767846e9d27a01b561f" responder_term: 10 status { last_received { term: 9 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:30:23.018487 29290 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:22.973102 29139 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:22.908210 29196 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:23.038995 26253 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:22.993793 29427 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:23.046671 29419 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:23.051604 29427 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:22.959208 29432 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:23.052886 26332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 508) took 1786 ms (client timeout 1652 ms). Trace:
W20260924 14:30:23.053149 29440 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260924 14:30:23.053412 29139 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:23.061340 26327 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:30:23.059342 26332 rpcz_store.cc:269] 0924 14:30:21.266173 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:21.307617 (+ 41444us) service_pool.cc:227] Handling call
0924 14:30:23.052654 (+1745037us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54005582d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:23.052806 (+   152us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2818304768}
I20260924 14:30:22.986815 29287 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:23.063112 29453 raft_consensus.cc:465] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Not starting pre-election -- already a leader
I20260924 14:30:23.060140 29224 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:23.064472 29453 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:23.064240 29224 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:23.065035 29453 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:23.065567 29348 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.067091 29453 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:23.069108 29233 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:23.070804 29290 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:23.072384 29353 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:23.073307 29368 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.073464 29432 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:23.050035 29376 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.906s	user 0.006s	sys 0.007s
I20260924 14:30:23.074143 29376 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:23.075209 26250 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:23.076182 29376 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:23.077643 29262 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.000s
W20260924 14:30:23.081029 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 117595us
I20260924 14:30:23.064646 29333 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.067332 29424 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:23.080199 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 129257us
W20260924 14:30:23.220008 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 224081us
I20260924 14:30:23.234381 26369 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:23.063333 29439 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:23.263223 26369 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:23.262969 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 169736us
W20260924 14:30:23.331007 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.004s	sys 0.044s
W20260924 14:30:23.331317 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 68198.9us
W20260924 14:30:23.331593 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.005s	sys 0.044s
W20260924 14:30:23.332087 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 528305us
I20260924 14:30:23.336673 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:23.337613 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:22.992847 29435 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:23.338265 26404 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 8
I20260924 14:30:23.262642 29146 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:23.327576 29300 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:23.262229 29368 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:23.094506 29375 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:23.358779 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:23.359169 27125 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.048s	sys 0.064s
I20260924 14:30:23.297940 29353 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:23.232672 29426 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260924 14:30:23.361935 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.085202 29427 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:23.380255 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.049s	sys 0.064s
I20260924 14:30:23.368122 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.365272 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:23.339655 29309 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:23.358238 29376 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:23.398520 27188 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:23.398128 27192 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:23.360898 29450 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:23.400966 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:23.409335 26325 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:23.371155 29448 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:23.414763 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.420163 26325 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:23.374709 29353 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:23.428803 29446 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260924 14:30:23.437608 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:23.438316 26332 raft_consensus.cc:2362] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7. Re-sending same reply.
I20260924 14:30:23.402485 29375 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:23.434615 27188 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:23.372803 29146 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:23.374948 29368 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:23.437074 29353 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:30:23.444232 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 46567.5us
I20260924 14:30:23.454442 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.455780 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.463020 26369 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:23.464876 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:23.456988 29287 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:23.457600 29408 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:23.457857 29224 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:23.459476 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:23.565196 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 99800.1us
I20260924 14:30:23.565730 26251 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:23.459971 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:23.460270 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:23.460928 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:23.340037 29439 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:23.572384 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:23.461659 26250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 768) took 1843 ms (client timeout 2359 ms). Trace:
I20260924 14:30:23.467823 29453 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:23.468297 29419 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:23.468806 29458 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:23.462088 29418 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:23.471163 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:23.588554 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.471714 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.473069 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.472198 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:23.589900 26332 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:23.590605 29262 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election won for term 7
W20260924 14:30:23.590690 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 118595us
I20260924 14:30:23.590919 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.591176 29262 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:23.592161 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.592070 29262 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:23.592653 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:23.593267 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.593793 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:23.594412 26398 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:23.474306 26329 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:23.474925 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:23.596158 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:23.476053 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.596618 29447 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:23.476743 29431 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.700s	user 0.001s	sys 0.000s
I20260924 14:30:23.476995 27123 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:23.570706 29224 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:23.597167 29447 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:23.597810 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 120898us
I20260924 14:30:23.574321 29436 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260924 14:30:23.599637 29414 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:23.600049 29414 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:23.600369 29414 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:23.600565 27123 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:23.480717 26323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1092) took 1640 ms (client timeout 1850 ms). Trace:
I20260924 14:30:23.448812 29375 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:23.482491 29260 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Stepping down as leader of term 6
W20260924 14:30:23.485059 29460 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260924 14:30:23.449636 29196 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:23.439424 29274 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.491282 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.532830 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 764) took 1950 ms. Trace:
I20260924 14:30:23.626132 26258 rpcz_store.cc:276] 0924 14:30:21.582488 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:21.583060 (+   572us) service_pool.cc:227] Handling call
0924 14:30:21.583353 (+   293us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:22.479683 (+896330us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:23.522669 (+1042986us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b54004f9f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:23.532762 (+ 10093us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2055575808}
W20260924 14:30:23.534348 26331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 513) took 1418 ms (client timeout 1584 ms). Trace:
W20260924 14:30:23.627606 26331 rpcz_store.cc:269] 0924 14:30:22.116116 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:22.116679 (+   563us) service_pool.cc:227] Handling call
0924 14:30:23.522175 (+1405496us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b540001dbd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:23.534277 (+ 12102us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":464559360}
W20260924 14:30:23.628005 29416 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.191s	user 0.006s	sys 0.008s
I20260924 14:30:23.628448 29416 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:23.629179 26249 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:23.535871 29146 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:23.539430 29036 consensus_meta.cc:287] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.000s
I20260924 14:30:23.498898 29184 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:23.543493 26326 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:23.546255 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.546959 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1091) took 1707 ms. Trace:
I20260924 14:30:23.638618 26327 rpcz_store.cc:276] 0924 14:30:21.839531 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:21.883344 (+ 43813us) service_pool.cc:227] Handling call
0924 14:30:23.523170 (+1639826us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400498754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:23.546897 (+ 23727us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2616504064}
I20260924 14:30:23.464540 29437 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260924 14:30:23.549204 29427 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:23.549921 26404 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:23.551337 29368 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:23.554149 29280 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.111s	user 0.006s	sys 0.005s
I20260924 14:30:23.433467 29448 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:23.561643 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:23.562775 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 161885us
I20260924 14:30:23.566747 26397 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:23.647680 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:23.560427 29351 raft_consensus.cc:3032] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
W20260924 14:30:23.573359 26250 rpcz_store.cc:269] 0924 14:30:21.604962 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:21.605175 (+   213us) service_pool.cc:227] Handling call
0924 14:30:23.448264 (+1843089us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54000c17d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:23.448381 (+   117us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3010838656}
I20260924 14:30:23.649180 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:23.586496 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.586665 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:23.598855 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:23.461095 29292 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:23.650348 26251 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:23.603488 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:23.601353 26323 rpcz_store.cc:269] 0924 14:30:21.839896 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:21.840816 (+   920us) service_pool.cc:227] Handling call
0924 14:30:23.480508 (+1639692us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54007cf6d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:23.480657 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2142886656}
I20260924 14:30:23.607393 27188 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:30:23.608129 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:23.571929 29382 consensus_meta.cc:287] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.307s	user 0.005s	sys 0.011s
I20260924 14:30:23.617033 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.624742 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:23.589519 29453 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:23.659997 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.630714 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:23.624650 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.599309 29447 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:23.671892 26295 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:23.672228 26250 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260924 14:30:23.660607 27186 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:23.627329 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:23.586041 29458 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:23.634267 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 68360.4us
I20260924 14:30:23.600908 29224 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.641362 26330 raft_consensus.cc:2396] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:23.608824 29260 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:23.644654 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:23.567072 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:23.657799 29280 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:23.595208 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:23.641074 28867 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:23.567066 29408 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:23.644224 29184 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:23.605556 29450 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260924 14:30:23.605022 29418 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:23.656076 26329 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:23.652071 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.634099 29328 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:23.653797 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:23.637598 29386 consensus_meta.cc:287] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.176s	user 0.007s	sys 0.007s
I20260924 14:30:23.654458 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:23.625260 29422 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.899s	user 0.004s	sys 0.009s
I20260924 14:30:23.711323 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.711802 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.663332 26397 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:23.714382 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:23.666199 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:23.653690 29292 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:23.715910 29292 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:23.589391 29419 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:23.669406 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:23.728399 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:23.681819 29425 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.000s
I20260924 14:30:23.740350 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:23.693367 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:23.695142 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:23.683733 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.636838 29435 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:23.698892 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:23.679787 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:23.654006 29351 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:23.676478 29462 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:23.723654 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 121584us
I20260924 14:30:23.696131 29458 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:23.700515 29213 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:23.675900 29382 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:23.728617 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:23.788622 26257 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:23.683889 29144 consensus_meta.cc:287] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.830s	user 0.010s	sys 0.002s
I20260924 14:30:23.730643 26251 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:23.693148 26406 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:23.736526 26328 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:23.737910 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:23.794934 26329 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:23.684002 29413 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:23.795755 29413 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:23.681559 29448 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:23.678011 29465 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:23.718235 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:23.809633 26397 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260924 14:30:23.809970 26397 pending_rounds.cc:76] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 3
I20260924 14:30:23.810209 26397 pending_rounds.cc:97] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260924 14:30:23.810518 26397 raft_consensus.cc:2866] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:23.806445 29144 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:23.812505 29144 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:23.813203 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1097) took 1055 ms. Trace:
I20260924 14:30:23.813859 29351 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:23.815266 29351 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:23.818120 29213 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:23.685735 29427 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260924 14:30:23.759867 27123 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.584s (SENT)
I20260924 14:30:23.781711 29450 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:23.683985 29259 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:23.689574 29184 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:23.685992 28867 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:23.687311 29408 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.778167 29419 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:23.778440 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.779353 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:23.685464 29375 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.785828 26298 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:23.839771 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 54022.1us
I20260924 14:30:23.741878 26327 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:23.710170 29386 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:23.804178 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1074) took 1055 ms. Trace:
W20260924 14:30:23.804929 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 106115us
I20260924 14:30:23.842268 26401 rpcz_store.cc:276] 0924 14:30:22.749117 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:22.749231 (+   114us) service_pool.cc:227] Handling call
0924 14:30:22.749499 (+   268us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:23.019685 (+270186us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:23.019719 (+    34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:23.019776 (+    57us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:23.019826 (+    50us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:23.803940 (+784114us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400217854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:23.803993 (+    53us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:23.804123 (+   130us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":620833536}
I20260924 14:30:23.717674 29328 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:23.796166 29413 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:23.806787 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:23.689277 29274 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260924 14:30:23.825904 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:23.744916 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 78798.1us
I20260924 14:30:23.821064 29428 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:23.813748 26324 rpcz_store.cc:276] 0924 14:30:22.757906 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:22.759548 (+  1642us) service_pool.cc:227] Handling call
0924 14:30:23.534781 (+775233us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:23.575876 (+ 41095us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:23.575911 (+    35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:23.588285 (+ 12374us) log.cc:842] Serialized 26 byte log entry
0924 14:30:23.736545 (+148260us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:23.737258 (+   713us) log.cc:842] Serialized 14 byte log entry
0924 14:30:23.811682 (+ 74424us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:23.811794 (+   112us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:23.811850 (+    56us) raft_consensus.cc:1671] finished
0924 14:30:23.811869 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:23.812992 (+  1123us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b540001dbd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:23.813133 (+   141us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":103666,"prepare.queue_time_us":149,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":48,"raft.queue_time_us":50770,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":15823,"spinlock_wait_cycles":1782001280,"thread_start_us":105340,"threads_started":1,"wal-append.queue_time_us":130834}
I20260924 14:30:23.780853 29216 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:23.827116 26400 raft_consensus.cc:2354] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11.
I20260924 14:30:23.831300 26403 raft_consensus.cc:2354] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:30:23.764070 29280 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.832574 29259 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:23.838976 26404 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:23.795251 26330 raft_consensus.cc:2354] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6.
I20260924 14:30:23.845531 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:23.850313 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:23.852798 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:23.825423 29427 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:23.864470 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:23.857187 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 97381.7us
I20260924 14:30:23.861932 26398 raft_consensus.cc:2354] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9.
I20260924 14:30:23.743665 29425 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:23.830435 29450 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:23.874675 29464 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:23.845563 29419 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:23.889883 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.928401 26327 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260924 14:30:23.929877 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:23.930675 26327 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:23.932686 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:23.933867 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 139649us
I20260924 14:30:23.858753 29216 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:23.791765 29462 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:23.936260 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 52237us
I20260924 14:30:23.939554 27123 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:23.940999 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:23.876660 29212 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:23.876853 29427 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:23.876127 29375 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:23.948597 26255 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260924 14:30:23.949651 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.952171 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:23.954011 29271 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.933s	user 0.010s	sys 0.000s
I20260924 14:30:23.867107 29259 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:24.008313 27185 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 828) took 1255 ms (client timeout 1520 ms). Trace:
I20260924 14:30:23.855471 29213 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:23.926184 29450 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:23.955658 29328 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:23.963260 29280 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:23.964763 29386 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:23.965440 29467 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260924 14:30:23.879142 29353 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:23.959201 29439 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:23.967305 29305 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:23.949940 29300 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:24.030236 29300 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:24.032330 29300 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.975687 29351 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:24.032963 29351 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:24.035038 29351 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:23.975920 29453 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.940402 29437 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260924 14:30:23.934479 29419 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:23.828015 29465 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:23.996677 26401 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:30:24.001135 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1208) took 1428 ms. Trace:
I20260924 14:30:24.001030 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:23.940402 29426 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:24.046626 29426 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:24.024689 29280 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:24.002782 29382 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:24.003101 29425 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:23.999053 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:23.948212 29216 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:24.051810 29353 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:24.013415 29375 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:24.009083 27185 rpcz_store.cc:269] 0924 14:30:22.750806 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:22.750960 (+   154us) service_pool.cc:227] Handling call
0924 14:30:24.005566 (+1254606us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540073c654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:24.005673 (+   107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":493543296}
I20260924 14:30:24.016335 29318 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:24.017379 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 76453us
I20260924 14:30:23.957325 29438 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:23.879949 29448 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260924 14:30:24.013731 26327 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:24.019205 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:24.021934 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:24.023874 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 55615.7us
I20260924 14:30:23.970463 29458 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.064054 29465 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:24.064978 29280 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:24.003417 29390 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.534s	user 0.000s	sys 0.005s
I20260924 14:30:24.029696 26397 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 3. Current term is 4. Ops: []
I20260924 14:30:24.068522 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:24.045313 26332 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:24.049192 29426 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:24.001868 29418 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:24.010278 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:24.044117 26249 rpcz_store.cc:276] 0924 14:30:22.572341 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:22.572468 (+   127us) service_pool.cc:227] Handling call
0924 14:30:22.572742 (+   274us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:23.629537 (+1056795us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:24.000910 (+371373us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540002b554. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:24.001073 (+   163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2429154560}
I20260924 14:30:24.001214 29462 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:24.056944 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:24.063199 29481 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:24.070621 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:24.067011 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 220150us
I20260924 14:30:24.070223 26295 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:24.063235 28883 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:30:24.077390 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 67186.9us
I20260924 14:30:24.091259 29419 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:24.091604 29418 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:24.095844 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:24.059092 29438 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:24.097280 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:24.110680 26327 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:24.098390 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:24.099123 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:24.119797 26328 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:24.096393 26402 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:24.124725 29428 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:24.129019 26224 leader_election.cc:398] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9.
I20260924 14:30:24.129451 26224 leader_election.cc:401] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:30:24.129267 29344 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:24.130427 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:24.131196 29344 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 9
I20260924 14:30:24.131824 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:24.131935 29375 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.133023 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:24.134157 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:24.094648 29480 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260924 14:30:24.137426 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:24.085963 29465 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:24.085990 29425 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:24.092440 29458 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:24.141863 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:24.086048 29481 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:24.106544 26332 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:24.143993 26327 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:24.068864 29411 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260924 14:30:24.154052 26369 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:24.154127 29340 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:24.154702 29340 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:24.155148 28867 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:24.155579 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:24.156154 26405 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:24.152767 29425 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.156946 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:24.158392 29425 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:24.158593 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:24.124837 29300 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260924 14:30:24.159684 29458 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:24.160174 29458 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:24.160501 29300 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:24.160872 29300 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:24.161113 29459 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:24.161233 29300 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:24.160985 29458 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.124483 29418 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:24.163723 29418 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:24.167764 29280 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:24.168390 29280 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.168972 29418 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:24.176287 29459 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:24.139963 29419 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:24.149910 29465 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:24.183772 29300 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:24.178110 29459 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.065887 29318 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:24.126603 29438 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:24.190905 29419 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:24.191795 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:24.191951 29465 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:24.093515 29414 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:24.193348 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:24.195145 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:24.188289 29425 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:24.196375 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:24.197463 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:24.229704 26328 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:24.185722 29450 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:24.282351 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:24.156956 29340 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:24.284013 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:24.292625 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:24.284444 29005 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:24.302513 29005 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:24.283284 29262 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:24.303192 29262 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.303789 29419 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:24.305352 29262 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:24.305904 29262 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:24.306345 29262 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:24.308425 29486 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260924 14:30:24.309507 29481 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:24.310672 29292 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.000s
I20260924 14:30:24.310925 29481 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:24.311128 29292 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:24.194432 29427 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:24.314709 29292 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:24.315532 29427 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:24.316388 26398 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:24.317502 29216 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.317910 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:24.319780 26405 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:24.320745 29260 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.321918 29260 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:24.322232 29344 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9.
I20260924 14:30:24.322455 29450 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:24.323108 26329 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:24.325310 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:24.331254 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:24.331355 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:24.333357 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:24.335145 26404 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:24.319111 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:24.347805 27190 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260924 14:30:24.352574 29186 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:24.353364 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 712782us
I20260924 14:30:24.353246 29186 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:24.355537 29292 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:24.355779 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:24.911357 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 919) took 2235 ms. Trace:
I20260924 14:30:24.912094 26256 rpcz_store.cc:276] 0924 14:30:22.675659 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:22.675813 (+   154us) service_pool.cc:227] Handling call
0924 14:30:22.676145 (+   332us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:22.975033 (+298888us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:22.975078 (+    45us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:22.975139 (+    61us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:22.975181 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:24.910575 (+1935394us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54002a1554. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:24.910734 (+   159us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:24.911101 (+   367us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":686781312}
I20260924 14:30:24.916052 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 782) took 1294 ms. Trace:
I20260924 14:30:24.919726 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:24.919945 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:24.921716 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 780) took 1313 ms. Trace:
I20260924 14:30:24.922139 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 530) took 1039 ms. Trace:
I20260924 14:30:24.923173 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 764) took 1255 ms. Trace:
I20260924 14:30:24.923349 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:24.579195 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 245402us
I20260924 14:30:24.929032 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 772) took 1185 ms. Trace:
I20260924 14:30:24.932358 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:24.947961 26255 rpcz_store.cc:276] 0924 14:30:23.611844 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.612041 (+   197us) service_pool.cc:227] Handling call
0924 14:30:23.613711 (+  1670us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:24.006284 (+392573us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:24.920991 (+914707us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b540015e5d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:24.921526 (+   535us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":769607552}
I20260924 14:30:24.943621 26253 rpcz_store.cc:276] 0924 14:30:23.621071 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.655934 (+ 34863us) service_pool.cc:227] Handling call
0924 14:30:23.656268 (+   334us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:23.692649 (+ 36381us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:23.692692 (+    43us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:23.692747 (+    55us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:23.692790 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:24.915427 (+1222637us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b54004538d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:24.915527 (+   100us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:24.915840 (+   313us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":83099520}
I20260924 14:30:24.930932 26326 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:24.364740 29300 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:24.948029 26324 rpcz_store.cc:276] 0924 14:30:23.882229 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.887189 (+  4960us) service_pool.cc:227] Handling call
0924 14:30:23.887642 (+   453us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:24.051854 (+164212us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:24.051886 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:24.051945 (+    59us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:24.051980 (+    35us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:24.921583 (+869603us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400237654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:24.921638 (+    55us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:24.921912 (+   274us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":377101056}
I20260924 14:30:24.714206 27185 raft_consensus.cc:2396] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260924 14:30:24.960970 26329 rpcz_store.cc:276] 0924 14:30:23.743591 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.795892 (+ 52301us) service_pool.cc:227] Handling call
0924 14:30:23.796192 (+   300us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:24.346487 (+550295us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:24.928364 (+581877us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54007d37d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:24.928817 (+   453us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1211371904}
I20260924 14:30:24.290122 29485 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:24.583299 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 258078us
I20260924 14:30:24.714910 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:24.709883 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:24.984792 29318 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:24.985349 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 488) took 1323 ms. Trace:
I20260924 14:30:24.712313 29287 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:24.987936 29373 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:24.989022 26398 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:24.989130 29373 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:24.591780 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:24.425881 29005 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:24.994346 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:24.994939 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:24.995751 29421 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:24.524837 29262 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.996815 29421 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:24.715819 29288 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:24.295135 29425 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:24.713475 29328 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:24.581563 29186 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:24.987190 29375 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.006s	sys 0.013s
W20260924 14:30:24.826927 29479 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.653s	user 0.003s	sys 0.000s
W20260924 14:30:24.834465 29484 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.705s	user 0.002s	sys 0.000s
I20260924 14:30:24.988602 29287 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:24.833287 29490 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260924 14:30:24.746984 29414 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:24.362628 29413 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.506s	user 0.007s	sys 0.000s
I20260924 14:30:24.436354 29466 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:24.715834 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.023756 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:25.023981 29186 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:25.024359 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1082) took 1431 ms. Trace:
I20260924 14:30:25.024906 26400 rpcz_store.cc:276] 0924 14:30:23.592927 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.595932 (+  3005us) service_pool.cc:227] Handling call
0924 14:30:25.023860 (+1427928us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b540035cfd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:25.024127 (+   267us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":512058752}
I20260924 14:30:25.025548 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:25.027212 29425 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:25.028103 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 770) took 1314 ms. Trace:
W20260924 14:30:24.995777 29497 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260924 14:30:25.028934 26402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1069) took 2282 ms (client timeout 1564 ms). Trace:
W20260924 14:30:25.029641 26402 rpcz_store.cc:269] 0924 14:30:22.745972 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:22.746126 (+   154us) service_pool.cc:227] Handling call
0924 14:30:25.028414 (+2282288us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400336654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:25.028613 (+   199us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2887269760}
I20260924 14:30:25.029955 29413 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:25.031951 26251 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:25.033824 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1218) took 1097 ms. Trace:
I20260924 14:30:25.034916 29427 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:25.040205 29287 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:24.715308 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.009131 29476 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:25.011615 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:25.012032 29375 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.007s	sys 0.013s
I20260924 14:30:25.012318 29383 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:25.013270 29328 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:24.425843 29438 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:24.581225 29280 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:25.005410 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:25.028571 26325 rpcz_store.cc:276] 0924 14:30:23.713708 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.713823 (+   115us) service_pool.cc:227] Handling call
0924 14:30:23.714192 (+   369us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:23.738161 (+ 23969us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:23.738190 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:23.768106 (+ 29916us) log.cc:842] Serialized 26 byte log entry
0924 14:30:24.007488 (+239382us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:24.007525 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:24.007585 (+    60us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:24.826168 (+818583us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:25.027481 (+201313us) raft_consensus.cc:1671] finished
0924 14:30:25.027524 (+    43us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:25.027907 (+   383us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":23591,"raft.queue_time_us":24109,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":37134,"spinlock_wait_cycles":31017856,"thread_start_us":177280,"threads_started":1,"wal-append.queue_time_us":337470}
I20260924 14:30:25.034566 26404 rpcz_store.cc:276] 0924 14:30:23.936490 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:24.009543 (+ 73053us) service_pool.cc:227] Handling call
0924 14:30:24.025742 (+ 16199us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:24.349297 (+323555us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:25.033078 (+683781us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540018ced4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:25.033532 (+   454us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":710967552}
I20260924 14:30:24.998971 29288 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.044160 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 855) took 1066 ms. Trace:
I20260924 14:30:24.995340 29005 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.048406 29367 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:25.049490 29367 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:25.050951 29328 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:24.702019 29216 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.051674 29383 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.053009 26255 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:25.054101 29438 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:25.054906 29383 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:25.041363 29427 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:25.055922 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 846) took 1653 ms. Trace:
I20260924 14:30:25.044145 29476 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:25.056921 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 852) took 1211 ms. Trace:
I20260924 14:30:25.057514 29427 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:25.059201 29373 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:25.060079 29425 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:25.047753 26405 rpcz_store.cc:276] 0924 14:30:23.977445 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:24.157602 (+180157us) service_pool.cc:227] Handling call
0924 14:30:24.319599 (+161997us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:24.320154 (+   555us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:25.043667 (+723513us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b540023f850. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:25.043966 (+   299us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":371507584}
I20260924 14:30:25.061787 29425 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:25.062311 29425 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:25.063892 27186 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:25.064046 29421 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:25.069736 27191 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:25.072273 29425 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:25.057339 26401 rpcz_store.cc:276] 0924 14:30:23.845116 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.852118 (+  7002us) service_pool.cc:227] Handling call
0924 14:30:25.056533 (+1204415us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54001e41d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:25.056751 (+   218us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":309036800}
I20260924 14:30:25.077813 29466 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:25.078441 29466 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:25.078691 29288 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.079783 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1087) took 1380 ms. Trace:
I20260924 14:30:25.080317 26397 rpcz_store.cc:276] 0924 14:30:23.698904 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.811317 (+112413us) service_pool.cc:227] Handling call
0924 14:30:23.821297 (+  9980us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:24.071740 (+250443us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:25.079247 (+1007507us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54001f5ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:25.079551 (+   304us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":478781184}
I20260924 14:30:25.024935 29186 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:24.948280 26323 rpcz_store.cc:276] 0924 14:30:23.667578 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.667732 (+   154us) service_pool.cc:227] Handling call
0924 14:30:23.668060 (+   328us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:23.691405 (+ 23345us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:23.691435 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:24.074844 (+383409us) spinlock_profiling.cc:243] Waited 5.07 ms on lock 0x7b2c001e1fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:24.075303 (+   459us) log.cc:842] Serialized 26 byte log entry
0924 14:30:24.426518 (+351215us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:24.426563 (+    45us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:24.426655 (+    92us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:24.922391 (+495736us) raft_consensus.cc:1671] finished
0924 14:30:24.922432 (+    41us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:24.922881 (+   449us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22954,"raft.queue_time_us":32367,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":17782,"spinlock_wait_cycles":11672448,"thread_start_us":333138,"threads_started":1,"wal-append.queue_time_us":674669}
I20260924 14:30:25.083940 27188 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:25.086046 29196 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:23.443425 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:25.056391 26399 rpcz_store.cc:276] 0924 14:30:23.402217 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.405364 (+  3147us) service_pool.cc:227] Handling call
0924 14:30:23.485645 (+ 80281us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:23.779433 (+293788us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:23.779483 (+    50us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:23.779524 (+    41us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:23.779556 (+    32us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:25.020528 (+1240972us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54003d59d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:25.020611 (+    83us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:25.055676 (+ 35065us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":675093376}
I20260924 14:30:24.986121 27182 rpcz_store.cc:276] 0924 14:30:23.662068 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.662178 (+   110us) service_pool.cc:227] Handling call
0924 14:30:23.662449 (+   271us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:23.690143 (+ 27694us) raft_consensus.cc:1520] Early marking committed up to index 3
0924 14:30:23.844838 (+154695us) log.cc:842] Serialized 14 byte log entry
0924 14:30:24.016329 (+171491us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:24.016385 (+    56us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:24.016429 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:24.984408 (+967979us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b54003c82d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:24.984511 (+   103us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:24.985011 (+   500us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":329575,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":93,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":63174912,"thread_start_us":277973,"threads_started":2,"wal-append.queue_time_us":171764}
I20260924 14:30:25.092636 29367 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.086745 29196 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.034839 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.105933 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 541) took 1466 ms. Trace:
I20260924 14:30:25.033788 29413 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:25.141757 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 944) took 1062 ms. Trace:
I20260924 14:30:25.097112 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1081) took 1504 ms. Trace:
I20260924 14:30:25.137019 27190 rpcz_store.cc:276] 0924 14:30:23.639836 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.640064 (+   228us) service_pool.cc:227] Handling call
0924 14:30:24.005693 (+365629us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:24.348430 (+342737us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:25.104633 (+756203us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b540073c654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:25.105460 (+   827us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540073c650. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:25.105712 (+   252us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1626013184}
I20260924 14:30:25.109653 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.143520 26251 rpcz_store.cc:276] 0924 14:30:24.079174 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:24.079322 (+   148us) service_pool.cc:227] Handling call
0924 14:30:24.108294 (+ 28972us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:25.032499 (+924205us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:25.141371 (+108872us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54000c21d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:25.141678 (+   307us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2123180928}
I20260924 14:30:25.101405 29108 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:25.147104 29108 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.143625 26403 rpcz_store.cc:276] 0924 14:30:23.592679 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.598598 (+  5919us) service_pool.cc:227] Handling call
0924 14:30:25.096601 (+1498003us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540036bad4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:25.096879 (+   278us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":407660288}
I20260924 14:30:25.150859 29427 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:25.151685 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.151793 29427 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.098611 29476 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:25.152583 26401 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:25.138125 29425 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:25.129035 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:25.129439 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:25.125310 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 90849us
I20260924 14:30:25.127318 26328 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:25.156529 29427 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:25.157064 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:25.157609 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 48293.5us
I20260924 14:30:25.158268 29438 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:25.096067 29186 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.163640 29485 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:25.180496 29185 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:25.181427 27189 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:25.181471 29485 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:25.126341 26326 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:25.182214 26402 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:25.166810 26325 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:25.113557 27191 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:25.126240 29259 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:25.120085 29052 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260924 14:30:25.124567 29458 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:25.106612 29328 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:25.135123 29265 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260924 14:30:25.134279 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1216) took 1199 ms. Trace:
I20260924 14:30:25.131660 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:25.132197 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:25.176293 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:25.120003 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 531197us
I20260924 14:30:25.156533 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.181173 29185 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.171429 26323 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:25.191345 26255 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:25.133945 27126 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:25.164264 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:25.191602 26406 rpcz_store.cc:276] 0924 14:30:23.934641 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:23.934771 (+   130us) service_pool.cc:227] Handling call
0924 14:30:23.935065 (+   294us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:24.037556 (+102491us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:24.037602 (+    46us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:24.037663 (+    61us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:24.037715 (+    52us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:25.133691 (+1095976us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54001e46d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:25.133788 (+    97us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:25.134086 (+   298us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":234436352}
I20260924 14:30:25.125792 29196 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:25.201488 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 98101.8us
I20260924 14:30:25.191283 29328 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.204881 26399 raft_consensus.cc:2379] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:25.206530 29328 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.207335 29185 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:25.222990 29458 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:25.223935 29458 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:25.208316 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.228855 26253 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:25.230918 29265 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:25.231567 29265 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:25.233541 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:25.233851 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.234473 27189 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
W20260924 14:30:25.235064 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 101546us
I20260924 14:30:25.235755 29485 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:25.236312 29485 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:25.236791 29485 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:25.237150 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:25.232692 29265 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.238679 29052 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:25.240938 29265 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:25.241369 29052 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:25.241946 29052 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:25.242594 29457 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:30:25.243192 29185 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:25.244006 29457 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:25.243934 29185 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:25.244457 27126 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:25.244457 29457 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:25.246287 29457 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:25.248067 29288 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:25.248525 29288 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:25.248872 29288 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:25.249305 26080 catalog_manager.cc:5495] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:25.256040 26325 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:25.191881 29108 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:25.260715 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.262209 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.264488 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:25.263154 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.269356 26404 raft_consensus.cc:2396] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:25.227083 29458 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:25.114305 29292 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:25.278697 26332 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:25.279820 29423 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:25.280509 29423 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:25.271591 26297 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.702s (SENT) [suppressed 10 similar messages]
I20260924 14:30:25.285379 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.288352 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:25.289499 29260 consensus_meta.cc:287] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.967s	user 0.007s	sys 0.015s
I20260924 14:30:25.290239 29419 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:25.262585 26080 catalog_manager.cc:5495] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 4 to 6, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 6 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:25.290796 29419 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:25.293045 26080 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 2 to 4, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 4 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:25.295979 29419 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:25.299052 27191 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260924 14:30:25.300983 26078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 5) took 20201 ms (client timeout 15000 ms). Trace:
I20260924 14:30:25.274113 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.277344 29457 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:25.301328 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.303581 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.305166 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.306555 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.309376 29458 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:25.310225 29458 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:25.310846 29458 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:25.309402 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.312352 27188 raft_consensus.cc:2379] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:25.312767 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1094) took 1170 ms. Trace:
I20260924 14:30:25.316818 29185 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.320878 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.323225 27189 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:25.329694 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.313126 26398 rpcz_store.cc:276] 0924 14:30:24.142017 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:25.019868 (+877851us) service_pool.cc:227] Handling call
0924 14:30:25.081894 (+ 62026us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:25.082538 (+   644us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:25.082567 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:25.083159 (+   592us) log.cc:842] Serialized 26 byte log entry
0924 14:30:25.189384 (+106225us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:25.189422 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:25.189487 (+    65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:25.312580 (+123093us) raft_consensus.cc:1671] finished
0924 14:30:25.312597 (+    17us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:25.312709 (+   112us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1708,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":227,"thread_start_us":85328,"threads_started":1,"wal-append.queue_time_us":227400}
I20260924 14:30:25.331911 29501 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:25.335306 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.329500 29288 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:25.328997 29485 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:25.330120 29052 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.332530 29501 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.341084 29185 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260924 14:30:25.376295 27190 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:25.372977 29501 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.380503 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:25.385016 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 193138us
I20260924 14:30:25.386338 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:25.389232 26406 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:25.395618 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:25.394625 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:25.404436 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:25.405026 26404 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:25.377218 29466 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.413094 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.413599 27191 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
W20260924 14:30:25.315141 26078 rpcz_store.cc:269] 0924 14:30:05.108957 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:05.109276 (+   319us) service_pool.cc:227] Handling call
0924 14:30:25.300568 (+20191292us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:30:18.379498 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:30:18.474178 (+ 94680us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:30:18.474217 (+    39us) write_op.cc:435] Acquired schema lock
0924 14:30:18.474234 (+    17us) tablet.cc:660] Decoding operations
0924 14:30:18.474894 (+   660us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:30:18.474976 (+    82us) write_op.cc:641] Partition lock acquired for write op
0924 14:30:18.475005 (+    29us) tablet.cc:683] Acquiring locks for 4 operations
0924 14:30:18.475407 (+   402us) tablet.cc:699] Row locks acquired
0924 14:30:18.475427 (+    20us) write_op.cc:260] PREPARE: finished
0924 14:30:18.475592 (+   165us) write_op.cc:270] Start()
0924 14:30:18.475703 (+   111us) write_op.cc:276] Timestamp: P: 1790260218475544 usec, L: 0
0924 14:30:18.475728 (+    25us) op_driver.cc:331] REPLICATION: starting
0924 14:30:18.476341 (+   613us) log.cc:842] Serialized 1717 byte log entry
0924 14:30:18.677656 (+201315us) op_driver.cc:444] REPLICATION: finished
0924 14:30:19.023225 (+345569us) write_op.cc:301] APPLY: starting
0924 14:30:19.026761 (+  3536us) tablet.cc:1365] starting BulkCheckPresence
0924 14:30:19.062512 (+ 35751us) tablet.cc:1368] finished BulkCheckPresence
0924 14:30:19.115418 (+ 52906us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:30:20.214580 (+1099162us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b5c00058d0c. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf6d7501 00007f09bf6d73f1 00007f09bf6d212c 00007f09bcc7891b 00007f09bcc785e3 00007f09bcd72b14 00007f09bcd5edfa 00007f09bcd62dd1 00007f09bcd62d89 00007f09bcd62d19 00007f09bcd62ce1
0924 14:30:20.214807 (+   227us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:30:20.215002 (+   195us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0924 14:30:20.215058 (+    56us) write_op.cc:312] APPLY: finished
0924 14:30:20.215286 (+   228us) log.cc:842] Serialized 40 byte log entry
0924 14:30:20.522783 (+307497us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:20.534317 (+ 11534us) write_op.cc:454] Released schema lock
0924 14:30:20.537142 (+  2825us) write_op.cc:341] FINISH: Updating metrics
0924 14:30:20.627837 (+ 90695us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:20.633071 (+  5234us) write_op.cc:454] Released schema lock
0924 14:30:21.084353 (+451282us) spinlock_profiling.cc:243] Waited 4.63 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
Metrics: {"mutex_wait_us":60264,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":320349,"apply.run_cpu_time_us":74049,"apply.run_wall_time_us":1618016,"num_ops":4,"prepare.queue_time_us":367434,"prepare.run_cpu_time_us":3633,"prepare.run_wall_time_us":117993,"replication_time_us":193314,"spinlock_wait_cycles":17856768,"thread_start_us":726424,"threads_started":3,"wal-append.queue_time_us":499427,"wal-append.run_cpu_time_us":6107,"wal-append.run_wall_time_us":1463649}]]}
I20260924 14:30:25.402796 29288 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:25.410940 29259 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:25.426772 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.428992 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:25.429260 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:25.429883 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:25.435753 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.435911 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.438292 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:25.435786 26329 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:25.441339 29423 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.441138 29185 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260924 14:30:25.441983 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 81533.9us
I20260924 14:30:25.443277 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.444398 29262 consensus_queue.cc:1046] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:30:25.445344 29262 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:25.446146 29262 raft_consensus.cc:3032] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:25.446547 29262 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:25.427569 29259 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:25.447744 29262 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.448441 29259 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.449084 29262 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:25.433530 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:25.444585 29292 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:25.458132 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1096) took 1297 ms. Trace:
I20260924 14:30:25.458305 26406 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:25.469545 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:25.472361 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 53372.2us
I20260924 14:30:25.474052 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:25.461359 26400 rpcz_store.cc:276] 0924 14:30:24.160885 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:25.026577 (+865692us) service_pool.cc:227] Handling call
0924 14:30:25.027280 (+   703us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:25.180062 (+152782us) raft_consensus.cc:1520] Early marking committed up to index 2
0924 14:30:25.180097 (+    35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:25.180611 (+   514us) log.cc:842] Serialized 26 byte log entry
0924 14:30:25.313949 (+133338us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:25.314097 (+   148us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:25.314286 (+   189us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:25.454273 (+139987us) raft_consensus.cc:1671] finished
0924 14:30:25.454291 (+    18us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:25.458065 (+  3774us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":8268,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":127,"thread_start_us":132964,"threads_started":1,"wal-append.queue_time_us":209745}
I20260924 14:30:25.471849 27126 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:25.478472 29340 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260924 14:30:25.481004 29506 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260924 14:30:25.458556 29292 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:25.482307 29292 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.483995 29292 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:25.484355 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.492600 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.464819 29052 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.496863 29151 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:25.497375 27191 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:25.497664 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.497980 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:25.498644 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:25.499091 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:25.499807 29447 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:25.497709 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.498518 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.494562 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.514714 27182 raft_consensus.cc:2396] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260924 14:30:25.515815 29151 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:25.515931 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:25.516817 29151 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:25.517769 29292 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:25.517546 26295 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:25.476791 29485 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:25.519294 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.518497 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.523075 26402 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:25.535692 27125 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:25.528693 29186 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:25.530705 29260 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:25.533075 29052 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:25.541131 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.546077 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:25.533933 29292 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:25.548892 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:25.550140 27182 raft_consensus.cc:2379] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:25.550163 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.553016 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:25.553882 26249 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:25.559475 26255 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:25.542124 29186 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:25.544167 29260 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:25.561965 26406 raft_consensus.cc:2396] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:25.545879 29466 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:25.568374 29300 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:25.568749 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 78145.5us
I20260924 14:30:25.569063 29300 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.569774 26369 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:25.570771 26369 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:25.571552 29419 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:25.572018 29419 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:25.572397 29419 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:25.572322 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.573429 26249 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:25.571617 29458 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:30:25.579667 29458 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:25.581867 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.579159 26404 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:25.563215 29186 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:25.585461 26406 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:25.564436 29260 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:25.589041 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:25.590075 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:25.592288 26256 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 8. Ops: []
I20260924 14:30:25.587056 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.588444 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.592120 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:25.595398 26255 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:25.565586 29383 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:25.596581 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.562950 29367 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260924 14:30:25.604104 26258 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:25.600795 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:25.612958 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:25.614627 26402 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:25.612099 29485 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:25.617161 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:25.617460 29510 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260924 14:30:25.625185 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.600544 29300 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:25.622743 26257 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:25.624578 29383 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.635543 29383 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:25.637408 26402 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:25.605093 29292 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.594803 29458 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.641794 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:25.653290 26249 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:25.655678 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.658130 27186 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:30:25.661890 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.663005 26406 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:25.657687 26398 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:30:25.665551 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.666312 26254 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:25.672322 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.673091 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.673293 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.673643 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.673761 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.674726 29300 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:25.675824 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:25.677016 27126 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:25.687799 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:25.689039 29427 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:25.689682 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:25.690548 29427 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:25.692061 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.694245 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:25.705762 29185 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:25.706336 29185 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:25.707197 29185 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.707944 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.708467 29185 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:25.708772 29300 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:25.710392 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.710435 29260 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.712502 29300 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:25.712743 29260 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:25.717278 29300 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:25.720753 29367 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:30:25.721123 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 71313.3us
I20260924 14:30:25.721773 26258 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:25.722829 29447 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader election won for term 4
I20260924 14:30:25.724239 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.724509 29447 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:25.725521 29447 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:25.726116 29300 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:25.730243 29419 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:25.740064 26255 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:25.739470 29501 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:25.745141 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:25.754456 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.755344 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:25.744125 29501 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.758327 29458 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260924 14:30:25.759815 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 67881.7us
I20260924 14:30:25.760620 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:25.761322 29458 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:25.762832 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:25.763171 29427 consensus_queue.cc:1046] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:25.763844 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:25.756084 29186 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.756595 26404 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260924 14:30:25.741647 29260 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:25.764573 29425 raft_consensus.cc:2741] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260924 14:30:25.766803 29300 raft_consensus.cc:2741] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260924 14:30:25.778633 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:25.800195 29442 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:25.800871 29419 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:25.800894 29442 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.801514 29419 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:25.781499 29185 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:25.803985 29185 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:25.805286 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.805847 29185 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.810215 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:25.803292 29442 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:25.811839 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 43625.4us
I20260924 14:30:25.814019 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.814873 26255 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:25.816915 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:25.779757 29260 raft_consensus.cc:3032] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:25.782600 29501 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:25.823423 29383 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:25.821324 29260 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:25.822933 29442 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:25.835594 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:25.823355 29367 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:25.830488 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:25.845968 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 81300.2us
I20260924 14:30:25.850063 29185 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:25.856604 29260 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:25.858354 29260 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:25.844516 29442 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:25.867666 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 101411us
I20260924 14:30:25.870932 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:25.871562 27188 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:25.877599 29515 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.133s	user 0.002s	sys 0.000s
I20260924 14:30:25.880404 27188 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:25.881892 29525 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:25.883818 26295 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:25.890322 26369 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:25.893648 29383 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:25.893781 29525 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:25.894538 29383 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:25.894547 29525 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:25.894886 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:25.895372 29525 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:25.895745 27188 raft_consensus.cc:595] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:25.896312 27188 raft_consensus.cc:603] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:25.895681 29383 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.902172 29186 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:25.907174 29466 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:25.907893 29466 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:25.869841 29442 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:25.914127 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:25.910140 29466 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:25.925705 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0756687d99e643c094531124c9438376"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:25.927214 29425 consensus_queue.cc:1046] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:25.928304 29425 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:30:25.928661 29425 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:25.929563 29425 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:25.930197 27191 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260924 14:30:25.930567 27191 pending_rounds.cc:76] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 5
I20260924 14:30:25.930749 29425 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:25.931918 26221 leader_election.cc:398] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6.
I20260924 14:30:25.932192 29151 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:25.932482 26221 leader_election.cc:401] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260924 14:30:25.932780 29151 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.933404 29300 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6.
I20260924 14:30:25.934813 29300 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:25.930809 27191 pending_rounds.cc:97] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260924 14:30:25.935097 29151 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:25.936087 27191 raft_consensus.cc:2866] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:25.926072 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:25.941372 27188 raft_consensus.cc:595] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:25.943116 28867 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:30:25.943519 29517 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.170s	user 0.000s	sys 0.002s
I20260924 14:30:25.945817 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:25.949636 26398 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:25.951256 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.952380 29151 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260924 14:30:25.954913 29516 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260924 14:30:25.955834 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.964031 29485 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:25.965608 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.967674 29485 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:25.968540 26298 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:25.984930 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:25.985301 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:25.990203 29300 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:25.990756 29300 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:25.991966 29300 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:25.993860 29300 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
W20260924 14:30:26.000927 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 65662.6us
I20260924 14:30:26.000840 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:26.001950 26332 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:26.004298 26298 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:25.995751 27188 raft_consensus.cc:595] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:26.011756 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:26.012142 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:26.023666 26298 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:25.986642 29185 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:26.013666 29151 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:26.033620 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:26.042114 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:26.036386 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:26.011068 29367 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:26.045109 26324 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:25.996533 29485 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260924 14:30:26.023792 29375 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:30:26.047410 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 104745us
I20260924 14:30:26.026961 29466 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:26.051148 26328 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:26.053767 29203 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:26.054701 29203 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:26.056469 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:26.057072 29203 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:26.057272 27188 raft_consensus.cc:595] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:26.054208 26326 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:26.059222 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:26.062809 26332 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:26.078759 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:26.082803 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 46490.4us
I20260924 14:30:26.083066 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:26.086724 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 73730.7us
I20260924 14:30:26.086675 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:26.088380 29532 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:26.080376 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:26.092568 29151 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:26.082748 29340 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:26.093482 26328 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:26.093802 29340 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:26.093801 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:26.107095 29442 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:26.116513 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:26.096192 29340 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:26.108148 29525 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:26.128379 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:26.129272 27182 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:26.129799 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:26.129981 29367 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:26.131567 29419 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:26.132109 29419 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:26.118543 29185 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:26.071978 29466 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:26.135066 27186 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:26.135828 27186 raft_consensus.cc:595] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:26.142774 26371 leader_election.cc:334] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.359s (SENT)
I20260924 14:30:26.143342 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:26.143057 29524 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:26.144955 29524 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:26.123063 29383 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.027s
I20260924 14:30:26.150198 26398 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:26.154817 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:26.149299 29262 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260924 14:30:26.142887 29427 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260924 14:30:26.149526 29523 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260924 14:30:26.160969 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:26.161502 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:26.162676 29538 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:30:26.140105 29466 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:26.162710 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:26.165627 29292 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:26.166954 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:26.165898 29540 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:26.168069 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:26.168835 27189 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:26.133882 29185 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:26.172639 26404 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:26.163057 29536 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:26.187902 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:26.191318 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:26.193742 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:30:26.142879 29419 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:26.120505 29413 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260924 14:30:26.205058 26332 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:26.210041 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:26.211491 29419 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:26.212191 29419 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:26.213361 29419 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:26.215291 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:26.216239 26324 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:26.198272 29534 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:26.194346 29525 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:26.238222 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:26.245003 26255 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:30:26.227260 29288 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:26.233558 29151 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:26.250102 29534 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:30:26.276991 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 85786.7us
I20260924 14:30:26.278533 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:26.279091 26254 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:26.234207 29185 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:26.280141 29442 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:26.293624 29501 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:30:26.343880 29259 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:26.267572 29151 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:26.345856 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 60196.9us
W20260924 14:30:26.346688 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 57743.9us
I20260924 14:30:26.347203 29526 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:26.378727 26398 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:26.379050 26400 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:30:26.381834 29151 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:26.383883 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:26.384984 29413 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:26.386029 29526 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:26.386494 29526 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:26.387400 29526 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:26.388803 29526 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:26.390161 29413 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:26.390605 29413 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:26.391463 29413 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:26.392693 29413 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:26.399513 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:26.356979 26324 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:26.361590 26254 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:26.415685 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:26.417776 26221 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:26.418876 29260 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:26.432291 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:26.433193 26331 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:26.449887 27182 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260924 14:30:26.464345 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 96311.8us
I20260924 14:30:26.465860 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:26.467041 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:26.469754 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 187455us
W20260924 14:30:26.474857 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 56027.1us
I20260924 14:30:26.474164 26331 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:26.468362 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:30:26.477434 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:26.481586 27191 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260924 14:30:26.482885 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:26.498720 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 85508.5us
I20260924 14:30:26.518756 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:26.477268 29300 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:26.476927 29540 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260924 14:30:26.521934 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 54942.2us
I20260924 14:30:26.518956 26369 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:26.565156 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 47543.4us
W20260924 14:30:26.571111 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 80112.9us
W20260924 14:30:26.573459 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 90643.2us
I20260924 14:30:26.580032 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:26.483224 29259 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260924 14:30:26.568449 29442 raft_consensus.cc:979] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
W20260924 14:30:26.569064 29425 consensus_meta.cc:287] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.638s	user 0.010s	sys 0.003s
W20260924 14:30:26.583953 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106626us
I20260924 14:30:26.589939 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:26.596882 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:26.597867 26255 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
W20260924 14:30:26.589694 29546 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.248s	user 0.000s	sys 0.003s
I20260924 14:30:26.583087 29108 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:26.582861 29265 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260924 14:30:26.564775 29292 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260924 14:30:26.628733 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:26.590138 29540 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:26.589936 29559 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:26.605610 29151 raft_consensus.cc:1067] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:26.621565 29538 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election won for term 12
W20260924 14:30:26.685566 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 65297.2us
I20260924 14:30:26.604346 29533 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:26.640519 29438 raft_consensus.cc:979] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
W20260924 14:30:26.641098 29552 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260924 14:30:26.691174 29541 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:26.693346 29300 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:30:26.695786 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:26.701522 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 100884us
I20260924 14:30:26.685032 29108 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:26.698073 29559 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:26.715427 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:26.716785 29555 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260924 14:30:26.712632 29447 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:26.708935 29052 raft_consensus.cc:1067] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:26.718037 27182 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:26.708966 29466 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.001s
W20260924 14:30:26.727157 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 125512us
I20260924 14:30:26.730422 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:26.741936 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:26.761059 26252 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 5: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:30:26.726752 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 121608us
I20260924 14:30:26.763293 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:26.764906 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:26.765496 26256 raft_consensus.cc:2362] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6. Re-sending same reply.
I20260924 14:30:26.768009 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:26.770447 26257 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:26.773578 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:26.709862 29538 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:26.776201 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:26.776502 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:26.776957 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:26.754470 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 100785us
I20260924 14:30:26.785248 26255 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:26.776134 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:26.789975 27186 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:26.797127 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:26.804275 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:26.753118 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 161181us
I20260924 14:30:26.801214 26251 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:27.047670 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:27.048979 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:27.050159 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:26.807891 26257 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:27.052251 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:27.053288 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:27.054144 26257 raft_consensus.cc:2396] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:27.054940 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:26.884184 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 172088us
I20260924 14:30:26.884558 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:27.056481 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:27.057729 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:27.059029 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:26.755468 29447 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:27.060105 29447 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:26.777194 29340 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:26.711766 29541 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:27.057617 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:27.065614 29541 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:27.066183 29541 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:27.066650 29541 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:26.919337 26252 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260924 14:30:26.923298 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 117483us
I20260924 14:30:26.775235 29466 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:26.949291 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:26.948709 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:27.076067 26255 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:26.942039 26253 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
W20260924 14:30:26.928809 29556 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
W20260924 14:30:26.957572 29567 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
W20260924 14:30:26.970137 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 75805.5us
I20260924 14:30:27.084586 26331 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:27.085498 29466 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:26.976488 26258 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:26.928303 29108 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:27.031334 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 222198us
I20260924 14:30:27.033962 29300 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:30:27.090591 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:27.091805 29300 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:27.092818 29260 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:27.093905 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:27.094432 29260 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:27.034181 29540 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:26.989843 29538 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:27.095299 29540 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:27.095686 29540 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:27.096490 29540 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:26.950415 29559 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:27.068308 26323 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260924 14:30:27.068152 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:27.106319 26325 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:27.069350 26326 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:27.051311 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:27.077690 26253 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:27.107402 26253 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:27.079766 26257 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260924 14:30:27.089704 29526 consensus_meta.cc:287] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.700s	user 0.011s	sys 0.001s
I20260924 14:30:27.093027 29300 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:26.886482 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 95807.6us
I20260924 14:30:27.099886 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:27.097826 29540 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:27.068365 26332 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260924 14:30:27.108351 26254 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:27.113597 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:27.107012 26251 raft_consensus.cc:2362] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5. Re-sending same reply.
I20260924 14:30:27.116319 26258 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0756687d99e643c094531124c9438376"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
I20260924 14:30:27.117033 26258 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:27.117404 26258 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:27.117447 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:27.119382 26297 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:27.127696 29383 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:27.183689 26323 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:27.120486 29466 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:27.123780 29586 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:27.120347 29591 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:27.204674 26404 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:27.121508 29423 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260924 14:30:27.205611 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:27.188737 29383 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:27.223845 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:27.261647 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 134886us
I20260924 14:30:27.273180 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:27.191926 29300 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:27.204883 29108 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:27.228165 29536 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260924 14:30:27.270875 29534 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:27.324857 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 147875us
I20260924 14:30:27.255688 26258 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:27.271641 29186 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:27.260499 29383 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:27.378886 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:26.565249 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:27.431187 26258 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:27.432698 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 159628us
I20260924 14:30:27.434355 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:27.440634 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61863.5us
W20260924 14:30:27.446137 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 209062us
W20260924 14:30:27.447469 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 196398us
I20260924 14:30:27.306680 29526 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260924 14:30:27.372676 29581 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:27.383172 29523 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:27.492427 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:27.382658 29427 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:27.375170 29574 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:27.431799 29532 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:27.340408 29108 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:27.505375 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:27.461607 29466 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:27.518101 29146 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:27.453264 29541 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:27.512683 29485 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:27.533052 27186 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:27.511351 29532 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:27.540860 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 106628us
I20260924 14:30:27.543700 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:27.520200 29466 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:27.567090 27190 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:27.521327 29526 consensus_queue.cc:1225] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260924 14:30:27.563233 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:27.525699 29185 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:27.532467 29534 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:27.583079 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 90746.7us
I20260924 14:30:27.568768 29608 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260924 14:30:27.573515 29582 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260924 14:30:27.460161 29598 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.099s
I20260924 14:30:27.574312 29108 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:27.574707 29609 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:27.504917 29581 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:27.581581 26327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1129) took 1529 ms (client timeout 1967 ms). Trace:
I20260924 14:30:27.514375 29574 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:27.558208 29586 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:27.510993 29523 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:27.600106 26253 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:27.571888 29466 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:27.584290 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:27.605538 26327 rpcz_store.cc:269] 0924 14:30:26.049084 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:26.049323 (+   239us) service_pool.cc:227] Handling call
0924 14:30:27.578365 (+1529042us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54005807d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:27.578539 (+   174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2050417536}
I20260924 14:30:27.601434 26077 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:27.547989 29485 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:27.619786 26255 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:27.539656 29146 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:27.636797 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 91184.6us
I20260924 14:30:27.643566 26253 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:27.652742 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:27.607970 29575 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:27.661067 26250 consensus_meta.cc:287] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.547s	user 0.011s	sys 0.001s
W20260924 14:30:27.664608 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 203830us
W20260924 14:30:27.677223 26256 consensus_meta.cc:287] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.000s
I20260924 14:30:27.661649 26250 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:27.696411 26256 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
W20260924 14:30:27.671852 29538 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.017s	sys 0.011s
W20260924 14:30:27.717864 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 125447us
W20260924 14:30:27.670769 29533 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.012s	sys 0.008s
I20260924 14:30:27.592118 29534 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:27.621466 29609 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260924 14:30:27.678280 29575 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:27.610080 29608 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.109s
W20260924 14:30:27.722644 29538 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.017s	sys 0.011s
I20260924 14:30:27.718391 29598 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:27.724509 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 80227.2us
I20260924 14:30:27.620472 29586 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:27.725652 29572 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.373s	user 0.001s	sys 0.000s
I20260924 14:30:27.652226 29587 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.230s
I20260924 14:30:27.736320 26253 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:27.724365 29458 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 14:30:27.757336 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 173140us
W20260924 14:30:27.730387 29533 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.013s	sys 0.009s
W20260924 14:30:27.749615 29580 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.360s	user 0.000s	sys 0.001s
I20260924 14:30:27.760955 29559 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:27.731419 29607 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:27.759107 29590 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.351s	user 0.001s	sys 0.000s
W20260924 14:30:27.763476 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 496617us
W20260924 14:30:27.771639 29447 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.711s	user 0.009s	sys 0.004s
I20260924 14:30:27.793102 26397 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:27.793547 29616 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:27.795646 29595 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260924 14:30:27.795958 29545 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:27.796134 29458 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:27.796344 29618 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:27.796764 29612 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:27.821735 29614 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260924 14:30:27.799106 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:27.842849 27182 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:27.765908 29598 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:27.803220 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:27.805027 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:27.808342 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:27.847509 27181 raft_consensus.cc:3032] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:27.809731 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:27.847894 27181 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:27.810628 29586 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:27.811589 26258 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260924 14:30:27.785411 29596 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
W20260924 14:30:27.822065 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61500.3us
I20260924 14:30:27.832672 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:27.854015 26326 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:27.833693 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:27.837472 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:27.796896 29559 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:27.840137 26404 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:27.840309 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:27.855515 26401 raft_consensus.cc:2379] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:27.855609 26327 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:27.842042 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:27.843150 27182 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:27.848189 26250 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:27.847561 26323 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:27.825356 29292 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:27.863466 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:27.858127 27181 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:27.865192 27181 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:27.840607 29536 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:27.840963 29581 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:27.857667 27182 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:27.870134 27182 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:27.870560 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:27.866662 29574 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:27.849495 29523 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:27.858470 29485 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:27.829347 29383 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:27.882803 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 55149us
I20260924 14:30:27.822384 29447 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:27.875217 29597 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.218s	user 0.001s	sys 0.001s
I20260924 14:30:27.825886 29616 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:27.841068 29458 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:27.896039 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1242) took 1124 ms. Trace:
I20260924 14:30:27.896725 26256 rpcz_store.cc:276] 0924 14:30:26.766534 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:26.767751 (+  1217us) service_pool.cc:227] Handling call
0924 14:30:27.890527 (+1122776us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b54007828d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:27.890651 (+   124us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":805409152}
I20260924 14:30:27.817869 29607 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:27.875391 29146 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:27.910270 27189 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
W20260924 14:30:27.915131 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 89832.3us
I20260924 14:30:27.916078 26325 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:27.850533 29612 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:27.920107 29612 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:27.923592 29575 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:27.925482 29447 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:27.927274 29538 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:27.927978 29611 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:27.928710 29621 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:27.929034 29616 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:27.929288 29621 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:27.931140 29574 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:27.867686 29292 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:27.943418 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:27.944404 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 73904.5us
I20260924 14:30:27.945395 26371 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:27.936113 27181 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:27.952847 26399 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:27.950845 26327 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:27.948825 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:27.910218 29609 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:27.844558 29618 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:27.952384 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:27.970491 26224 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:27.968590 26329 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
W20260924 14:30:27.948707 29540 consensus_meta.cc:287] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.830s	user 0.013s	sys 0.000s
I20260924 14:30:27.976181 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:27.927342 29458 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:27.974164 26401 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:27.933903 29621 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:27.874496 29536 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:27.936208 29628 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:27.949759 29538 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:27.931630 29534 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:27.986470 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1247) took 1732 ms. Trace:
I20260924 14:30:27.972669 29611 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:28.020936 29627 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:28.021286 29559 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:28.021600 29052 raft_consensus.cc:979] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:28.021875 29559 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:28.022198 29052 raft_consensus.cc:1067] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:27.950798 29466 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:28.033592 27182 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
W20260924 14:30:28.037266 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 59011.8us
I20260924 14:30:27.995586 29616 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:28.039233 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:27.999439 26402 rpcz_store.cc:276] 0924 14:30:26.253747 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:26.253933 (+   186us) service_pool.cc:227] Handling call
0924 14:30:26.254338 (+   405us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:26.260829 (+  6491us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:30:26.566371 (+305542us) log.cc:842] Serialized 14 byte log entry
0924 14:30:26.697429 (+131058us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:26.977259 (+279830us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b2c0013ed68. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:26.977710 (+   451us) log.cc:842] Serialized 26 byte log entry
0924 14:30:26.977884 (+   174us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:26.978183 (+   299us) log.cc:842] Serialized 14 byte log entry
0924 14:30:26.989098 (+ 10915us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:26.989212 (+   114us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:27.986221 (+997009us) raft_consensus.cc:1671] finished
0924 14:30:27.986250 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:27.986398 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":325252,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":1718,"raft.queue_time_us":13645,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":13506,"spinlock_wait_cycles":2314112,"thread_start_us":314951,"threads_started":2,"wal-append.queue_time_us":531947}
I20260924 14:30:28.051884 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:27.992581 29458 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:27.969359 29574 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:28.003396 29540 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:28.058332 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.066589 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.021920 29622 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
W20260924 14:30:28.074376 29632 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260924 14:30:28.080358 29615 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:28.081028 29615 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:28.081530 29615 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:28.081601 29423 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:28.085764 29423 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:28.089051 29423 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:28.095849 29636 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:28.096837 29636 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:28.100952 29259 raft_consensus.cc:979] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:28.101596 29259 raft_consensus.cc:1067] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:28.107350 29607 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:28.108917 29536 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.109830 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.110577 26397 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:28.110795 29383 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:28.110927 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.111534 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.111791 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.111979 29637 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:28.112567 29637 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:28.113022 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.113659 26399 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:28.114316 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.114735 29637 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:28.115617 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:28.115375 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.116465 29292 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.116801 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.118394 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:28.096181 29594 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.790s	user 0.000s	sys 0.005s
I20260924 14:30:28.118732 29466 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:28.119009 27182 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:28.120507 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:28.120959 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.121462 29466 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:28.121850 29466 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:28.123128 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.123349 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.123764 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.124428 27185 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:28.123924 27181 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:28.116397 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.128196 26326 raft_consensus.cc:2396] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260924 14:30:28.122193 29466 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:28.060456 29540 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:28.003126 29538 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:28.057612 29616 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:28.140848 27126 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:28.068606 29598 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:28.001551 29628 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:28.058241 29625 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
W20260924 14:30:28.151379 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 175310us
I20260924 14:30:28.152320 29607 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:28.152524 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:28.152900 29607 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:28.153290 29628 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.154191 29628 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:28.154292 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:28.155086 29607 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.155126 29611 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:28.156073 29618 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:28.156762 29645 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:28.148826 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.156860 29628 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:28.143194 29540 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:28.158262 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:28.185232 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.185942 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.192332 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:28.193276 26252 raft_consensus.cc:2396] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:28.193120 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:28.021679 29627 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:28.172122 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:28.172876 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.172580 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.179987 26323 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:28.063054 29458 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:28.198134 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 50068.5us
I20260924 14:30:28.200045 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:28.205322 26331 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:28.213547 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:28.216645 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.217026 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:28.217648 26249 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:28.219123 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:28.220041 26253 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:28.214566 26254 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:28.223089 29260 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:28.223742 29627 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:28.223667 29260 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:28.225133 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:28.224661 29466 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:28.225652 29260 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.226261 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:28.228411 29640 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:28.229084 29640 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:28.230260 29640 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:28.231657 29618 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:28.239553 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:28.232270 29618 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:28.246665 29636 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.247327 29553 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:28.247820 29553 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:28.248939 26406 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:28.248102 29618 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:28.250162 29574 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:28.252105 29458 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:28.253540 29573 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:28.254117 29573 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:28.254520 29573 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:28.256610 29648 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:28.200987 29598 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:28.257288 29648 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:28.257843 29598 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:28.260433 29598 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:28.260819 29616 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:28.262537 29466 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.271236 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:28.246567 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 88379.5us
I20260924 14:30:28.271966 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:28.274893 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:28.258435 29538 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.276144 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.277094 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.278200 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.245088 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:28.319821 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 80405us
I20260924 14:30:28.220876 29545 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:28.324496 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.331102 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:28.244887 29627 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:28.338375 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 93374.1us
I20260924 14:30:28.337024 26330 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:28.248441 29553 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:28.335065 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.284202 29622 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
W20260924 14:30:28.341279 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 66916.8us
I20260924 14:30:28.281744 29534 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:28.365844 27123 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:28.283160 29636 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:28.370158 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.369660 26249 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:28.371840 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:28.374825 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 202777us
I20260924 14:30:28.376616 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.324765 29533 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:28.375377 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.381661 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.340112 29340 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:28.336195 29538 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:28.390684 26401 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:30:28.389454 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.396168 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.388908 26253 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:28.401480 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:28.283373 29526 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260924 14:30:28.340451 29423 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:28.360963 29645 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:28.405845 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.406818 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.407974 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.413826 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:28.362550 29607 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:28.418807 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:28.446219 26405 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260924 14:30:28.397722 29635 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260924 14:30:28.447733 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:28.465304 26323 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:28.465462 26249 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:28.460601 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:28.373795 29636 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:28.365141 29648 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.463662 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.464545 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:28.480581 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:28.480894 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.480581 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.498071 26254 raft_consensus.cc:2396] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260924 14:30:28.500885 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.501405 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:28.502609 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.397982 29340 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:28.495970 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:28.507799 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:28.405726 29260 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:28.514238 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:28.513513 29648 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:28.446890 29607 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:28.517733 26323 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:28.518383 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:28.512861 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 88411.2us
I20260924 14:30:28.380231 29534 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:28.523221 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:28.523262 26252 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
I20260924 14:30:28.525309 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:28.498350 29636 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:28.592316 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.515620 29340 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:28.595798 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:28.572129 26253 raft_consensus.cc:2396] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260924 14:30:28.521426 29616 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:28.523495 29607 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:28.482297 29423 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:28.609078 26252 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:28.609627 26252 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:28.607697 29647 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:28.593218 29260 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:28.617166 29605 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:28.445992 29533 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:28.618413 29592 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.245s	user 0.000s	sys 0.014s
I20260924 14:30:28.593261 29536 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:28.724232 29536 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:28.726970 29536 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.727960 26249 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:28.623612 29661 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:28.624063 29643 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:28.624468 29646 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:28.630074 29534 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:28.462689 29645 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:28.632473 29642 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:28.720242 26257 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:28.597362 29538 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:28.731563 26253 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
I20260924 14:30:28.843241 26253 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:28.843676 26253 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:28.605715 29616 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:28.811425 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.819566 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.728955 29647 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:28.791847 29646 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:28.875169 26400 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:28.886515 29655 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:28.887130 29655 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:28.861912 29642 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:28.889283 29655 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:28.890584 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.898068 29423 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.898993 29645 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.900436 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:28.900780 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:28.901611 27192 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:28.901196 26404 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 2.
I20260924 14:30:28.890723 29642 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:28.732770 26252 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:28.880558 29615 consensus_meta.cc:287] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.795s	user 0.012s	sys 0.000s
I20260924 14:30:28.998962 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:28.873184 29616 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:29.000496 26224 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:28.871629 29640 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:29.014791 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:29.007947 26253 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.015828 26328 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
W20260924 14:30:29.045360 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45040.7us
I20260924 14:30:28.884317 29646 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:28.871238 29538 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:29.085933 26253 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:28.734786 29533 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:29.087769 29540 consensus_meta.cc:287] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.870s	user 0.010s	sys 0.001s
W20260924 14:30:29.118050 27125 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.000s	sys 0.021s
W20260924 14:30:29.118571 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.000s	sys 0.021s
I20260924 14:30:29.078935 29637 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:28.820358 29643 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:29.042182 29642 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:28.792465 29661 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.131112 26256 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:29.138674 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.018s	sys 0.010s
W20260924 14:30:29.139092 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.018s	sys 0.010s
I20260924 14:30:29.157241 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:29.015360 29615 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.079633 29640 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:29.164266 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:29.144516 29618 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.010s	sys 0.017s
W20260924 14:30:29.168787 29618 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.010s	sys 0.018s
I20260924 14:30:29.168835 29645 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:29.170555 26256 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:29.125883 29637 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:29.129361 29660 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:29.170866 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.002s	sys 0.027s
W20260924 14:30:29.209012 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.004s	sys 0.033s
I20260924 14:30:29.218122 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:29.166213 29609 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:29.204005 29637 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:29.130769 29540 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:29.140774 29642 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:29.244132 26404 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
W20260924 14:30:29.242316 29612 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.026s	sys 0.050s
W20260924 14:30:29.298456 29612 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.026s	sys 0.051s
I20260924 14:30:29.298162 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:29.300450 29665 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260924 14:30:29.302510 29657 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 14:30:29.302985 29657 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:29.303323 29657 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:29.304057 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:29.265902 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:29.306526 29634 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:29.307154 29634 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.267336 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:29.309715 26253 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:29.244556 29538 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:29.241783 29609 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:29.263345 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1011) took 2166 ms. Trace:
I20260924 14:30:29.284759 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1125) took 1170 ms. Trace:
I20260924 14:30:29.314810 26406 rpcz_store.cc:276] 0924 14:30:28.113919 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.115107 (+  1188us) service_pool.cc:227] Handling call
0924 14:30:29.284528 (+1169421us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54002fc1d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:29.284664 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":304218368}
I20260924 14:30:29.316244 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:29.279161 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1146) took 1451 ms. Trace:
W20260924 14:30:29.281282 27186 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 893) took 1777 ms (client timeout 1697 ms). Trace:
W20260924 14:30:29.256520 29340 consensus_meta.cc:287] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.648s	user 0.004s	sys 0.008s
W20260924 14:30:29.318183 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52373.9us
W20260924 14:30:29.298229 29659 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260924 14:30:29.314311 26255 rpcz_store.cc:276] 0924 14:30:27.096826 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:27.099376 (+  2550us) service_pool.cc:227] Handling call
0924 14:30:27.099827 (+   451us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:27.620887 (+521060us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:29.263071 (+1642184us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54002a1554. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:29.263284 (+   213us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1177218944}
I20260924 14:30:29.316897 26325 rpcz_store.cc:276] 0924 14:30:27.827506 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:27.827651 (+   145us) service_pool.cc:227] Handling call
0924 14:30:27.843917 (+ 16266us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:27.937141 (+ 93224us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:29.278879 (+1341738us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b54007cecd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:29.279077 (+   198us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":154295168}
W20260924 14:30:29.317178 27186 rpcz_store.cc:269] 0924 14:30:27.504087 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:27.505116 (+  1029us) service_pool.cc:227] Handling call
0924 14:30:29.280999 (+1775883us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b5400453b54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:29.281152 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":45128064}
I20260924 14:30:29.231676 29618 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260924 14:30:29.254397 29668 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:29.259406 29651 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.625s	user 0.000s	sys 0.001s
I20260924 14:30:29.330488 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:29.316296 29609 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:29.336866 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:29.223531 29660 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:29.317890 29186 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:29.149999 29647 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.321488 29538 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:29.322284 29203 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:29.339926 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:29.343863 27185 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:29.346786 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:29.333017 29540 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:29.358075 27185 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:29.338824 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:29.358280 26257 raft_consensus.cc:2362] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11. Re-sending same reply.
I20260924 14:30:29.363651 26255 raft_consensus.cc:2362] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8. Re-sending same reply.
I20260924 14:30:29.364590 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:29.366169 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:29.294601 29534 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
W20260924 14:30:29.350867 29667 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260924 14:30:29.367584 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:29.369118 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:29.387414 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:29.364316 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:29.335747 29634 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:29.336679 29533 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:29.371212 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:29.362692 27185 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.436558 27185 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:29.337387 29606 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:29.430568 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:29.325812 29340 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:29.430799 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:29.431149 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:29.360847 29540 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:29.389053 27194 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:29.434372 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:29.381930 29108 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:29.436518 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 65396.7us
I20260924 14:30:29.357402 29609 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:29.436379 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1264) took 1244 ms. Trace:
I20260924 14:30:29.361678 29538 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:29.386880 29658 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.310s	user 0.000s	sys 0.001s
I20260924 14:30:29.363195 29618 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:29.438333 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 57505.6us
W20260924 14:30:29.443284 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 79057.4us
I20260924 14:30:29.339962 29668 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.388275 29203 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:29.446458 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:29.445390 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 55148us
I20260924 14:30:29.473536 26258 rpcz_store.cc:276] 0924 14:30:28.191584 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.192055 (+   471us) service_pool.cc:227] Handling call
0924 14:30:28.217708 (+ 25653us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:28.232912 (+ 15204us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:28.232955 (+    43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:28.242286 (+  9331us) log.cc:842] Serialized 26 byte log entry
0924 14:30:28.730174 (+487888us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:29.142930 (+412756us) log.cc:842] Serialized 14 byte log entry
0924 14:30:29.436033 (+293103us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:29.436115 (+    82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:29.436156 (+    41us) raft_consensus.cc:1671] finished
0924 14:30:29.436171 (+    15us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:29.436319 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17389,"prepare.queue_time_us":523828,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":155080,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"spinlock_wait_cycles":7751552,"thread_start_us":1084291,"threads_started":3,"wal-append.queue_time_us":862413}
I20260924 14:30:29.493971 29643 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:29.361452 29186 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:29.444578 29607 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.827s	user 0.008s	sys 0.003s
I20260924 14:30:29.391315 29185 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:29.386334 29602 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:29.437441 29655 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:29.543993 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:29.544553 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:29.442066 29423 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:29.387809 29581 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260924 14:30:29.562726 27192 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
W20260924 14:30:29.554175 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 348667us
I20260924 14:30:29.495009 29538 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:29.565932 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:29.571368 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:29.572178 26406 raft_consensus.cc:2396] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260924 14:30:29.545614 29675 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260924 14:30:29.556567 26326 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:29.555987 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 519) took 2039 ms. Trace:
I20260924 14:30:29.571955 26253 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:29.576347 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:29.442437 29533 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.551676 29607 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:29.563213 29636 consensus_meta.cc:287] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.954s	user 0.008s	sys 0.004s
I20260924 14:30:29.578105 27189 rpcz_store.cc:276] 0924 14:30:27.516890 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:27.522020 (+  5130us) service_pool.cc:227] Handling call
0924 14:30:27.522888 (+   868us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:27.937437 (+414549us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:29.555809 (+1618372us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400070654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:29.555940 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":890451328}
I20260924 14:30:29.546597 29672 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:29.584358 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:29.585242 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:29.582834 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 136470us
I20260924 14:30:29.622051 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:29.637573 26406 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:29.555223 29676 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:29.559104 29640 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:29.537573 29540 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.649204 26253 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260924 14:30:29.445451 29606 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:29.549527 29622 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260924 14:30:29.645247 26258 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:29.566741 29642 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:29.545787 29108 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:29.640654 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 74356.9us
I20260924 14:30:29.642445 29636 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:29.651739 29570 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:29.652926 29184 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:29.642486 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:29.643220 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:29.643615 26224 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:30:29.693094 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50647.6us
I20260924 14:30:29.659235 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1163) took 1063 ms. Trace:
W20260924 14:30:29.641026 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 56755.2us
I20260924 14:30:29.587105 29146 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:29.549194 29618 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:29.696743 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1164) took 1063 ms. Trace:
I20260924 14:30:29.693611 26323 rpcz_store.cc:276] 0924 14:30:28.595645 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.595761 (+   116us) service_pool.cc:227] Handling call
0924 14:30:28.608065 (+ 12304us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:28.608487 (+   422us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:28.608514 (+    27us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:30:28.728175 (+119661us) log.cc:842] Serialized 52 byte log entry
0924 14:30:29.169994 (+441819us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:29.170050 (+    56us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:29.170164 (+   114us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:29.658916 (+488752us) raft_consensus.cc:1671] finished
0924 14:30:29.658952 (+    36us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:29.659154 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1097,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":23589,"thread_start_us":441399,"threads_started":1,"wal-append.queue_time_us":609869}
I20260924 14:30:29.594162 29643 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:29.587662 29661 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:29.693182 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 49648.8us
I20260924 14:30:29.660702 26401 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:29.568318 29185 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:29.565904 29602 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.660156 29375 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:29.730911 26250 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:29.639552 29608 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260924 14:30:29.697149 26331 rpcz_store.cc:276] 0924 14:30:28.632962 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.633085 (+   123us) service_pool.cc:227] Handling call
0924 14:30:28.633438 (+   353us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:28.729398 (+ 95960us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:28.729443 (+    45us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:28.729502 (+    59us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:28.729559 (+    57us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:29.696518 (+966959us) spinlock_profiling.cc:243] Waited 95.7 ms on lock 0x7b54002774d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:29.696567 (+    49us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:29.696688 (+   121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":220084096}
I20260924 14:30:29.660512 29551 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:29.639817 29560 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:29.699023 29559 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:29.707201 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:29.731629 27185 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:29.688131 29526 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:29.718197 29553 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.344s	user 0.012s	sys 0.001s
W20260924 14:30:29.743862 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 100714us
I20260924 14:30:29.755892 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1127) took 1576 ms. Trace:
I20260924 14:30:29.692142 29642 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:29.758299 26253 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:29.697063 29640 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:29.695956 29622 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:29.725549 29570 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:29.775669 29570 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:29.776749 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:29.779182 26327 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:29.768700 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:29.769295 27192 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:29.767110 26398 rpcz_store.cc:276] 0924 14:30:28.179600 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.179767 (+   167us) service_pool.cc:227] Handling call
0924 14:30:28.180198 (+   431us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:28.180463 (+   265us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:28.180498 (+    35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:28.220066 (+ 39568us) log.cc:842] Serialized 26 byte log entry
0924 14:30:28.407725 (+187659us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:28.407787 (+    62us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:28.407881 (+    94us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:29.755673 (+1347792us) raft_consensus.cc:1671] finished
0924 14:30:29.755698 (+    25us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:29.755834 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":175161,"threads_started":1,"wal-append.queue_time_us":629209}
I20260924 14:30:29.780510 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:29.781116 26327 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:29.781301 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:29.745592 29678 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260924 14:30:29.784825 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 66947.2us
I20260924 14:30:29.784363 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:29.759485 29616 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.000s
I20260924 14:30:29.809602 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:29.822870 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:29.760494 29587 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:29.645910 29672 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:29.666334 29676 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:29.771251 29553 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:29.830642 27182 raft_consensus.cc:2396] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260924 14:30:29.828711 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 115515us
I20260924 14:30:29.831368 29676 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:29.831856 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:29.775092 29622 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:29.831378 29553 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:29.832906 26327 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:29.834311 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:29.835240 26327 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:29.758090 29559 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:29.785106 29573 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.530s	user 0.010s	sys 0.006s
I20260924 14:30:29.849728 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:29.773265 28867 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:29.757056 29146 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260924 14:30:29.767879 29534 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:29.720530 29203 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:29.860778 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:29.809325 29340 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:29.836139 29622 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:29.871049 29685 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:29.871086 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:29.821079 29423 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:29.823302 29660 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:29.786301 29634 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:29.825253 29678 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:29.875787 26331 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:29.876693 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:29.836917 29616 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:29.887508 29610 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260924 14:30:29.894564 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:29.767040 29526 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.755718 29375 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.889410 29672 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:29.759277 29560 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:29.862748 29587 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:29.869632 29643 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:29.855710 29617 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260924 14:30:29.914754 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 43745.7us
I20260924 14:30:29.821931 29642 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:29.917104 29423 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:29.912963 29559 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:29.881012 28867 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:29.860507 29573 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:29.784473 29570 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:29.918697 29607 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.265985 29695 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:30.266507 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:30.267035 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:29.994442 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65982us
W20260924 14:30:30.106123 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 340985us
I20260924 14:30:30.137405 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:30.268198 26298 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 1.628s (SENT) [suppressed 18 similar messages]
I20260924 14:30:30.123531 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:30.137424 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1027) took 1538 ms. Trace:
I20260924 14:30:30.269649 27190 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:30.277406 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:30.278402 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:30.273160 26250 rpcz_store.cc:276] 0924 14:30:28.598808 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.598979 (+   171us) service_pool.cc:227] Handling call
0924 14:30:28.599318 (+   339us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:29.756571 (+1157253us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:30.137162 (+380591us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540002bf54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:30.137358 (+   196us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2589274496}
I20260924 14:30:30.279770 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:29.881431 29606 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.327705 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:30.328043 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:30.329005 27189 raft_consensus.cc:2354] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7.
I20260924 14:30:30.326251 26295 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:30.293038 26331 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:30.327378 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 889) took 3517 ms. Trace:
I20260924 14:30:30.322872 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:30.334960 26257 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:30.234135 29108 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:30.105603 29682 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:30.101883 29551 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:29.915345 29668 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:30.274874 29185 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.345263 29697 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:30.332922 27191 rpcz_store.cc:276] 0924 14:30:26.764211 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:26.765214 (+  1003us) service_pool.cc:227] Handling call
0924 14:30:26.778210 (+ 12996us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:26.809883 (+ 31673us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:26.809928 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:26.922748 (+112820us) log.cc:842] Serialized 26 byte log entry
0924 14:30:27.031590 (+108842us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:27.031634 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:27.031705 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:27.792646 (+760941us) raft_consensus.cc:1671] finished
0924 14:30:27.792670 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:30.281057 (+2488387us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b5400155c50. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:30.281259 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31708,"raft.queue_time_us":27388,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":85634,"spinlock_wait_cycles":28988800,"thread_start_us":59685,"threads_started":1,"wal-append.queue_time_us":234043,"wal-append.run_cpu_time_us":3589,"wal-append.run_wall_time_us":1629395}
I20260924 14:30:30.346220 27181 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:30.166483 29618 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:29.920286 29640 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:30.276302 29678 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:30.347884 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:30.348222 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:30.338492 29300 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:30.106631 29186 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:30.175256 29466 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:30.344697 29664 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:30.263254 29611 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:30.017948 29533 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:30.274653 29607 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:30.359819 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:30.365883 27194 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.015s	sys 0.018s
W20260924 14:30:30.507109 27194 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.015s	sys 0.018s
I20260924 14:30:30.128280 29636 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.116847 29540 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:30.474260 29686 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260924 14:30:30.268404 29602 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:30.265862 29587 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:30.437872 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:30.460039 26331 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:29.932615 29634 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:30.425518 29637 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.143s	user 0.009s	sys 0.003s
I20260924 14:30:30.521185 29637 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:30.466578 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 114734us
I20260924 14:30:30.467705 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 925) took 2338 ms. Trace:
I20260924 14:30:30.523734 26397 rpcz_store.cc:276] 0924 14:30:28.111357 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.115683 (+  4326us) service_pool.cc:227] Handling call
0924 14:30:28.115950 (+   267us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:28.155737 (+ 39787us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:28.155769 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:28.155812 (+    43us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:28.155846 (+    34us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:30.449487 (+2293641us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b5400476654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:30.449535 (+    48us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:30.449689 (+   154us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":91054464}
I20260924 14:30:30.477489 29419 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:30.478761 28867 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:30.346259 29052 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.113s
I20260924 14:30:30.435875 29541 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:30.509272 26250 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:30.482018 29611 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:30.284879 29700 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:30.520435 29639 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:30.526006 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1021) took 2026 ms. Trace:
I20260924 14:30:30.556559 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:30.517657 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 523) took 2941 ms. Trace:
I20260924 14:30:30.504853 29697 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:30.481853 29466 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:30.545851 29637 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.475270 29383 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260924 14:30:30.547744 29698 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260924 14:30:30.525051 29419 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:30.559144 26250 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:30.560789 29526 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:30.526137 29672 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:30.556900 26254 rpcz_store.cc:276] 0924 14:30:28.499311 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.499477 (+   166us) service_pool.cc:227] Handling call
0924 14:30:28.518428 (+ 18951us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:29.147049 (+628621us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:30:29.367256 (+220207us) log.cc:842] Serialized 14 byte log entry
0924 14:30:29.844508 (+477252us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:29.844567 (+    59us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:29.844614 (+    47us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:30.525729 (+681115us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b540015e0d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:30.525787 (+    58us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:30.525940 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":186550,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":47849,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":6896,"spinlock_wait_cycles":1477333504,"thread_start_us":696770,"threads_started":3,"wal-append.queue_time_us":400709}
I20260924 14:30:30.557399 27188 rpcz_store.cc:276] 0924 14:30:27.575753 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:27.575873 (+   120us) service_pool.cc:227] Handling call
0924 14:30:27.576239 (+   366us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:27.787170 (+210931us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:27.787200 (+    30us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:30:27.790859 (+  3659us) log.cc:842] Serialized 52 byte log entry
0924 14:30:27.847173 (+ 56314us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:27.847228 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:30.509124 (+2661896us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400326254. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:30.509200 (+    76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:30.509267 (+    67us) raft_consensus.cc:1671] finished
0924 14:30:30.509285 (+    18us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:30.517577 (+  8292us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":63855,"raft.queue_time_us":66465,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":21384,"spinlock_wait_cycles":237858176,"thread_start_us":49082,"threads_started":1,"wal-append.queue_time_us":117560,"wal-append.run_cpu_time_us":2562,"wal-append.run_wall_time_us":1181491}
I20260924 14:30:30.575495 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1271) took 1236 ms. Trace:
I20260924 14:30:30.570330 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:30.570914 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 833) took 1221 ms. Trace:
I20260924 14:30:30.577666 26250 rpcz_store.cc:276] 0924 14:30:29.349072 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:30.558566 (+1209494us) service_pool.cc:227] Handling call
0924 14:30:30.558923 (+   357us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:30.570606 (+ 11683us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:30.570845 (+   239us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:30.516166 29607 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:30.576040 26249 rpcz_store.cc:276] 0924 14:30:29.339288 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.355545 (+ 16257us) service_pool.cc:227] Handling call
0924 14:30:29.384736 (+ 29191us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:29.432950 (+ 48214us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:29.432998 (+    48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:29.441309 (+  8311us) log.cc:842] Serialized 26 byte log entry
0924 14:30:29.722101 (+280792us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:29.764185 (+ 42084us) log.cc:842] Serialized 14 byte log entry
0924 14:30:29.764406 (+   221us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:29.764481 (+    75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:30.575227 (+810746us) raft_consensus.cc:1671] finished
0924 14:30:30.575247 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:30.575400 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":71632,"prepare.queue_time_us":40650,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":141581,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":49229,"thread_start_us":173360,"threads_started":1,"wal-append.queue_time_us":344447}
W20260924 14:30:30.585642 29705 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260924 14:30:30.565531 29639 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:30.586119 29375 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:30.514808 29664 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:30.586509 29639 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:30.590250 29560 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:30.453090 29682 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:30.449038 29551 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:30.575680 29672 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:30.531819 29532 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:30.557200 29541 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:30.607547 26249 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:30.567142 29615 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:30.617358 29607 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.602871 29642 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:30.610693 29616 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.639994 29541 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:30.640748 29466 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:30.583907 29611 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:30.558851 29700 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:30.614950 29570 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:30.649821 29419 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.660056 29700 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.634073 29573 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:30.635277 29664 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:30.661834 29383 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:30.683836 29383 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:30.685451 29634 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:30.666636 26251 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:30.669004 29672 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:30.635972 29532 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:30.669361 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 602) took 1090 ms. Trace:
I20260924 14:30:30.696828 26326 rpcz_store.cc:276] 0924 14:30:29.578615 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.578909 (+   294us) service_pool.cc:227] Handling call
0924 14:30:29.579180 (+   271us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:29.759496 (+180316us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:29.759531 (+    35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:29.759568 (+    37us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:29.759596 (+    28us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:30.669041 (+909445us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54004100d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:30.669102 (+    61us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:30.669286 (+   184us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":414226688}
W20260924 14:30:30.690974 26330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 789) took 2746 ms (client timeout 1908 ms). Trace:
W20260924 14:30:30.692353 27181 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 908) took 2408 ms (client timeout 2188 ms). Trace:
I20260924 14:30:30.693732 29669 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:30.700552 27181 rpcz_store.cc:269] 0924 14:30:28.270627 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.270941 (+   314us) service_pool.cc:227] Handling call
0924 14:30:30.679012 (+2408071us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b540000c154. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:30.679190 (+   178us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4753686400}
I20260924 14:30:30.712257 29571 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:30.713311 26325 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.046s	sys 0.074s
W20260924 14:30:30.713673 26325 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.046s	sys 0.074s
I20260924 14:30:30.719246 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 919) took 1097 ms. Trace:
I20260924 14:30:30.719825 27189 rpcz_store.cc:276] 0924 14:30:29.621455 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.621668 (+   213us) service_pool.cc:227] Handling call
0924 14:30:30.718945 (+1097277us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b540000e954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:30.719169 (+   224us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1589370240}
W20260924 14:30:30.701120 29702 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260924 14:30:30.699582 26330 rpcz_store.cc:269] 0924 14:30:27.928800 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:27.932358 (+  3558us) service_pool.cc:227] Handling call
0924 14:30:30.674594 (+2742236us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54007d46d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:30.674749 (+   155us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":835706240}
I20260924 14:30:30.648432 29682 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:30.689152 29646 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.579s	user 0.013s	sys 0.000s
I20260924 14:30:30.769795 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:30.730190 29672 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:30.771034 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:30.777411 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:30.763626 29551 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:30.648737 29697 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:30.803668 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:30.804740 26398 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:30.759388 29646 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:30.810369 26403 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 8: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:30:30.802961 29701 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260924 14:30:30.777338 29672 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:30.820227 29711 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:30.823387 29711 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:30.839125 29646 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:30.713146 29571 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:30.850433 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:30.831521 29657 consensus_meta.cc:287] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.528s	user 0.015s	sys 0.000s
I20260924 14:30:30.772732 29669 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:30.848521 29627 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:30.912426 26252 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:30.868638 29700 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:30.915715 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:55802 (request call id 1024) took 2390 ms. Trace:
W20260924 14:30:30.910033 29538 consensus_meta.cc:287] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.323s	user 0.011s	sys 0.003s
I20260924 14:30:30.918767 26252 rpcz_store.cc:276] 0924 14:30:28.522952 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.523146 (+   194us) service_pool.cc:227] Handling call
0924 14:30:30.905732 (+2382586us) spinlock_profiling.cc:243] Waited 84.9 ms on lock 0x7b54000c1cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b9345da7 00007f09be291ee0 00007f09b7983784 00007f09b7983712 00007f09b7983641 00007f09b79835bc 00007f09b7982872 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:30.913344 (+  7612us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":195232640}
I20260924 14:30:30.903044 29292 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:30.907050 29607 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:30.910374 29419 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:30.924589 29700 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:30.930404 29645 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:30.940382 29711 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.942199 29712 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:30.931519 29571 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:30.925200 29538 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:30.913009 29627 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:30.902576 29657 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:30.953716 27125 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.017s	sys 0.024s
W20260924 14:30:30.961175 27125 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.018s	sys 0.025s
W20260924 14:30:30.962230 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 921054us
I20260924 14:30:30.941332 29645 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:30.968386 26255 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:30.970119 26369 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.012s	sys 0.029s
I20260924 14:30:30.942833 29712 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:30.976261 29427 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.028s	sys 0.037s
I20260924 14:30:30.977072 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:30.981736 29700 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:30.966641 29260 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:30.969645 29501 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:30.969803 29708 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:30.984490 29538 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:30.974579 29185 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260924 14:30:30.985203 29645 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:30.986560 29383 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:30.943548 29419 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:30.935369 29292 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:30.967041 26371 connection.cc:232] RPC call timeout handler was delayed by -1.1755521619998035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
W20260924 14:30:31.035439 26371 connection.cc:232] RPC call timeout handler was delayed by -1.1620521619997817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260924 14:30:31.035898 26371 connection.cc:232] RPC call timeout handler was delayed by -1.1235912959991765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
W20260924 14:30:31.036186 26371 connection.cc:232] RPC call timeout handler was delayed by -1.0398912959999507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
I20260924 14:30:30.941151 29607 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:30.963136 29560 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:30.992764 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 467335us
I20260924 14:30:30.994722 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 539) took 1454 ms. Trace:
W20260924 14:30:30.993979 29427 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.029s	sys 0.038s
W20260924 14:30:30.995162 26369 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.012s	sys 0.029s
W20260924 14:30:31.057920 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 610519us
W20260924 14:30:30.994341 29203 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.000s	sys 0.016s
W20260924 14:30:31.013486 29609 consensus_meta.cc:287] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.466s	user 0.016s	sys 0.002s
I20260924 14:30:31.015111 29672 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:31.004427 26405 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:31.005551 29501 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:31.061161 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.004361 29260 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:31.012748 29708 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:31.054999 27194 rpcz_store.cc:276] 0924 14:30:29.540142 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.548480 (+  8338us) service_pool.cc:227] Handling call
0924 14:30:29.561025 (+ 12545us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:29.561242 (+   217us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:30:30.507672 (+946430us) log.cc:842] Serialized 14 byte log entry
0924 14:30:30.993750 (+486078us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:30.993804 (+    54us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:30.993854 (+    50us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:30.993917 (+    63us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:30.994666 (+   749us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":441044,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":57,"raft.queue_time_us":2422,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":2561,"thread_start_us":1330488,"threads_started":2}
I20260924 14:30:31.072532 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.074827 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:31.076532 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:31.078118 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:31.081557 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.084640 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:31.065639 29203 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.000s	sys 0.018s
I20260924 14:30:31.118140 26397 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:30:31.057277 29682 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:31.065168 29645 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:31.127270 26295 leader_election.cc:398] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7.
I20260924 14:30:31.130540 26295 leader_election.cc:401] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:30:31.130596 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.116811 29525 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:31.118003 29716 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:31.116400 29599 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:31.171590 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.117187 29587 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:31.168852 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 51029.6us
I20260924 14:30:31.131203 29657 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:31.064951 29560 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:31.172367 26399 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:31.069967 29609 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:31.168217 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.180115 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:31.181051 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.054s	sys 0.067s
I20260924 14:30:31.181769 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.182618 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:31.177405 29715 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.169s	user 0.001s	sys 0.001s
I20260924 14:30:31.132313 29573 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:31.209709 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.175818 29185 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 7
I20260924 14:30:31.167974 29288 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:31.160394 29646 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:31.217141 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:31.219236 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.054s	sys 0.071s
I20260924 14:30:31.218109 27190 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:31.136013 29682 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:31.225219 26404 raft_consensus.cc:2354] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9.
I20260924 14:30:31.176059 29664 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:31.237484 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:31.185417 29712 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:31.237912 27185 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:31.209272 29466 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:31.210793 29650 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:31.300433 29650 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:31.303040 29650 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:31.308241 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.309880 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.310909 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.180670 29599 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:31.316064 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 576) took 1974 ms. Trace:
W20260924 14:30:31.316646 26403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1133) took 2581 ms (client timeout 1771 ms). Trace:
I20260924 14:30:31.316557 27186 rpcz_store.cc:276] 0924 14:30:29.341502 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.383295 (+ 41793us) service_pool.cc:227] Handling call
0924 14:30:29.383602 (+   307us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:29.723318 (+339716us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:29.723361 (+    43us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:29.723419 (+    58us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:29.723474 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.315749 (+1592275us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b540000e454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:31.315805 (+    56us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:31.315997 (+   192us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":780684160}
W20260924 14:30:31.317153 26403 rpcz_store.cc:269] 0924 14:30:28.735197 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.735355 (+   158us) service_pool.cc:227] Handling call
0924 14:30:31.316424 (+2581069us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b540023fd54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:31.316572 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4476959488}
I20260924 14:30:31.318568 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.319500 26403 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:31.320567 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.326218 29559 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:31.237032 29288 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:31.329877 29288 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:31.330926 29678 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.849s	user 0.011s	sys 0.004s
I20260924 14:30:31.332031 29678 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:31.333365 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.336745 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.318585 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 596) took 2684 ms. Trace:
W20260924 14:30:31.268146 29722 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260924 14:30:31.344971 29599 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:31.348594 29442 raft_consensus.cc:3032] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:31.349223 29442 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:31.350217 26250 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:31.350457 29442 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:31.352281 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1035) took 1477 ms. Trace:
I20260924 14:30:31.338732 26332 rpcz_store.cc:276] 0924 14:30:28.633939 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.634058 (+   119us) service_pool.cc:227] Handling call
0924 14:30:29.283602 (+649544us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:30:29.283916 (+   314us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:29.384288 (+100372us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:30:29.765664 (+381376us) log.cc:842] Serialized 14 byte log entry
0924 14:30:30.267874 (+502210us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:30.267923 (+    49us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:30.267965 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.318341 (+1050376us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54005587d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:31.318389 (+    48us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:31.318505 (+   116us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3222,"prepare.queue_time_us":503684,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":95297,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":46348,"spinlock_wait_cycles":308864128,"thread_start_us":743361,"threads_started":2,"wal-append.queue_time_us":709046}
I20260924 14:30:31.352623 26250 rpcz_store.cc:276] 0924 14:30:29.875263 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:30.593609 (+718346us) service_pool.cc:227] Handling call
0924 14:30:31.350020 (+756411us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:31.350928 (+   908us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.352054 (+  1126us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b54006d2ed0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:31.352214 (+   160us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1738850176}
I20260924 14:30:31.183346 29691 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:31.219106 29716 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:31.184630 29587 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:31.240329 29525 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:31.259835 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:31.359833 26256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1023) took 2845 ms (client timeout 1677 ms). Trace:
I20260924 14:30:31.250134 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:31.360365 26256 rpcz_store.cc:269] 0924 14:30:28.514657 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.514969 (+   312us) service_pool.cc:227] Handling call
0924 14:30:31.359640 (+2844671us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54004fa454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:31.359741 (+   101us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1223899520}
W20260924 14:30:31.361389 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 111328us
I20260924 14:30:31.362232 27194 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:31.362389 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:31.359790 29716 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:31.239992 29664 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:31.363844 29587 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:31.250013 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.364732 26326 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:31.365023 29716 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:31.365455 29664 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:31.365768 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.366120 29664 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:31.366616 29664 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:31.366927 29664 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:31.367133 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.360304 26323 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:31.367624 26399 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:31.368355 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.368633 26326 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:31.369410 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.247553 29560 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:31.245487 29466 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:31.370956 29587 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:31.372378 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.372766 29587 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:31.252242 29639 consensus_meta.cc:287] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.660s	user 0.015s	sys 0.000s
I20260924 14:30:31.249540 26295 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:31.374161 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 124691us
I20260924 14:30:31.269362 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.269146 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:31.269670 29720 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260924 14:30:31.272073 29729 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260924 14:30:31.272790 29419 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:31.352288 29730 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260924 14:30:31.225644 29573 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:31.276646 29615 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:31.276758 29607 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:31.365810 29442 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:31.283684 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.285236 29611 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:31.286230 29682 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:31.288306 29708 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:31.219585 29629 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:31.289920 29501 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:31.296324 29260 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:31.298626 29703 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.817s	user 0.000s	sys 0.003s
I20260924 14:30:31.241484 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:31.350162 26257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 832) took 2001 ms (client timeout 2116 ms). Trace:
I20260924 14:30:31.434597 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.358206 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1042) took 1472 ms. Trace:
I20260924 14:30:31.353127 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.357951 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1256) took 4244 ms. Trace:
I20260924 14:30:31.357465 29691 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:31.369612 26323 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: []
W20260924 14:30:31.373049 26224 connection.cc:232] RPC call timeout handler was delayed by -1.0476036059999387s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1771s for a response
I20260924 14:30:31.245029 29292 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:31.436199 26403 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:31.271191 29609 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:31.353526 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.370613 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.435737 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.241742 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.388962 26398 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:31.390926 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.442267 26406 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:31.416006 29643 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:31.273660 29383 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:31.398283 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:31.362982 29532 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.662s	user 0.013s	sys 0.001s
I20260924 14:30:31.408108 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.410068 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:31.410358 26401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1268) took 3014 ms (client timeout 2653 ms). Trace:
I20260924 14:30:31.278374 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 604) took 1474 ms. Trace:
I20260924 14:30:31.413686 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 828) took 2066 ms. Trace:
I20260924 14:30:31.387547 29587 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:30:31.414407 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 1.14734e+06us
I20260924 14:30:31.415184 29695 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:31.421840 29718 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:31.422401 29533 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:31.372823 29466 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:31.373764 29639 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:31.388525 29678 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:31.403795 29598 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:31.487545 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.455763 29708 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:31.435959 26250 rpcz_store.cc:276] 0924 14:30:29.885832 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.357328 (+1471496us) service_pool.cc:227] Handling call
0924 14:30:31.357674 (+   346us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:31.357873 (+   199us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:31.357900 (+    27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:31.357954 (+    54us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:31.357985 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.358044 (+    59us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:31.358155 (+   111us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:31.436630 26251 rpcz_store.cc:276] 0924 14:30:27.113440 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:27.126548 (+ 13108us) service_pool.cc:227] Handling call
0924 14:30:27.126840 (+   292us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:30.686168 (+3559328us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.357692 (+671524us) spinlock_profiling.cc:243] Waited 3.54 s on lock 0x7b54000c08d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:31.357862 (+   170us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8139768576}
W20260924 14:30:31.435066 26257 rpcz_store.cc:269] 0924 14:30:29.348763 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.430856 (+ 82093us) service_pool.cc:227] Handling call
0924 14:30:31.349834 (+1918978us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54006d2ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:31.350067 (+   233us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2049978624}
I20260924 14:30:31.437395 27185 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
W20260924 14:30:31.440878 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 199506us
I20260924 14:30:31.374995 26331 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:31.392354 27126 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:31.436894 29629 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:31.446683 26400 raft_consensus.cc:2379] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:31.415690 29615 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:31.447798 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 830) took 2099 ms. Trace:
I20260924 14:30:31.448767 27125 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:31.441953 27188 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
W20260924 14:30:31.450933 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 170743us
W20260924 14:30:31.453469 26401 rpcz_store.cc:269] 0924 14:30:28.397265 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.480649 (+ 83384us) service_pool.cc:227] Handling call
0924 14:30:31.410077 (+2929428us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540023f854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:31.410265 (+   188us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2673654656}
I20260924 14:30:31.464148 29581 consensus_queue.cc:1046] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260924 14:30:31.453617 26325 rpcz_store.cc:276] 0924 14:30:29.804084 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.804288 (+   204us) service_pool.cc:227] Handling call
0924 14:30:29.837114 (+ 32826us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:29.867402 (+ 30288us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:29.867537 (+   135us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:29.881286 (+ 13749us) log.cc:842] Serialized 26 byte log entry
0924 14:30:30.714041 (+832755us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:30.714099 (+    58us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:30.714191 (+    92us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:31.278068 (+563877us) raft_consensus.cc:1671] finished
0924 14:30:31.278105 (+    37us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:31.278286 (+   181us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4069,"raft.queue_time_us":35964,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":7081,"thread_start_us":713872,"threads_started":1,"wal-append.queue_time_us":1058110}
I20260924 14:30:31.474082 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.475270 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.475270 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.476322 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.477208 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:31.485227 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47954.9us
I20260924 14:30:31.436699 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.485831 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.496037 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.502046 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 577) took 1863 ms. Trace:
I20260924 14:30:31.466737 29536 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:31.509123 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 116902us
I20260924 14:30:31.512187 29526 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:31.453701 26258 rpcz_store.cc:276] 0924 14:30:29.347573 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.694974 (+347401us) service_pool.cc:227] Handling call
0924 14:30:29.724484 (+ 29510us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:29.727925 (+  3441us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:29.727972 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:29.728522 (+   550us) log.cc:842] Serialized 26 byte log entry
0924 14:30:29.866093 (+137571us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:29.891470 (+ 25377us) log.cc:842] Serialized 14 byte log entry
0924 14:30:29.891657 (+   187us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.364528 (+1472871us) spinlock_profiling.cc:243] Waited 90.1 ms on lock 0x7b18003d3840. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09bd1f6bcf 00007f09b64267b6 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d
0924 14:30:31.413399 (+ 48871us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b54000c26d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:31.413450 (+    51us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:31.413506 (+    56us) raft_consensus.cc:1671] finished
0924 14:30:31.413519 (+    13us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:31.413624 (+   105us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":35617,"prepare.queue_time_us":33106,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":17127,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":19260,"spinlock_wait_cycles":214456960,"thread_start_us":80196,"threads_started":1,"wal-append.queue_time_us":414810}
I20260924 14:30:31.514043 26253 rpcz_store.cc:276] 0924 14:30:29.348162 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.819391 (+471229us) service_pool.cc:227] Handling call
0924 14:30:29.858567 (+ 39176us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:29.875752 (+ 17185us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:29.875782 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:29.887477 (+ 11695us) log.cc:842] Serialized 26 byte log entry
0924 14:30:30.001308 (+113831us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:30.177189 (+175881us) log.cc:842] Serialized 14 byte log entry
0924 14:30:31.297735 (+1120546us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:31.404546 (+106811us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.404657 (+   111us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:31.447483 (+ 42826us) raft_consensus.cc:1671] finished
0924 14:30:31.447598 (+   115us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:31.447725 (+   127us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1271739,"prepare.queue_time_us":132322,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":1277030,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":7632,"spinlock_wait_cycles":13890432,"thread_start_us":169536,"threads_started":2,"wal-append.queue_time_us":459457}
I20260924 14:30:31.567334 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:31.567809 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 119129us
I20260924 14:30:31.569447 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:31.570266 26298 leader_election.cc:334] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 1.697s (SENT)
I20260924 14:30:31.570634 27125 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:31.572153 26328 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:31.568153 26251 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:31.480191 29647 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:31.532943 27192 rpcz_store.cc:276] 0924 14:30:29.638276 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.638465 (+   189us) service_pool.cc:227] Handling call
0924 14:30:29.638870 (+   405us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:29.779863 (+140993us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.501653 (+1721790us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400155c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:31.501953 (+   300us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":299372288}
I20260924 14:30:31.490450 29718 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:31.582667 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:31.574005 27181 raft_consensus.cc:2396] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260924 14:30:31.583971 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.624878 26250 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:31.576000 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:31.586774 26325 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:31.587112 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.585103 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.587332 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:31.615003 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.629710 26331 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:31.587765 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.583524 29734 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:31.587805 29573 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:31.584954 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.590133 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:31.590355 29669 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:31.581135 29536 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:31.592001 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1165) took 2695 ms. Trace:
I20260924 14:30:31.591909 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.574162 27126 leader_election.cc:398] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9.
W20260924 14:30:31.647430 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 80660.9us
I20260924 14:30:31.593773 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:31.594066 29691 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:31.604787 26077 catalog_manager.cc:5495] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:31.658430 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.605556 29627 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:31.608100 29581 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:31.670217 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.608634 29697 raft_consensus.cc:3032] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:31.609345 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.609673 29485 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:30:31.675289 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:31.484946 29533 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:31.610385 29695 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:31.646688 29559 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:31.696864 26253 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:31.490959 29694 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:31.518251 29663 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:31.591104 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1153) took 3490 ms. Trace:
I20260924 14:30:31.623211 27194 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:31.627691 27191 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 5.
I20260924 14:30:31.624547 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:31.637362 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 47308.1us
W20260924 14:30:31.636406 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 117057us
I20260924 14:30:31.583475 29526 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:31.659891 27126 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:31.708518 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 48776.8us
I20260924 14:30:31.646705 26329 rpcz_store.cc:276] 0924 14:30:28.896440 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.006118 (+109678us) service_pool.cc:227] Handling call
0924 14:30:30.676046 (+1669928us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:30.676470 (+   424us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:30.676510 (+    40us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:30.676568 (+    58us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:30.676619 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:30.676694 (+    75us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:31.591817 (+915123us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54007d46d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:31.591920 (+   103us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":577,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":12492,"spinlock_wait_cycles":3667252992}
I20260924 14:30:31.574340 29678 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:31.657380 26404 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:31.714659 27182 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
W20260924 14:30:31.671249 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 99652us
I20260924 14:30:31.672992 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:31.674127 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 100409us
I20260924 14:30:31.610029 29599 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:31.678898 26331 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:31.513384 29598 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:31.681465 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.636534 29718 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:31.611266 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:31.635730 29571 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:31.582445 29708 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260924 14:30:31.698694 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:31.652936 26250 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:31.749042 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.749316 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:31.704633 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:31.701270 27192 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:31.706238 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:31.750598 26258 raft_consensus.cc:2362] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11. Re-sending same reply.
I20260924 14:30:31.705314 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:31.700624 26324 rpcz_store.cc:276] 0924 14:30:28.100796 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:28.107129 (+  6333us) service_pool.cc:227] Handling call
0924 14:30:28.107509 (+   380us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:28.114381 (+  6872us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:30:28.114672 (+   291us) log.cc:842] Serialized 14 byte log entry
0924 14:30:29.258842 (+1144170us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:30.449096 (+1190254us) spinlock_profiling.cc:243] Waited 8.47 ms on lock 0x7b18000c7ea0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b8fdd82c 00007f09b64089dc 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6423155 00007f09b93463d0 00007f09b9358576 00007f09b93690d7
0924 14:30:30.468311 (+ 19215us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:30.468381 (+    70us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:30.468440 (+    59us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.590858 (+1122418us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b54007ad854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:31.590909 (+    51us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:31.591043 (+   134us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17127,"prepare.queue_time_us":863,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":73,"raft.queue_time_us":49091,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":37339,"spinlock_wait_cycles":227907072,"thread_start_us":362837,"threads_started":2,"wal-append.queue_time_us":490238,"wal-append.run_cpu_time_us":3838,"wal-append.run_wall_time_us":1744322}
I20260924 14:30:31.620028 29647 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:31.703837 29663 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:31.712414 29639 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:31.754827 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:31.755296 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 50731.2us
I20260924 14:30:31.749758 26250 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:31.757656 26250 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:30:31.715221 27182 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:31.758811 27182 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:31.721195 26295 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:31.760398 27182 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:31.675362 29697 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:31.682108 29185 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7.
I20260924 14:30:31.733475 29645 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:31.763065 26077 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 8, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 8 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:31.736021 29625 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:31.735872 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.736260 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.739830 26332 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:31.743673 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 171957us
I20260924 14:30:31.745266 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.695093 29627 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:31.746562 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.749034 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.707676 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:31.787045 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.695200 29559 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:31.788682 29559 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:31.787923 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:31.695178 29679 consensus_queue.cc:1159] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "45ab97b763804767846e9d27a01b561f" responder_term: 9 status { last_received { term: 9 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 9 index: 6 } }
I20260924 14:30:31.734486 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:31.763423 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.765405 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:31.767560 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62290.3us
I20260924 14:30:31.733747 29676 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:31.793689 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:31.794049 29676 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:31.788239 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:31.794765 29676 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:31.795838 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.770545 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.714363 29678 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:31.799396 29678 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:31.793403 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.791218 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:31.804837 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.784893 26327 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:31.805799 26250 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:31.734798 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:31.770010 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:31.801241 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:31.797683 27190 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:30:31.804953 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:31.807209 26329 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:31.809275 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.810887 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.794327 27188 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:31.811760 26250 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:31.813068 27182 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:31.813136 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.744578 29621 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:31.815483 26255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 836) took 2453 ms (client timeout 1674 ms). Trace:
I20260924 14:30:31.817087 26249 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
W20260924 14:30:31.816013 26255 rpcz_store.cc:269] 0924 14:30:29.361738 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:29.364861 (+  3123us) service_pool.cc:227] Handling call
0924 14:30:31.815214 (+2450353us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b540002ba54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:31.815375 (+   161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3642072960}
W20260924 14:30:31.818686 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 84273.8us
W20260924 14:30:31.806231 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 85121.2us
I20260924 14:30:31.826496 26295 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:31.830271 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.707154 29694 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:31.767800 29609 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260924 14:30:31.837225 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 102477us
I20260924 14:30:31.838070 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:31.831056 26324 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:31.830930 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:31.762218 29663 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:31.839424 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:31.840513 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1037) took 1964 ms. Trace:
I20260924 14:30:31.845175 26249 rpcz_store.cc:276] 0924 14:30:29.875831 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:30.623611 (+747780us) service_pool.cc:227] Handling call
0924 14:30:31.816893 (+1193282us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:31.817565 (+   672us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.832628 (+ 15063us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540002ba50. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:31.840419 (+  7791us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2694557568}
I20260924 14:30:31.846989 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:31.835220 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.848196 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.848894 26397 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:31.850317 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.851025 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.851433 26329 raft_consensus.cc:2379] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:31.851758 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.852928 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.854074 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.855077 26250 raft_consensus.cc:2396] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260924 14:30:31.853914 27182 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:31.857260 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.802294 29740 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:31.815812 29734 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:31.831508 29673 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:31.865031 26258 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:31.801476 29640 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:31.754590 29571 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:31.862936 26327 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:31.862964 27188 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:31.833743 29678 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:31.778291 29560 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260924 14:30:31.842872 29609 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:31.816377 29625 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:31.801378 29645 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:31.876324 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:31.837637 29621 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:31.880429 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47242us
I20260924 14:30:31.842545 29716 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:31.870359 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:31.860258 28867 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:31.877925 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.893414 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.929920 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:31.985137 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:31.985173 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 114958us
I20260924 14:30:31.883801 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:31.938462 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 98335.4us
I20260924 14:30:31.987172 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.927386 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:31.863849 29639 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:31.874437 29691 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:31.992941 28867 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:31.993498 28867 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:31.986166 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:31.935209 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:31.906142 29689 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:31.989732 29185 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election won for term 9
W20260924 14:30:31.990872 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 114660us
I20260924 14:30:31.884440 29615 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:31.991514 29540 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:31.857899 29697 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:31.873427 29629 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:31.996536 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:31.982730 29609 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:32.012053 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.874289 29673 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:32.012569 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:32.015174 26399 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:32.045348 26325 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:32.045773 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:31.953747 29501 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:32.049211 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:32.002882 29532 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:32.025893 29733 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:31.874801 29740 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:32.022096 29616 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260924 14:30:32.027192 29288 consensus_queue.cc:1046] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:32.029381 27125 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:31.979743 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45400.5us
I20260924 14:30:32.030186 29633 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:31.986719 26256 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
W20260924 14:30:32.031203 29741 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260924 14:30:32.033524 29734 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:31.980199 29716 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:32.036026 29621 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:32.036672 29694 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:32.017097 29689 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:32.004074 29617 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:31.983481 29526 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:31.969641 29611 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.545s	user 0.007s	sys 0.003s
I20260924 14:30:32.047777 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:32.049166 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:31.994783 26253 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:32.052402 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:32.012666 29605 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:32.027333 29536 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:32.029682 26258 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:31.949425 29718 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:32.061024 26224 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:32.080907 29734 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:32.069563 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 197761us
I20260924 14:30:32.070873 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.049671 29673 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:32.064499 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.078187 27188 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:32.078418 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:32.083312 29571 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:32.048134 29598 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:32.080315 26249 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:32.021754 29559 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:32.059669 29733 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:32.090794 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:32.062752 29633 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:32.100979 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 48779.8us
I20260924 14:30:32.024349 29185 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:32.081580 29599 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:32.160545 29633 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:32.116497 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:32.161696 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45321.5us
I20260924 14:30:32.162470 26298 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:32.119307 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:32.164067 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 44891.5us
I20260924 14:30:32.164942 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:32.050274 29643 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:32.120527 26250 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:32.060853 29532 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:32.093585 29689 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:32.137013 26323 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:32.083863 29697 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:32.116971 29605 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:32.102718 29606 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:31.892169 29640 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:32.102378 29526 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:32.135528 29611 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:32.157541 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:32.088939 29485 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260924 14:30:32.122577 29536 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:32.160081 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 834) took 2809 ms. Trace:
I20260924 14:30:32.193987 26254 rpcz_store.cc:276] 0924 14:30:29.350620 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:30.573588 (+1222968us) service_pool.cc:227] Handling call
0924 14:30:30.574075 (+   487us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:31.430487 (+856412us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:31.430533 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:31.431307 (+   774us) log.cc:842] Serialized 26 byte log entry
0924 14:30:31.649737 (+218430us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:31.649789 (+    52us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.649880 (+    91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:32.159832 (+509952us) raft_consensus.cc:1671] finished
0924 14:30:32.159869 (+    37us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:32.160011 (+   142us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":903019,"raft.queue_time_us":856070,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":1912,"thread_start_us":183271,"threads_started":1,"wal-append.queue_time_us":374472}
I20260924 14:30:32.160511 29599 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:32.081969 29716 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:32.198346 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:32.169903 29661 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:32.199285 29599 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:32.200850 29639 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:30:32.159431 29442 consensus_meta.cc:287] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.735s	user 0.014s	sys 0.000s
W20260924 14:30:32.188444 29581 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260924 14:30:32.172607 29466 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:32.190045 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.202250 29466 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:32.202687 29466 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:32.203013 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.203717 26253 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:32.204524 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:32.191372 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.172937 29419 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:32.167119 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:32.189152 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.193120 29643 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:32.176549 29560 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:32.206794 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:32.211617 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:32.212724 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.209744 29611 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:32.176174 29532 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:32.193235 29679 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:32.193938 29536 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:32.197598 29625 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:32.234494 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:32.261204 26400 raft_consensus.cc:2379] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:32.260972 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:32.215385 29707 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:32.166781 29733 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:32.258150 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:32.237215 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:32.261648 26401 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:32.262229 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.269970 26254 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:32.273166 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 295713us
I20260924 14:30:32.274471 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:32.260856 29536 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:32.284858 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1292) took 1039 ms. Trace:
I20260924 14:30:32.285725 26403 rpcz_store.cc:276] 0924 14:30:31.245463 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.439791 (+194328us) service_pool.cc:227] Handling call
0924 14:30:31.446401 (+  6610us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:31.450508 (+  4107us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:31.450541 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:31.458956 (+  8415us) log.cc:842] Serialized 26 byte log entry
0924 14:30:31.607204 (+148248us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b18000d3c00. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:30:31.698543 (+ 91339us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:31.829763 (+131220us) log.cc:842] Serialized 14 byte log entry
0924 14:30:31.848416 (+ 18653us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:31.848491 (+    75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:32.279718 (+431227us) raft_consensus.cc:1671] finished
0924 14:30:32.284565 (+  4847us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:32.284783 (+   218us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":77711,"prepare.queue_time_us":109658,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":32273,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":28971,"spinlock_wait_cycles":2750208,"thread_start_us":91153,"threads_started":1,"wal-append.queue_time_us":654931}
I20260924 14:30:32.339712 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:32.347275 26254 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:32.276005 29740 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:32.259714 29679 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:32.362607 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:32.288136 29540 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:32.276441 29707 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:32.316696 29650 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:32.285768 29640 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:32.381438 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:32.382364 26250 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:32.349800 29532 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:32.388348 26077 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 9, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 9 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:32.393703 26254 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:32.390282 26077 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 7, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 7 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:32.262167 29185 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:32.399384 26077 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 9, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 9 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:32.419217 26077 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 9, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 9 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:32.421641 26077 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 7, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 7 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:32.427721 26077 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 5, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 5 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:32.269598 29605 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:32.381104 29628 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:32.442608 26250 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
W20260924 14:30:32.443929 26080 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 139) took 27909 ms (client timeout 15000 ms). Trace:
I20260924 14:30:32.396960 29532 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:32.436465 29540 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:32.379398 29559 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:32.388190 29718 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:32.452474 26250 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260924 14:30:32.447811 26080 rpcz_store.cc:269] 0924 14:30:04.534289 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:04.534656 (+   367us) service_pool.cc:227] Handling call
0924 14:30:32.443833 (+27909177us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:30:25.679295 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:30:25.688521 (+  9226us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:30:25.688572 (+    51us) write_op.cc:435] Acquired schema lock
0924 14:30:25.688583 (+    11us) tablet.cc:660] Decoding operations
0924 14:30:25.689234 (+   651us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:30:25.689316 (+    82us) write_op.cc:641] Partition lock acquired for write op
0924 14:30:25.689344 (+    28us) tablet.cc:683] Acquiring locks for 3 operations
0924 14:30:25.689771 (+   427us) tablet.cc:699] Row locks acquired
0924 14:30:25.689800 (+    29us) write_op.cc:260] PREPARE: finished
0924 14:30:25.689985 (+   185us) write_op.cc:270] Start()
0924 14:30:25.690119 (+   134us) write_op.cc:276] Timestamp: P: 1790260225689920 usec, L: 0
0924 14:30:25.690150 (+    31us) op_driver.cc:331] REPLICATION: starting
0924 14:30:25.690858 (+   708us) log.cc:842] Serialized 1325 byte log entry
0924 14:30:25.873021 (+182163us) op_driver.cc:444] REPLICATION: finished
0924 14:30:26.094017 (+220996us) write_op.cc:301] APPLY: starting
0924 14:30:26.094121 (+   104us) tablet.cc:1365] starting BulkCheckPresence
0924 14:30:26.094309 (+   188us) tablet.cc:1368] finished BulkCheckPresence
0924 14:30:26.094426 (+   117us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:30:26.095103 (+   677us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:30:26.095397 (+   294us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0924 14:30:26.095481 (+    84us) write_op.cc:312] APPLY: finished
0924 14:30:26.095865 (+   384us) log.cc:842] Serialized 34 byte log entry
0924 14:30:26.096070 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:26.096177 (+   107us) write_op.cc:454] Released schema lock
0924 14:30:26.096243 (+    66us) write_op.cc:341] FINISH: Updating metrics
0924 14:30:26.096579 (+   336us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:26.096614 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":124898,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":199536,"apply.run_cpu_time_us":2974,"apply.run_wall_time_us":2974,"num_ops":3,"prepare.queue_time_us":186851,"prepare.run_cpu_time_us":20997,"prepare.run_wall_time_us":107555,"raft.queue_time_us":70202,"raft.run_cpu_time_us":77050,"raft.run_wall_time_us":90142,"replication_time_us":182753,"thread_start_us":305043,"threads_started":4,"wal-append.queue_time_us":83310,"wal-append.run_cpu_time_us":40035,"wal-append.run_wall_time_us":1336039}]]}
W20260924 14:30:32.453951 29749 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260924 14:30:32.387862 29650 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:32.457504 29532 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:32.488798 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.430912 26077 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 4 to 6, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:32.544203 26077 catalog_manager.cc:5495] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:32.546447 26077 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 7 to 8, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 8 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:32.438917 29679 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:32.571344 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:32.552258 26077 catalog_manager.cc:5495] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:32.567585 26254 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:32.615989 26077 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 7, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 7 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:32.619185 26077 catalog_manager.cc:5495] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 8, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 8 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:32.667434 26077 catalog_manager.cc:5495] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 6, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 6 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:32.727897 26077 catalog_manager.cc:5495] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
W20260924 14:30:32.733315 27191 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 940) took 1340 ms (client timeout 1774 ms). Trace:
W20260924 14:30:32.734193 27191 rpcz_store.cc:269] 0924 14:30:31.392726 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.392852 (+   126us) service_pool.cc:227] Handling call
0924 14:30:32.732292 (+1339440us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400130e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:32.733218 (+   926us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":400247680}
I20260924 14:30:32.741962 26402 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
W20260924 14:30:32.747975 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.122s	sys 0.226s
W20260924 14:30:32.748492 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.122s	sys 0.226s
I20260924 14:30:32.749140 26077 catalog_manager.cc:5495] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 1 to 3, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 3 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:32.751732 26077 catalog_manager.cc:5495] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 6, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 6 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:32.454741 29628 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:32.770169 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:32.772092 26404 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.012s	sys 0.037s
I20260924 14:30:32.727134 29559 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:32.783178 26254 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:32.784046 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.784783 26250 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:32.790838 26403 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:32.481969 29707 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:32.801568 26256 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.021s	sys 0.023s
W20260924 14:30:32.808790 26256 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.021s	sys 0.023s
W20260924 14:30:32.790145 29663 consensus_meta.cc:287] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.928s	user 0.013s	sys 0.004s
I20260924 14:30:32.474399 29540 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:32.774580 26404 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.012s	sys 0.038s
I20260924 14:30:32.831122 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260924 14:30:32.813658 29664 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.446s	user 0.006s	sys 0.011s
I20260924 14:30:32.759326 26077 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 7, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 7 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
W20260924 14:30:32.821542 29627 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.991s	user 0.012s	sys 0.001s
W20260924 14:30:32.810681 29617 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.016s	sys 0.037s
I20260924 14:30:32.812312 29571 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:32.787024 29559 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:32.838307 29716 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.001s
W20260924 14:30:32.843582 29689 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.660s	user 0.010s	sys 0.000s
I20260924 14:30:32.858253 26250 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
W20260924 14:30:32.845268 29617 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.016s	sys 0.038s
I20260924 14:30:32.847462 26254 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260924 14:30:32.846220 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.104s	sys 0.127s
W20260924 14:30:32.876875 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.104s	sys 0.127s
I20260924 14:30:32.852687 29643 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:32.872417 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:32.835085 29663 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:32.874413 29676 consensus_meta.cc:287] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.078s	user 0.011s	sys 0.004s
W20260924 14:30:32.845340 29695 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.026s	sys 0.025s
I20260924 14:30:32.857311 29581 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260924 14:30:32.873322 29617 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:32.889437 29661 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.013s	sys 0.013s
I20260924 14:30:32.902273 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:32.918430 29661 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.013s	sys 0.013s
I20260924 14:30:32.918689 26330 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:32.919286 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:32.919893 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:32.920467 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:32.921015 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:32.921605 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:32.922144 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:32.922622 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:32.845108 26077 catalog_manager.cc:5495] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 2 to 3, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 3 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:32.858047 29571 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:32.889171 29643 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:32.872794 29716 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:32.902436 29695 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.026s	sys 0.025s
I20260924 14:30:32.891356 29647 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:32.899360 29586 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:32.873024 29689 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:32.847920 29664 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:32.923305 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:33.005416 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:33.033802 27189 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260924 14:30:32.973285 26323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 613) took 1246 ms (client timeout 1626 ms). Trace:
W20260924 14:30:33.180476 26249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 849) took 1338 ms (client timeout 1544 ms). Trace:
I20260924 14:30:33.126390 26297 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:33.182736 26249 rpcz_store.cc:269] 0924 14:30:31.841962 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.846230 (+  4268us) service_pool.cc:227] Handling call
0924 14:30:33.180154 (+1333924us) spinlock_profiling.cc:243] Waited 96.8 ms on lock 0x7b54007b07d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:33.180381 (+   227us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":222631168}
I20260924 14:30:33.179535 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:33.184458 26329 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:33.184948 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 953) took 1186 ms. Trace:
W20260924 14:30:33.182729 26323 rpcz_store.cc:269] 0924 14:30:31.727227 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.729859 (+  2632us) service_pool.cc:227] Handling call
0924 14:30:32.973016 (+1243157us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b540001f4d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:32.973191 (+   175us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":782467968}
I20260924 14:30:33.166185 26330 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:33.185479 27189 rpcz_store.cc:276] 0924 14:30:31.998433 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:33.031439 (+1033006us) service_pool.cc:227] Handling call
0924 14:30:33.033359 (+  1920us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:33.184636 (+151277us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:33.184893 (+   257us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:32.940804 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:32.993911 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1188) took 1065 ms. Trace:
I20260924 14:30:32.897920 29650 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:33.164878 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:33.000360 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:33.192615 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:33.246404 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:33.204777 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:33.039017 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:32.958088 29755 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260924 14:30:32.956525 29759 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260924 14:30:32.984504 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:33.188712 26326 rpcz_store.cc:276] 0924 14:30:31.928168 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.938185 (+ 10017us) service_pool.cc:227] Handling call
0924 14:30:32.992991 (+1054806us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:30:32.993286 (+   295us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:32.993515 (+   229us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:32.993552 (+    37us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:32.993607 (+    55us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:32.993649 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:32.993718 (+    69us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:32.993846 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":24971136}
I20260924 14:30:33.260249 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:33.260952 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 72790.9us
I20260924 14:30:32.923873 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.256083 26398 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:32.987897 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 611) took 1267 ms. Trace:
W20260924 14:30:33.188621 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62333us
I20260924 14:30:33.236135 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:33.264734 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:32.989040 29609 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.938s	user 0.023s	sys 0.000s
W20260924 14:30:33.250993 26327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1184) took 1324 ms (client timeout 1685 ms). Trace:
W20260924 14:30:33.164450 29466 consensus_meta.cc:287] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.961s	user 0.010s	sys 0.001s
I20260924 14:30:33.158131 29533 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:33.092471 29599 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.891s	user 0.012s	sys 0.011s
I20260924 14:30:33.008473 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 561) took 1167 ms. Trace:
W20260924 14:30:33.193641 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 188354us
W20260924 14:30:32.952407 26296 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.007s	sys 0.019s
W20260924 14:30:33.318393 26296 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.012s	sys 0.025s
I20260924 14:30:33.156314 29714 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:32.991938 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:32.987238 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 948) took 1245 ms. Trace:
I20260924 14:30:33.315820 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:33.321117 27190 rpcz_store.cc:276] 0924 14:30:31.742233 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.808180 (+ 65947us) service_pool.cc:227] Handling call
0924 14:30:31.852873 (+ 44693us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:32.017346 (+164473us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:32.017378 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:32.017421 (+    43us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:32.017466 (+    45us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:32.986927 (+969461us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400003054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:32.986992 (+    65us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:32.987165 (+   173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":377707776}
I20260924 14:30:33.256093 26331 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:33.197479 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:33.154788 29668 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:33.323316 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1187) took 1395 ms. Trace:
I20260924 14:30:33.323743 26332 rpcz_store.cc:276] 0924 14:30:31.927849 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.929451 (+  1602us) service_pool.cc:227] Handling call
0924 14:30:32.991744 (+1062293us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:30:33.323256 (+331512us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":27625728}
I20260924 14:30:32.958305 29598 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:33.259395 26257 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.001s	sys 0.038s
I20260924 14:30:33.265059 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:33.325039 26257 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.001s	sys 0.038s
I20260924 14:30:32.938076 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.156127 29611 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:33.325017 27181 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:33.268278 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:33.251497 26324 raft_consensus.cc:2379] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:30:33.182755 29536 consensus_meta.cc:287] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.845s	user 0.010s	sys 0.002s
I20260924 14:30:32.997900 29573 raft_consensus.cc:3032] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:30:33.265625 26325 rpcz_store.cc:276] 0924 14:30:31.720899 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.721151 (+   252us) service_pool.cc:227] Handling call
0924 14:30:32.987701 (+1266550us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54005d8c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:32.987829 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":64027904}
W20260924 14:30:33.311539 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 387713us
W20260924 14:30:33.331431 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 393394us
I20260924 14:30:32.999354 29663 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:32.890239 29676 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.246987 29718 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:32.997920 29628 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:32.952913 29559 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:33.317101 27194 rpcz_store.cc:276] 0924 14:30:31.841112 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.841306 (+   194us) service_pool.cc:227] Handling call
0924 14:30:32.732477 (+891171us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:32.732756 (+   279us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:32.732795 (+    39us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:32.732868 (+    73us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:32.732914 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:32.732992 (+    78us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:33.001463 (+268471us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b5400130e50. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:33.001575 (+   112us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2048622976}
I20260924 14:30:33.216248 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:33.396438 26249 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:33.193424 26402 raft_consensus.cc:2396] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:33.226066 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:32.916697 29673 consensus_meta.cc:287] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.781s	user 0.009s	sys 0.002s
W20260924 14:30:33.314435 26327 rpcz_store.cc:269] 0924 14:30:31.926733 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.929146 (+  2413us) service_pool.cc:227] Handling call
0924 14:30:33.000156 (+1071010us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:30:33.250924 (+250768us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":26036608}
W20260924 14:30:33.246791 29678 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.261s	user 0.006s	sys 0.006s
I20260924 14:30:33.198376 26252 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:32.979620 29621 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:33.327867 27185 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
W20260924 14:30:32.957280 29752 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260924 14:30:33.329306 26323 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:33.330247 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1185) took 1403 ms. Trace:
I20260924 14:30:33.332891 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:33.392755 29755 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260924 14:30:32.954797 29615 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.358s
I20260924 14:30:32.953666 29560 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:33.454985 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.030s	sys 0.058s
W20260924 14:30:33.455466 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.031s	sys 0.058s
W20260924 14:30:33.337276 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 72632.8us
I20260924 14:30:33.354893 26332 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:33.254918 29650 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:33.311498 29726 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:33.315585 29647 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:33.385469 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:33.378400 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45694.7us
W20260924 14:30:33.269727 28867 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.259s	user 0.008s	sys 0.004s
I20260924 14:30:33.386915 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:33.695477 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:33.687903 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:32.936733 29758 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.421s
W20260924 14:30:33.683013 29733 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.402s	user 0.009s	sys 0.009s
I20260924 14:30:33.331629 29573 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:33.250319 29586 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:33.382361 29763 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:33.402966 29621 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:33.337692 29598 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.403152 29673 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:33.338356 29536 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:33.405776 29697 consensus_meta.cc:287] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.227s	user 0.005s	sys 0.007s
I20260924 14:30:33.407055 29678 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:33.402967 29628 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:33.409952 29689 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:33.410816 29645 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:33.411275 29716 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:33.412693 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 616) took 1680 ms. Trace:
I20260924 14:30:33.412961 29676 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:33.423863 29700 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:33.427268 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1043) took 1908 ms. Trace:
I20260924 14:30:33.744887 26257 rpcz_store.cc:276] 0924 14:30:31.519148 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.519616 (+   468us) service_pool.cc:227] Handling call
0924 14:30:31.569035 (+ 49419us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:31.683111 (+114076us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:31.683161 (+    50us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:32.148150 (+464989us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b2c001c3658. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:32.148644 (+   494us) log.cc:842] Serialized 26 byte log entry
0924 14:30:32.725894 (+577250us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:33.325522 (+599628us) log.cc:842] Serialized 14 byte log entry
0924 14:30:33.325748 (+   226us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:33.325812 (+    64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:33.426933 (+101121us) raft_consensus.cc:1671] finished
0924 14:30:33.427043 (+   110us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:33.427192 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":103807,"prepare.queue_time_us":673399,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":114447,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":29708,"spinlock_wait_cycles":27886976,"thread_start_us":922446,"threads_started":2,"wal-append.queue_time_us":681708}
I20260924 14:30:33.427997 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1300) took 1842 ms. Trace:
I20260924 14:30:33.747092 26399 rpcz_store.cc:276] 0924 14:30:31.585221 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.636354 (+ 51133us) service_pool.cc:227] Handling call
0924 14:30:31.833554 (+197200us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:32.045439 (+211885us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:33.005408 (+959969us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540036bad4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:33.427723 (+422315us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540036bad0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:33.427921 (+   198us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":812029568}
I20260924 14:30:33.429400 29571 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:33.695508 29726 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.433563 29664 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:33.436177 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 103378us
I20260924 14:30:33.712622 29733 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:32.968554 29640 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:33.329869 29668 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:33.441140 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.030s	sys 0.074s
W20260924 14:30:33.778131 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.030s	sys 0.074s
W20260924 14:30:33.778437 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 1.01637e+06us
I20260924 14:30:33.441637 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1279) took 1259 ms. Trace:
I20260924 14:30:33.779151 26253 rpcz_store.cc:276] 0924 14:30:32.182251 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:32.205332 (+ 23081us) service_pool.cc:227] Handling call
0924 14:30:33.180718 (+975386us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:33.180965 (+   247us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:33.181011 (+    46us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:33.181070 (+    59us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:33.181114 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:33.181188 (+    74us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:33.441421 (+260233us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b54007b07d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:33.441566 (+   145us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2242365312}
I20260924 14:30:33.344143 29718 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:30:33.396674 29734 consensus_meta.cc:287] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.269s	user 0.007s	sys 0.006s
I20260924 14:30:33.329788 29606 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:33.344304 29695 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:30:33.269549 29587 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.788s	user 0.007s	sys 0.008s
W20260924 14:30:33.782387 29532 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.037s	user 0.009s	sys 0.009s
I20260924 14:30:33.782496 29587 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.782848 29532 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:33.784685 29587 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:33.784802 29532 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:33.312942 29609 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.543151 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1038) took 3667 ms. Trace:
I20260924 14:30:33.785602 26252 rpcz_store.cc:276] 0924 14:30:29.876102 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:30.925230 (+1049128us) service_pool.cc:227] Handling call
0924 14:30:30.925561 (+   331us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:33.402463 (+2476902us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:33.542853 (+140390us) spinlock_profiling.cc:243] Waited 2.27 s on lock 0x7b540015ead4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:33.543073 (+   220us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5226296064}
W20260924 14:30:33.631834 29677 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.007s	sys 0.009s
W20260924 14:30:33.618892 26371 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.002s	sys 0.024s
W20260924 14:30:33.786630 26371 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.003s	sys 0.029s
W20260924 14:30:33.464994 26256 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.035s	sys 0.060s
I20260924 14:30:33.679750 27192 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:33.346094 29762 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:33.463568 29650 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:33.329710 29617 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:33.329777 29599 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:33.441063 29679 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260924 14:30:33.620810 29185 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.006s	sys 0.001s
W20260924 14:30:33.801230 29185 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.008s	sys 0.002s
I20260924 14:30:33.390851 29714 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.337534 29611 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.471968 29442 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260924 14:30:33.804208 26325 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:33.681020 26249 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []
I20260924 14:30:33.317694 29466 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.805197 29673 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:33.805974 29599 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:33.805734 29668 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:33.806303 29466 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:33.542066 29605 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.000s	sys 0.008s
I20260924 14:30:33.806617 27182 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
W20260924 14:30:33.806967 29605 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.000s	sys 0.010s
I20260924 14:30:33.807268 29668 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:33.681483 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.682209 26323 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260924 14:30:33.692719 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:33.808831 26406 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:33.788705 29734 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:33.812081 27189 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:33.812932 29734 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:33.697113 26400 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:33.696096 27181 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:33.789060 29643 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:33.701506 26397 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:33.824155 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:33.824826 26252 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:33.788286 29606 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:33.825567 29621 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:33.826970 29606 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:33.682994 29647 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:33.695853 29540 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.836s	user 0.012s	sys 0.002s
I20260924 14:30:33.827590 29716 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:33.827783 29540 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:33.828329 29716 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.781412 29640 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:33.829984 29540 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:33.727656 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:33.830262 29716 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:33.830504 29640 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:33.731741 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:33.734879 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:33.831809 27186 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:33.740460 26330 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:33.832444 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:33.833235 26399 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:33.740332 26331 rpcz_store.cc:276] 0924 14:30:31.732181 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.732365 (+   184us) service_pool.cc:227] Handling call
0924 14:30:31.733746 (+  1381us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:33.322352 (+1588606us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:33.412375 (+ 90023us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540001ead4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:33.412607 (+   232us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3491410176}
I20260924 14:30:33.727761 29661 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:33.833762 29661 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.827020 29621 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:33.684015 29560 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:33.827670 29647 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:30:33.836774 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 155443us
I20260924 14:30:33.836896 29560 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:33.749751 29593 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:33.837193 29682 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:33.837608 29593 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:33.838094 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:33.838328 29682 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:33.838680 29682 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:33.430069 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1157) took 1583 ms. Trace:
I20260924 14:30:33.760694 29586 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:33.764319 26255 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:33.764822 29536 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:33.704165 28867 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.765890 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1048) took 2184 ms. Trace:
I20260924 14:30:33.751236 29689 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260924 14:30:33.749120 29573 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:33.744633 29700 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:33.768262 29598 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:33.774163 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.839419 29682 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.848379 26400 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
W20260924 14:30:33.848619 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 74547.3us
I20260924 14:30:33.849078 29682 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:33.849418 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:33.850301 26221 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.435371 26324 rpcz_store.cc:276] 0924 14:30:31.927181 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.929614 (+  2433us) service_pool.cc:227] Handling call
0924 14:30:33.038800 (+1109186us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:30:33.330194 (+291394us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":26187520}
W20260924 14:30:33.851207 29753 consensus_queue.cc:1159] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "45ab97b763804767846e9d27a01b561f" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
I20260924 14:30:33.851643 29606 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:33.851945 29643 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:33.852376 29606 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260924 14:30:33.775849 29707 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.022s	sys 0.062s
W20260924 14:30:33.853077 29707 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.022s	sys 0.062s
I20260924 14:30:33.852782 29679 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:33.777781 29457 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260924 14:30:33.853747 27126 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:33.787220 26256 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.977s	user 0.050s	sys 0.064s
W20260924 14:30:33.786918 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 917619us
I20260924 14:30:33.742714 29628 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:33.792853 26401 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:33.780773 29718 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:33.839856 26398 rpcz_store.cc:276] 0924 14:30:31.846875 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.847032 (+   157us) service_pool.cc:227] Handling call
0924 14:30:31.847328 (+   296us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:33.263949 (+1416621us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:33.429737 (+165788us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400336654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:33.429972 (+   235us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3234172800}
I20260924 14:30:33.796468 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:33.774461 29763 consensus_queue.cc:1046] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260924 14:30:33.800168 29778 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:33.680599 29551 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:33.816386 26406 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:33.819236 29609 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:33.823354 27191 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: []
W20260924 14:30:33.682315 29633 consensus_meta.cc:287] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.521s	user 0.019s	sys 0.005s
I20260924 14:30:33.740625 29663 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:33.839634 29466 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:33.840184 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:33.775314 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 481202us
I20260924 14:30:33.883996 26249 raft_consensus.cc:2354] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10.
I20260924 14:30:33.885648 26252 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:33.795413 29650 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:33.887920 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 91561.6us
I20260924 14:30:33.852391 29643 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:33.888586 29643 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:33.888792 27123 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:33.889410 29605 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:33.889931 29605 raft_consensus.cc:686] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:33.890460 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:33.859092 27181 raft_consensus.cc:2396] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260924 14:30:33.843703 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:33.892185 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:33.840601 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:33.863837 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:33.839718 29777 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:33.890743 29605 consensus_queue.cc:237] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.868422 26401 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:33.897361 29757 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:33.897826 29757 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:33.865087 27186 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:33.846745 29573 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:33.898171 29757 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:33.847513 29700 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.871613 29628 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:33.899839 29700 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:33.845497 26251 rpcz_store.cc:276] 0924 14:30:31.581372 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.581502 (+   130us) service_pool.cc:227] Handling call
0924 14:30:31.581794 (+   292us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:31.683720 (+101926us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:31.683778 (+    58us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:31.830225 (+146447us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b180036ade0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a
0924 14:30:32.029253 (+199028us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b2c001c3658. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:32.029747 (+   494us) log.cc:842] Serialized 26 byte log entry
0924 14:30:32.281732 (+251985us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:32.281787 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:32.281903 (+   116us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:33.765672 (+1483769us) raft_consensus.cc:1671] finished
0924 14:30:33.765695 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:33.765832 (+   137us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":101529,"raft.queue_time_us":113402,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":21191,"spinlock_wait_cycles":33317504,"thread_start_us":251593,"threads_started":1,"wal-append.queue_time_us":726296}
I20260924 14:30:33.841696 29726 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:33.841871 29611 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:33.845371 28867 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:33.891103 29663 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:33.841516 29661 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:33.904970 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:33.905781 26401 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:33.906443 29663 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:33.884979 29633 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.908008 26328 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:33.908496 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:33.908844 29633 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:33.909368 26328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1189) took 1980 ms (client timeout 1676 ms). Trace:
I20260924 14:30:33.909538 27126 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:33.909772 26328 rpcz_store.cc:269] 0924 14:30:31.928456 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.929268 (+   812us) service_pool.cc:227] Handling call
0924 14:30:32.984295 (+1055027us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:30:33.909163 (+924868us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54007cf1d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:33.909293 (+   130us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1497924736}
I20260924 14:30:33.883651 29778 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.912053 26405 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:33.913067 29778 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:33.913720 29677 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:33.840458 29185 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260924 14:30:33.873327 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 843) took 2058 ms. Trace:
I20260924 14:30:33.871261 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:33.874437 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1276) took 1693 ms. Trace:
I20260924 14:30:33.915618 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:33.915611 26255 rpcz_store.cc:276] 0924 14:30:31.814799 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:31.830663 (+ 15864us) service_pool.cc:227] Handling call
0924 14:30:33.873164 (+2042501us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b540015e5d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:33.873263 (+    99us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4356801792}
I20260924 14:30:33.876698 26397 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:33.916589 26399 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:33.916755 29689 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:30:33.883926 29551 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.919560 29551 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:33.920264 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 56544.5us
I20260924 14:30:33.920647 29536 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:33.921033 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.921223 29536 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.922281 29689 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:33.922791 29689 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:33.879674 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:33.880303 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:33.840842 29771 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260924 14:30:33.923754 29695 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:33.881196 29586 raft_consensus.cc:3032] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:33.824841 29714 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:33.924448 29586 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:33.923766 26253 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:33.925328 29627 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:33.923776 29678 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260924 14:30:33.926070 29627 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.926446 29689 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.928607 29627 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:33.924028 26254 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:33.842641 29733 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:33.929926 29586 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.875733 29560 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:33.931174 29586 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:33.930972 29560 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.897104 29540 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:30:33.931910 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:33.932246 29540 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:33.932998 29560 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:33.839596 29593 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:33.933840 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:33.854833 29501 consensus_queue.cc:1159] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c22052275034c369b7f7dca086fbd1e" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:30:33.877565 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:33.893779 27125 leader_election.cc:398] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10.
I20260924 14:30:33.935379 26406 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:33.935684 27125 leader_election.cc:401] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260924 14:30:33.921806 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 81296.2us
I20260924 14:30:33.894294 29754 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:33.936350 29754 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:33.936582 29536 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:33.936772 29754 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:33.936878 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.872912 26371 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.839848 29673 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260924 14:30:33.932744 29540 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:33.938091 29560 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:33.938002 29185 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:30:33.938411 29673 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:33.939193 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 66347.3us
I20260924 14:30:33.939293 29673 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:33.939826 29673 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:33.930502 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:33.924388 29695 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:33.916265 26256 rpcz_store.cc:276] 0924 14:30:32.181314 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:32.190928 (+  9614us) service_pool.cc:227] Handling call
0924 14:30:32.191213 (+   285us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:32.215911 (+ 24698us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:30:32.809999 (+594088us) log.cc:842] Serialized 14 byte log entry
0924 14:30:33.859642 (+1049643us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:33.859723 (+    81us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:33.859787 (+    64us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:33.874229 (+ 14442us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b54002cbfd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:33.874283 (+    54us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:33.874386 (+   103us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4738,"prepare.queue_time_us":1104400,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":34962,"raft.queue_time_us":4859,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":7594,"spinlock_wait_cycles":56096000,"thread_start_us":1625946,"threads_started":2,"wal-append.queue_time_us":942349}
I20260924 14:30:33.933974 29679 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:33.941458 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.954042 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:33.956018 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:33.956835 26255 raft_consensus.cc:2362] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8. Re-sending same reply.
I20260924 14:30:33.959879 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:33.960649 26255 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:33.960817 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:33.941848 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:33.942526 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 50406.7us
I20260924 14:30:33.943881 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.964761 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.869661 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:33.944581 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 170917us
I20260924 14:30:33.967602 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:33.968425 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:33.971495 26252 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:33.968801 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.968870 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:33.974138 26255 raft_consensus.cc:2379] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:33.970804 26257 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:33.976747 26256 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:33.977756 26253 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:33.980698 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.978217 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.990690 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.006224 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.010113 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.015195 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:33.984565 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:34.016021 26258 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:34.016340 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.012768 29668 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:34.018392 29668 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:34.018714 29677 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:34.019358 29677 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:34.019245 29668 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.020484 29677 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:33.940797 29673 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:34.018420 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 108959us
W20260924 14:30:34.023195 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 129519us
I20260924 14:30:34.013497 29778 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:34.023594 29673 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:34.023819 29778 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:34.024046 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:34.024284 29778 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:34.024758 27126 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:34.025209 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:34.011436 29633 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:34.026777 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:33.948262 29679 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:34.027499 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:34.028105 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 65277.9us
I20260924 14:30:34.028892 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:33.948403 29606 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:33.948058 29697 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:33.980108 29560 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:33.981721 29714 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:34.005280 29627 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:34.026917 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:33.979318 29696 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:34.033934 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:34.035020 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.037778 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.049960 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:33.979473 29661 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:34.030512 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:33.951787 29753 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260924 14:30:34.059487 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:34.063776 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.064663 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.062340 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:34.041486 29606 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:34.069309 26371 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.116s (SENT)
I20260924 14:30:34.057044 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:34.065366 26402 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:34.028291 29678 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:34.031680 29536 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:34.056380 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:34.078581 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.078862 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.057819 29621 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:34.041706 29611 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:34.043326 29581 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:34.097695 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:34.073652 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:34.077716 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:34.079345 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:34.080497 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50101.7us
I20260924 14:30:34.076957 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.079762 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.062243 29609 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:34.100122 26328 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:34.047622 29627 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:34.100569 29627 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:34.043251 29457 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:34.100888 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.057116 29716 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:34.055976 29438 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:34.102733 29438 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:34.103132 29438 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:34.103417 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.103864 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.104379 26249 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:30:34.104794 26254 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:34.100888 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.090845 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.056550 29661 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:34.066334 29645 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:34.067163 29560 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:34.058017 29758 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260924 14:30:34.066130 29734 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:34.099020 26330 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:34.108542 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.072769 29606 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:34.135211 26330 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:34.109390 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:34.110671 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 46156.3us
I20260924 14:30:34.110797 27181 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:34.073114 29501 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:34.113722 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.115976 26295 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:34.137749 27181 raft_consensus.cc:595] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:34.117238 29664 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:34.138192 27181 raft_consensus.cc:603] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:34.075647 29640 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:34.096493 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:34.139372 26331 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:34.140532 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.142305 26331 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:34.122357 29540 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:34.124195 29782 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260924 14:30:34.124665 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:34.069108 29559 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260924 14:30:34.127642 29762 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:30:34.062423 29707 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260924 14:30:34.127964 29560 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.145323 29560 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:34.092231 29621 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:34.146137 29621 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:34.073549 29617 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:34.092746 26326 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:34.094681 29697 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:34.142014 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:34.134262 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:34.152766 27181 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0756687d99e643c094531124c9438376"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:34.155768 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:34.164364 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.115252 29676 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:34.075986 29714 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260924 14:30:34.166983 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.167733 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.167646 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.119903 29661 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:34.113137 29726 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260924 14:30:34.182152 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 66331.6us
I20260924 14:30:34.118135 29647 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:34.171500 26254 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:34.135509 29457 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:34.172469 29559 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:34.172868 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:34.173125 29707 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:34.173825 29185 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:34.174255 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.082230 29695 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.212548 26369 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.075737 29643 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.166832 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.112426 29780 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:34.178169 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.187330 27192 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:34.194867 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.158329 29501 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:34.112959 29763 consensus_queue.cc:1046] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260924 14:30:34.162067 29288 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260924 14:30:34.198596 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.110466 29611 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:34.201278 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 76713.6us
I20260924 14:30:34.234622 27190 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:30:34.240545 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:34.203065 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.198917 29790 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:34.172533 29762 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:34.205744 26250 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:34.174607 27181 raft_consensus.cc:595] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:34.175199 29757 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.175472 29777 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:34.216580 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.219573 27194 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:34.197929 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.225785 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.182569 29617 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:34.108804 29679 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:34.222100 29501 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:34.287927 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47508.6us
I20260924 14:30:34.225071 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.197142 29559 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:34.168421 29718 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:34.244439 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 209483us
I20260924 14:30:34.212708 29726 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Stepping down as leader of term 9
W20260924 14:30:34.298652 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 142522us
I20260924 14:30:34.246469 27181 raft_consensus.cc:603] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:34.175195 29442 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:34.211790 29605 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:34.305119 27182 raft_consensus.cc:2379] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 7: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:30:34.305382 26371 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.145s (SENT)
I20260924 14:30:34.305883 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:34.203857 29707 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:34.227953 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.250514 27186 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:34.243300 29714 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:34.229125 29697 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:34.246019 29762 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:34.233276 29700 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:34.242648 29628 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:34.205214 29185 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:34.245323 29668 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:34.308192 26254 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:34.310458 29419 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:30:34.310506 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 82632.9us
I20260924 14:30:34.313716 29691 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:34.358276 29786 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260924 14:30:34.358475 29691 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.308562 29442 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.249423 29790 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:34.360138 29438 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.360235 29790 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.360912 29442 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:34.315783 26329 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:34.317782 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.305159 29605 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:34.301131 29733 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:34.324424 29185 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:34.333047 29781 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
W20260924 14:30:34.333971 29785 log.cc:924] Time spent T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260924 14:30:34.433809 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:34.334756 29783 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260924 14:30:34.434629 26324 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:34.415609 29691 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:34.435580 29650 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.000s
I20260924 14:30:34.436101 29650 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.437171 26325 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:34.438314 29650 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:34.438190 29778 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:34.337360 29559 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.439556 29609 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:34.440049 29559 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:34.440310 26401 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:34.412752 29605 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:34.299425 29726 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:34.302084 29587 consensus_queue.cc:1046] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260924 14:30:34.304320 29457 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.332113 29697 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:34.411975 29733 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:34.295670 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.232573 29753 consensus_queue.cc:1046] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260924 14:30:34.320950 27123 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.364459 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.365097 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.368221 29438 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:34.320633 29645 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:34.303876 29617 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.315954 29714 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.442968 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:34.435026 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:34.461743 29621 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.462740 26400 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:34.351712 29676 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.122s
I20260924 14:30:34.233516 29780 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.362712 29679 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:34.306097 29629 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260924 14:30:34.434758 29757 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:34.602988 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 238591us
I20260924 14:30:34.569473 26404 raft_consensus.cc:1261] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:30:34.481057 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:34.558723 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:34.601298 26251 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:34.499305 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:34.530066 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.607883 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:34.608501 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 302851us
W20260924 14:30:34.610472 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 167566us
I20260924 14:30:34.606334 27188 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
W20260924 14:30:34.611896 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 112711us
I20260924 14:30:34.539628 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:34.612340 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 316747us
I20260924 14:30:34.612931 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:34.613432 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.614382 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.620378 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:34.533598 29673 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.505s	user 0.005s	sys 0.005s
I20260924 14:30:34.621717 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.629849 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:34.626629 26369 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:34.631605 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:34.627202 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.628731 26332 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:34.570824 29647 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:34.606108 29733 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:34.635608 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.466115 29758 consensus_queue.cc:1046] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.157s
I20260924 14:30:34.732383 29758 raft_consensus.cc:3032] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:34.732797 29758 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:34.616043 29466 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:30:34.620630 29540 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:34.620297 29532 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:34.637454 29628 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260924 14:30:34.679318 29791 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
I20260924 14:30:34.625634 29801 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:34.624423 29611 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:34.680404 29695 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:34.462741 29707 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:34.630378 29718 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:34.687989 29778 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:34.689010 29716 consensus_meta.cc:287] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.587s	user 0.012s	sys 0.000s
I20260924 14:30:34.693763 29677 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:34.696732 29621 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:34.640928 29700 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:34.701005 29501 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.616088 29726 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:34.638561 29757 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:34.623044 29609 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:30:34.622882 29419 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:34.623773 29770 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:34.634289 29640 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:34.765851 29700 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:34.775676 29700 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:30:34.711035 29586 consensus_meta.cc:287] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.779s	user 0.011s	sys 0.000s
W20260924 14:30:34.711813 29627 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.000s
W20260924 14:30:34.779600 29754 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.842s	user 0.009s	sys 0.006s
W20260924 14:30:34.764434 29661 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260924 14:30:34.780067 29627 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.780494 29661 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.781543 26331 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:34.782321 29627 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:34.783939 29661 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:34.785532 29780 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:34.786522 29340 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:34.786965 29340 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:34.712455 29617 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:34.477795 29762 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.788021 29762 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:34.787844 29340 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.789024 29340 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:34.712967 29606 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.001s
I20260924 14:30:34.791728 29606 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.792277 29645 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:34.793733 29606 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:34.714133 29714 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:34.625648 29794 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:30:34.724985 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 114777us
W20260924 14:30:34.703732 29581 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.595s	user 0.005s	sys 0.005s
I20260924 14:30:34.794812 29581 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10.
I20260924 14:30:34.794976 26258 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:34.616537 29598 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:34.769310 29726 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:34.731232 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 95688.6us
I20260924 14:30:34.730050 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:34.735523 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 109997us
I20260924 14:30:34.797585 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:34.798491 26297 leader_election.cc:334] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.876s (SENT)
I20260924 14:30:34.749302 29758 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:34.799808 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:34.800211 29714 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:34.800709 29714 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:34.801038 29714 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:34.751035 29628 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:34.801648 29628 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:30:34.803637 29645 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:34.804538 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:34.804790 29645 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:34.805148 29645 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:34.805388 26402 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:34.805929 29645 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.811025 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:34.739530 29559 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:34.825217 27194 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:34.748023 29532 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:34.764014 29716 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:34.709681 29605 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:34.811272 26328 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
W20260924 14:30:34.763772 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 194393us
I20260924 14:30:34.848734 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:34.849654 26402 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:34.850736 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:34.762867 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 441945us
W20260924 14:30:34.817489 26371 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.800s (SENT)
I20260924 14:30:34.794405 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1068) took 1062 ms. Trace:
I20260924 14:30:34.851966 26250 rpcz_store.cc:276] 0924 14:30:33.732234 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:33.732463 (+   229us) service_pool.cc:227] Handling call
0924 14:30:33.732858 (+   395us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:34.248048 (+515190us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:34.794179 (+546131us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54004fa454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:34.794348 (+   169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1087072128}
I20260924 14:30:34.852646 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.853471 26323 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:34.825358 27182 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:34.826164 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 199015us
I20260924 14:30:34.825256 27188 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:34.832288 27189 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:34.854625 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.832556 26398 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:34.851228 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:34.807217 29645 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:34.838392 26400 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:34.840945 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:34.847492 26331 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:34.848024 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.859403 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.800251 29758 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:34.862984 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:34.866483 27186 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:34.768712 29501 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:34.873732 26326 raft_consensus.cc:2379] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:34.840319 29532 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:34.882798 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.884337 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.888319 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:34.888690 26323 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:34.885658 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:34.889503 26249 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:34.864847 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:34.891492 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.881147 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.892560 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.893847 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.889456 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.895004 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.897564 26330 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:30:34.898375 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47265.2us
I20260924 14:30:34.902216 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.903416 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.896692 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:34.904640 26331 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:34.897531 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:34.874586 29586 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:34.906810 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.906924 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.910961 27188 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:34.911486 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:34.908783 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.920661 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.780086 29754 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:34.879359 29679 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:34.904045 27186 raft_consensus.cc:2362] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11. Re-sending same reply.
I20260924 14:30:34.908738 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.962138 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:34.961580 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52907.3us
I20260924 14:30:34.963331 27191 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:34.923615 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.924947 27190 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 4. Current term is 5. Ops: []
I20260924 14:30:34.922912 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.961236 26327 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:34.922577 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.972360 26325 raft_consensus.cc:2396] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260924 14:30:34.964015 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:34.968168 26255 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260924 14:30:34.965133 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:34.974673 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:34.972105 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.973418 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:34.976475 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.971982 26256 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:34.972415 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.968860 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:34.964962 27185 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:34.980091 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.973001 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.981629 26254 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:34.982496 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.982985 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:34.983166 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:34.984294 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:34.983279 26252 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:34.985397 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.986349 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:34.971361 27182 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:34.987360 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.878545 29571 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260924 14:30:34.907303 29716 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260924 14:30:34.989123 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 91682.4us
I20260924 14:30:35.025549 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:35.043915 26252 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:35.044620 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:35.021996 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:35.045768 26255 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:35.027869 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:34.903680 29559 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:34.988719 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:34.980899 29185 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.000s
I20260924 14:30:35.017385 26256 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:34.906255 29814 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.115s
I20260924 14:30:34.923257 29581 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:34.993268 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:35.080365 26257 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:34.889202 29794 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260924 14:30:34.897704 29647 consensus_queue.cc:1046] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260924 14:30:34.920570 29540 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:35.045502 26253 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:34.963871 29809 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:35.024991 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:35.044647 29807 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:34.970129 29586 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:34.971666 29419 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260924 14:30:35.061849 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 76009.8us
I20260924 14:30:35.068601 29629 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:34.920888 29810 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:35.065058 26256 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:35.034276 26249 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:34.965230 29442 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260924 14:30:34.900765 29627 consensus_queue.cc:1046] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260924 14:30:35.127152 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:35.095307 29804 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:35.043413 29716 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:35.083724 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:35.135286 26256 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:35.135746 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 212852us
I20260924 14:30:35.091272 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:35.027428 29609 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:35.121150 29586 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:35.098249 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:34.966208 28867 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260924 14:30:34.903903 29718 consensus_queue.cc:1046] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.123s
W20260924 14:30:35.121438 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 198590us
I20260924 14:30:34.904075 29466 consensus_queue.cc:1046] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.131s
I20260924 14:30:35.149807 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:35.146130 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:35.151072 26323 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260924 14:30:35.152099 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:35.152355 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:35.127486 26250 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:35.153376 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:35.135677 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52051.5us
I20260924 14:30:35.153635 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:35.136250 26249 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:35.136972 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45826.5us
I20260924 14:30:35.155028 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:34.964795 29770 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:35.159747 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:35.126725 29810 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:35.160710 26324 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:35.162407 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:35.138916 29716 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:35.121021 29419 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:35.084022 26255 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:30:35.143362 26256 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:35.138170 29609 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:35.113997 29809 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:35.147886 29606 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:35.148440 29822 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:35.152232 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:35.132840 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:35.129788 29581 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:35.180124 29801 raft_consensus.cc:3032] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:35.180816 29801 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:35.180184 29581 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.076119 29185 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:35.138170 29560 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:35.166126 26369 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:35.183077 29834 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260924 14:30:35.185719 29834 raft_consensus.cc:3032] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:35.186106 29834 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:35.186197 29185 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:35.186966 26326 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:35.160173 29605 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:35.166060 29821 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:35.159020 29770 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:35.170370 29419 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:35.193602 29819 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260924 14:30:35.100909 29540 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:35.174446 29647 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:35.163090 29810 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:35.187536 29571 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260924 14:30:35.192801 29812 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260924 14:30:35.183261 29581 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:35.164260 29559 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260924 14:30:35.203938 29817 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260924 14:30:35.204543 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:35.206542 26402 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:30:35.182005 29801 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:35.167778 29673 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:35.213186 27190 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:35.219691 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41190 (request call id 12) took 1067 ms. Trace:
I20260924 14:30:35.224069 29633 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:35.224432 29700 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.164605 29765 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260924 14:30:35.227102 29288 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:35.255897 29288 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:35.257584 29700 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:35.257285 29288 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:35.177726 29609 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:35.131872 29807 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.175941 29829 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:35.178846 29822 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:35.188814 29466 raft_consensus.cc:3032] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:35.194787 29629 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:35.208670 29560 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:35.240643 29571 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:35.243014 29754 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:35.252588 29823 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260924 14:30:35.244190 29633 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:35.228503 29770 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:35.229008 29826 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:35.242951 29559 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:35.164952 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:35.368549 29559 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:35.328274 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:35.328298 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:35.330422 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:35.169212 26257 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:35.193800 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 631) took 1060 ms. Trace:
I20260924 14:30:35.372875 29540 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:35.370998 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 205970us
I20260924 14:30:35.373134 26328 rpcz_store.cc:276] 0924 14:30:34.133879 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.134023 (+   144us) service_pool.cc:227] Handling call
0924 14:30:35.193613 (+1059590us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54002765d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:35.193728 (+   115us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1449864576}
W20260924 14:30:35.187937 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 55250.6us
I20260924 14:30:35.377830 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:35.195726 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:30:35.382572 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 186741us
I20260924 14:30:35.269179 29629 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:35.230690 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:35.386437 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1196) took 2481 ms. Trace:
I20260924 14:30:35.389679 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1209) took 1308 ms. Trace:
I20260924 14:30:35.390375 26332 rpcz_store.cc:276] 0924 14:30:34.081196 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.100545 (+ 19349us) service_pool.cc:227] Handling call
0924 14:30:35.389079 (+1288534us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b540001e0d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:35.389356 (+   277us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1212048384}
I20260924 14:30:35.348479 26403 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:35.350184 26405 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:35.391681 26255 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260924 14:30:35.387140 26329 rpcz_store.cc:276] 0924 14:30:32.904724 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:33.344130 (+439406us) service_pool.cc:227] Handling call
0924 14:30:33.344487 (+   357us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:34.362124 (+1017637us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:35.385621 (+1023497us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b540012eb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:35.386094 (+   473us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2233325824}
I20260924 14:30:35.386184 29629 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:35.355650 26400 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:35.242851 27181 rpcz_store.cc:276] 0924 14:30:34.151903 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.152558 (+   655us) service_pool.cc:227] Handling call
0924 14:30:35.219340 (+1066782us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b5400001c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b9348bc3 00007f09be29281a 00007f09b7985d04 00007f09b7985c92 00007f09b7985bc1 00007f09b7985b3c 00007f09b7984df2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:35.219557 (+   217us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":48171264}
I20260924 14:30:35.402132 29573 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:35.349910 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:35.252033 27192 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:35.428572 29438 raft_consensus.cc:3032] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:35.437098 29438 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:35.262710 29288 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:35.274071 29560 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:35.440670 29826 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:35.441131 29560 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.269320 29571 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:35.265110 29466 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:35.374828 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:35.468729 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 94195.3us
I20260924 14:30:35.469051 29466 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.469992 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:35.375982 26249 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:35.373225 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:35.276300 29633 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.276443 29628 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.362730 27185 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260924 14:30:35.266016 29801 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:35.283370 29804 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:35.491616 29581 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260924 14:30:35.287897 29807 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:35.494060 29643 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:35.287643 29809 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:35.282455 29822 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.497300 29822 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:35.194304 29834 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.428504 29673 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:35.504205 29679 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:35.504886 29679 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:35.504585 29673 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:35.506264 29679 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:35.431670 29611 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:35.432624 29551 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:30:35.508260 26255 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:35.508381 29551 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:35.509680 29551 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.434603 26402 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:35.445329 27182 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:35.438856 29438 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.453305 29700 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:35.462917 27194 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:35.463733 29586 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.470034 29627 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260924 14:30:35.283712 29733 consensus_meta.cc:287] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.004s
I20260924 14:30:35.574766 29733 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.471302 26256 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:35.472096 29726 consensus_queue.cc:1046] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:35.472976 29830 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:35.475268 29647 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:30:35.475606 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:35.476172 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:35.477696 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 986) took 1451 ms. Trace:
W20260924 14:30:35.583293 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 108014us
I20260924 14:30:35.583742 26401 rpcz_store.cc:276] 0924 14:30:34.026214 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.027718 (+  1504us) service_pool.cc:227] Handling call
0924 14:30:35.477272 (+1449554us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54003d59d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:35.477483 (+   211us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":832874240}
W20260924 14:30:35.423295 29844 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260924 14:30:35.456076 29419 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:35.477653 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:35.520385 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 143030us
W20260924 14:30:35.534775 26399 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.000s	sys 0.013s
I20260924 14:30:35.537628 29609 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:35.538771 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 69239.1us
I20260924 14:30:35.589188 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:35.522929 29679 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:35.266402 29714 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.594949 29714 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:35.546204 29829 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:35.539958 26402 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:35.547338 29676 raft_consensus.cc:3032] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:35.548024 27192 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:35.553162 29628 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:35.553666 29707 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:30:35.567221 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:35.586781 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 56950us
W20260924 14:30:35.610530 26399 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.001s	sys 0.015s
W20260924 14:30:35.595551 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 118262us
I20260924 14:30:35.539388 29700 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:35.598497 29185 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:30:35.599505 29726 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:35.600382 29796 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:35.601612 29611 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:35.602427 29815 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.453s	user 0.000s	sys 0.002s
W20260924 14:30:35.602572 29758 consensus_meta.cc:287] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.729s	user 0.015s	sys 0.004s
I20260924 14:30:35.576011 29586 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:35.607899 26404 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:35.584996 29733 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:35.618845 29830 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:30:35.619077 26398 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:35.619727 26397 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:35.667897 29830 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:35.620041 29765 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260924 14:30:35.622018 27125 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:35.669324 29830 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.669986 29609 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:35.670205 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48324.2us
I20260924 14:30:35.671469 27125 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:35.672463 29599 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:35.672878 29599 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:35.624418 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 133853us
I20260924 14:30:35.673182 29457 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:35.673616 29501 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:35.589783 29673 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:35.625355 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 978) took 1097 ms. Trace:
I20260924 14:30:35.638995 27192 raft_consensus.cc:595] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:35.678210 27192 raft_consensus.cc:603] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:35.633783 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:35.609234 29676 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:35.631966 29726 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:35.622942 29647 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:30:35.625335 29606 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:35.673251 29599 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:35.676152 29501 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:35.679636 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:35.680158 29501 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:35.677830 27189 rpcz_store.cc:276] 0924 14:30:34.528123 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.528262 (+   139us) service_pool.cc:227] Handling call
0924 14:30:34.528591 (+   329us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:34.859197 (+330606us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:35.624898 (+765701us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b5400155c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:35.625155 (+   257us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":698009472}
I20260924 14:30:35.667852 29826 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:35.683589 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:35.696076 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:35.695487 26255 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:35.702616 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:35.659643 29559 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:35.704730 29700 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.719475 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:35.719937 26401 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:35.720454 27181 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:35.720840 29850 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:35.721673 29794 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:35.721503 29850 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.722167 29794 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:35.724953 29677 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:35.725665 29677 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.727803 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:35.729193 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 55047.5us
I20260924 14:30:35.730130 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 566) took 1564 ms. Trace:
I20260924 14:30:35.730280 29695 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:35.730574 27191 rpcz_store.cc:276] 0924 14:30:34.165499 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.165835 (+   336us) service_pool.cc:227] Handling call
0924 14:30:34.187165 (+ 21330us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:34.964070 (+776905us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:35.729560 (+765490us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54002c51d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:35.729918 (+   358us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1784514048}
I20260924 14:30:35.731654 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:35.731778 29676 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:35.733035 29606 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:30:35.713891 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:35.678042 29796 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.742669 29677 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:35.744045 27123 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.600s (SENT) [suppressed 23 similar messages]
I20260924 14:30:35.689392 29611 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.738560 29606 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:35.791889 27191 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:35.794497 27191 raft_consensus.cc:595] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:35.795083 27191 raft_consensus.cc:603] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:35.801740 26403 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:35.801944 29826 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:35.821422 27186 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:35.825047 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1330) took 1209 ms. Trace:
I20260924 14:30:35.830637 26404 rpcz_store.cc:276] 0924 14:30:34.615303 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.615512 (+   209us) service_pool.cc:227] Handling call
0924 14:30:35.824770 (+1209258us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b540023f854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:35.824981 (+   211us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1815345920}
I20260924 14:30:35.826133 29551 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:35.827096 27126 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:35.833467 29788 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:35.834048 29788 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:35.834558 29788 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:35.838148 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 94299.4us
I20260924 14:30:35.812790 29419 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:35.813809 29850 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:35.836477 29676 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:35.723353 29794 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:35.852385 29593 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260924 14:30:35.852511 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 67364us
I20260924 14:30:35.852090 29606 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.852725 29695 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:35.856130 29606 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:35.855950 29695 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:35.860195 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 88164.6us
I20260924 14:30:35.862557 29765 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:35.863253 29765 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:35.829447 29185 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260924 14:30:35.843142 29457 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:35.868408 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:35.872939 27194 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:30:35.874680 29849 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:35.875594 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 235086us
I20260924 14:30:35.877169 27192 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:35.879230 27192 raft_consensus.cc:595] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:35.879652 27192 raft_consensus.cc:603] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:35.882757 29457 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.883674 29185 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:35.884495 29185 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:35.885512 29457 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:35.814782 29726 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:35.877902 29765 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:35.872074 29794 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:35.877424 29840 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
W20260924 14:30:35.887820 29532 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.937s	user 0.010s	sys 0.014s
I20260924 14:30:35.927389 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:35.875583 29850 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:35.887936 29810 consensus_meta.cc:287] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.007s
I20260924 14:30:35.896843 29832 raft_consensus.cc:3032] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:35.943915 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:35.921479 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:30:35.888724 29726 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:35.897264 29560 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:35.890169 29762 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:35.887338 29185 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:35.875077 29609 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260924 14:30:35.959175 26255 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:35.955724 29832 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:35.966641 29762 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:35.985432 26369 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:35.982183 29419 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:35.988139 29288 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.004s
I20260924 14:30:35.987998 29560 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.988423 29419 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:35.988665 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45023.5us
I20260924 14:30:35.989809 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:35.990624 29560 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:35.990746 29419 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:35.988036 29787 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:35.992096 29787 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:35.992554 29609 consensus_queue.cc:1225] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260924 14:30:35.992915 29571 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.508s	user 0.015s	sys 0.000s
W20260924 14:30:35.993036 29716 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.681s	user 0.002s	sys 0.013s
I20260924 14:30:35.993394 29571 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:35.994194 29787 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:35.994298 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:35.995127 26402 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:35.995998 29796 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:35.996757 29587 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:35.997706 29587 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:36.004032 29695 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:35.993417 29599 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:36.011334 27194 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:36.011833 29714 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:30:35.959239 29532 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.011730 29762 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.012581 29714 raft_consensus.cc:686] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:36.013384 29762 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:36.013875 26251 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:36.013684 29714 consensus_queue.cc:237] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:36.016578 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:36.017248 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1308) took 1047 ms. Trace:
I20260924 14:30:35.944752 29849 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.017596 26254 rpcz_store.cc:276] 0924 14:30:34.969891 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:35.060608 (+ 90717us) service_pool.cc:227] Handling call
0924 14:30:35.084898 (+ 24290us) raft_consensus.cc:1457] Updating replica for 3 ops
0924 14:30:35.328255 (+243357us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:35.328345 (+    90us) raft_consensus.cc:1525] Triggering prepare for 3 ops
0924 14:30:35.329934 (+  1589us) log.cc:842] Serialized 78 byte log entry
0924 14:30:35.483078 (+153144us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:35.488667 (+  5589us) log.cc:842] Serialized 14 byte log entry
0924 14:30:35.500268 (+ 11601us) log.cc:842] Serialized 14 byte log entry
0924 14:30:35.539280 (+ 39012us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:35.545930 (+  6650us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:36.016857 (+470927us) raft_consensus.cc:1671] finished
0924 14:30:36.016878 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:36.017073 (+   195us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":83529,"prepare.queue_time_us":22607,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":104405,"raft.run_cpu_time_us":753,"raft.run_wall_time_us":65194,"spinlock_wait_cycles":174592,"thread_start_us":347576,"threads_started":2,"wal-append.queue_time_us":259375}
I20260924 14:30:36.019351 29560 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:36.020315 29849 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:36.020982 29560 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:30:36.015197 29832 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.022168 29796 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:36.022608 29796 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:35.993500 29716 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:35.986533 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:35.965749 29559 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:36.032200 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 133163us
I20260924 14:30:36.035400 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.032076 27181 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:36.038508 26404 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:36.039209 27185 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:36.040107 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.040666 29716 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.041605 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:36.042434 26402 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:36.043725 26406 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:36.011651 29682 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.047782 29599 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:36.047969 29682 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.050808 29682 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.023871 29532 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:36.052101 29832 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:36.052884 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:36.054162 29822 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.007s
I20260924 14:30:36.054385 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:36.055508 29824 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:36.056133 29824 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:36.056500 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.057189 26326 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:36.057096 29824 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.056617 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:35.971853 29850 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:35.971678 29810 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:36.058589 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 72152.3us
I20260924 14:30:36.064055 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.064365 29845 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:36.065369 26325 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:36.065973 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.066726 29466 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:36.068987 29770 consensus_meta.cc:287] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.798s	user 0.012s	sys 0.003s
I20260924 14:30:36.079006 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:36.077436 26406 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:36.061050 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.085028 26254 raft_consensus.cc:2379] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 3: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:36.071904 26297 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:36.107741 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:36.119196 26404 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:36.117323 29845 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:36.126904 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 95169.6us
I20260924 14:30:36.127053 29845 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:36.128868 29850 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.130609 29810 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.131750 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.122076 29788 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.133662 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:36.134759 26253 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:36.136467 26329 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260924 14:30:36.137704 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.104880 29770 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.140477 27188 raft_consensus.cc:2354] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:30:36.161504 27181 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:36.140656 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.140067 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.186444 26250 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:36.189783 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.193461 26405 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:36.185854 29716 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:36.194278 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.194151 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:36.195178 29615 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:30:36.195732 26257 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:36.195909 29790 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:36.196687 29790 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:36.196910 29682 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.194144 29716 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.197513 29682 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.197912 29790 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.198796 29716 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.089421 29796 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.207986 26297 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:36.208956 29633 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:36.209493 29707 consensus_meta.cc:287] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.002s
I20260924 14:30:36.210150 29778 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.210994 29765 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:30:36.211611 29765 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:36.211935 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:36.210642 29778 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.195200 26254 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:36.192189 29788 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:36.186357 29850 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.227435 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:36.228242 26329 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260924 14:30:36.212595 29765 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.221712 29682 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:36.219777 26251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1304) took 1238 ms (client timeout 1640 ms). Trace:
I20260924 14:30:36.243649 26253 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 8
I20260924 14:30:36.235708 29795 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:36.219621 29593 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:36.238883 29763 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:36.180688 29501 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:36.240587 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1342) took 1051 ms. Trace:
W20260924 14:30:36.256536 26251 rpcz_store.cc:269] 0924 14:30:34.968604 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.976225 (+  7621us) service_pool.cc:227] Handling call
0924 14:30:36.207166 (+1230941us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400782dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:36.207381 (+   215us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2382465920}
I20260924 14:30:36.245148 29716 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.267710 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:36.269430 26406 rpcz_store.cc:276] 0924 14:30:35.189375 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:35.189533 (+   158us) service_pool.cc:227] Handling call
0924 14:30:36.240360 (+1050827us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b5400217854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:36.240521 (+   161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1847266560}
I20260924 14:30:36.275723 26250 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:36.277490 27125 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:36.262498 29593 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:36.285058 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:36.290231 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1341) took 1096 ms. Trace:
I20260924 14:30:36.285916 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:36.290766 26397 rpcz_store.cc:276] 0924 14:30:35.189023 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:35.189167 (+   144us) service_pool.cc:227] Handling call
0924 14:30:35.189465 (+   298us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:35.668128 (+478663us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:36.284087 (+615959us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54002fc6d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:36.285581 (+  1494us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":986558848}
I20260924 14:30:36.290346 29617 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:36.299849 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:36.256183 29850 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.300626 29611 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:36.300709 26397 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:36.300848 29770 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.301996 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:36.302585 26397 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 8
I20260924 14:30:36.287765 29593 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:36.304204 27126 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:36.299904 29617 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:36.306059 29617 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:36.302515 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 230792us
I20260924 14:30:36.296846 26254 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:36.310251 26406 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:36.298278 26255 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:30:36.324424 26404 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:36.331681 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.332661 27181 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 3.
I20260924 14:30:36.309195 29668 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:36.281518 29716 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.329504 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.302260 29850 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:36.344720 29466 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:36.348884 29466 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.349470 29716 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:36.351069 29466 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:36.351404 26329 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:36.339126 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:36.345139 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.364295 29834 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:36.364823 29501 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:36.324834 29830 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260924 14:30:36.368221 29830 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:36.368470 26332 raft_consensus.cc:1261] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:30:36.368655 29830 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:36.369267 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:36.338887 29668 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:36.369956 29668 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:36.369810 29830 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.371053 29830 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:36.369951 26404 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:36.302155 29801 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.328161 29532 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:36.368265 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.408828 26251 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:36.331131 29788 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.329551 29794 consensus_queue.cc:1046] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260924 14:30:36.418833 26325 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:36.419950 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:36.441694 26404 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:36.447830 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.449458 29716 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:36.451992 27189 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260924 14:30:36.457894 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:36.461795 29716 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.462450 29822 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:36.463892 29822 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:36.464956 29822 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:36.467778 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:36.468667 29763 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:36.469256 29763 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:36.452565 27189 pending_rounds.cc:76] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 6
I20260924 14:30:36.471032 27189 pending_rounds.cc:97] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260924 14:30:36.471343 27189 raft_consensus.cc:2866] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:36.470448 29763 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.475000 26332 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:36.479123 29842 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.787s	user 0.000s	sys 0.004s
I20260924 14:30:36.481734 26324 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:36.483400 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:36.484864 27188 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 607) took 1601 ms (client timeout 1967 ms). Trace:
I20260924 14:30:36.483723 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.517853 26330 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:36.518893 26326 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:36.519788 26325 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:36.520164 29716 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:36.521816 26332 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:36.526266 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 836) took 1353 ms. Trace:
I20260924 14:30:36.526767 26327 rpcz_store.cc:276] 0924 14:30:35.173866 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:35.175147 (+  1281us) service_pool.cc:227] Handling call
0924 14:30:35.175705 (+   558us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:35.176120 (+   415us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:35.176149 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:35.176655 (+   506us) log.cc:842] Serialized 26 byte log entry
0924 14:30:35.564046 (+387391us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:35.564186 (+   140us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:35.568547 (+  4361us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:36.478451 (+909904us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:36.525833 (+ 47382us) raft_consensus.cc:1671] finished
0924 14:30:36.525851 (+    18us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:36.526070 (+   219us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3964672,"thread_start_us":309998,"threads_started":1,"wal-append.queue_time_us":401782}
I20260924 14:30:36.522029 29532 raft_consensus.cc:3032] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:36.530078 29532 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:36.511135 27188 rpcz_store.cc:269] 0924 14:30:34.883757 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.971843 (+ 88086us) service_pool.cc:227] Handling call
0924 14:30:36.484519 (+1512676us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400776854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:36.484774 (+   255us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2598792960}
I20260924 14:30:36.423425 29679 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:36.422772 29185 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:36.497613 29726 consensus_meta.cc:287] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.526s	user 0.003s	sys 0.009s
I20260924 14:30:36.570438 26253 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:36.563194 26397 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
W20260924 14:30:36.578734 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 170414us
I20260924 14:30:36.579850 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:36.581534 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:36.410012 29801 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.571086 29716 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.586833 29814 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:36.562476 29794 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:36.592000 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.460083 29788 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:36.593413 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 347789us
I20260924 14:30:36.531124 29532 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.585127 29832 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:36.575215 29679 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:36.613602 26224 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:36.615792 26329 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260924 14:30:36.617154 26323 raft_consensus.cc:2362] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7. Re-sending same reply.
I20260924 14:30:36.576771 29185 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.644721 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:36.648308 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:36.650895 26404 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:36.644348 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.607192 29814 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:36.641656 29633 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260924 14:30:36.660640 29633 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:36.660723 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.661052 29679 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:36.661902 29679 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:36.662222 29801 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:36.607663 29794 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.664165 29560 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.664656 29560 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.664997 29794 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:36.665742 29794 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:36.666353 29794 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:36.666672 29560 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.667179 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.667853 29830 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.668226 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.668377 29830 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:36.669369 29185 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:36.670034 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.659797 29340 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.670463 29790 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.670754 29340 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:36.671105 29790 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.662449 29679 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:36.672492 29862 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.634s	user 0.002s	sys 0.007s
I20260924 14:30:36.673228 29790 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:36.673365 29679 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:36.674460 29668 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.616685 29716 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.676683 29668 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:36.677309 29716 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:36.677423 29668 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.678035 29668 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.678814 26256 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:36.612130 29832 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:36.656836 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.661131 29633 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:36.682217 26327 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:36.683740 29340 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:36.686975 29814 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:36.689070 29843 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.878s	user 0.000s	sys 0.003s
I20260924 14:30:36.690667 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1339) took 1880 ms. Trace:
I20260924 14:30:36.692670 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.693456 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.694113 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.696105 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:36.696422 29869 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260924 14:30:36.643337 29788 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.697470 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.698346 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.708254 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:36.699412 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.700963 29571 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.702898 29827 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:36.691736 29668 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:36.705103 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:36.705440 26398 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
W20260924 14:30:36.706064 27186 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 602) took 1824 ms (client timeout 1944 ms). Trace:
I20260924 14:30:36.707278 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1345) took 1259 ms. Trace:
I20260924 14:30:36.694201 26332 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:36.715150 26298 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:36.691051 26399 rpcz_store.cc:276] 0924 14:30:34.809985 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.810565 (+   580us) service_pool.cc:227] Handling call
0924 14:30:34.847947 (+ 37382us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:34.848188 (+   241us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:34.848211 (+    23us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:34.848644 (+   433us) log.cc:842] Serialized 26 byte log entry
0924 14:30:35.684847 (+836203us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:35.699645 (+ 14798us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:35.709450 (+  9805us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:36.688321 (+978871us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:36.690317 (+  1996us) raft_consensus.cc:1671] finished
0924 14:30:36.690334 (+    17us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:36.690603 (+   269us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6770560,"thread_start_us":777365,"threads_started":1,"wal-append.queue_time_us":825924}
I20260924 14:30:36.670439 29830 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:36.723423 27186 rpcz_store.cc:269] 0924 14:30:34.882152 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:34.962606 (+ 80454us) service_pool.cc:227] Handling call
0924 14:30:36.705876 (+1743270us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54003b79d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:36.706013 (+   137us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1450949504}
I20260924 14:30:36.681488 29794 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:36.726863 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:36.728194 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.728909 29633 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.723623 26403 rpcz_store.cc:276] 0924 14:30:35.448110 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:35.472982 (+ 24872us) service_pool.cc:227] Handling call
0924 14:30:35.473427 (+   445us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:35.805932 (+332505us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:36.706974 (+901042us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b5400475c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:36.707195 (+   221us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":720594176}
I20260924 14:30:36.733640 29790 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:36.734707 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1344) took 1342 ms. Trace:
I20260924 14:30:36.737144 29764 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:36.740484 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.768442 26327 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:36.741576 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.732985 29532 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:36.686100 29832 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:36.723794 29827 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:36.759030 29824 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.759714 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.760637 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:36.761042 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.778010 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.696846 29593 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.766244 26405 rpcz_store.cc:276] 0924 14:30:35.392726 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:35.393645 (+   919us) service_pool.cc:227] Handling call
0924 14:30:35.394227 (+   582us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:36.193963 (+799736us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:36.734463 (+540500us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b540036bad4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:36.734643 (+   180us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1837399424}
I20260924 14:30:36.788290 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.770380 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:36.774799 29765 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.775117 29633 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:36.759776 29794 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:36.777573 26323 raft_consensus.cc:2379] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:36.764424 27186 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260924 14:30:36.744393 29560 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:36.780197 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:36.789529 26402 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:36.810282 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:36.795720 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.811167 26331 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:36.811496 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:36.802441 29185 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:36.803120 29466 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:36.812299 29788 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:36.804749 29593 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.813095 29834 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:36.812690 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:36.815574 29834 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:36.805502 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.807209 29587 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:36.807996 29801 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:30:36.808434 26405 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260924 14:30:36.816910 26369 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:36.765877 29790 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:36.818145 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:36.834052 26332 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 6: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:36.826442 27190 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:36.829334 29765 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:36.838060 29765 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:36.834697 29790 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:36.840613 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:36.841334 27188 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:30:36.844363 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:36.850571 29832 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.853258 26251 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:36.855078 29700 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:36.851099 29778 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:36.868388 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 288621us
I20260924 14:30:36.869819 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:36.874117 26327 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:36.878794 29830 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260924 14:30:36.900103 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 55825.1us
I20260924 14:30:36.794129 29824 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:36.917908 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 82640.6us
I20260924 14:30:36.923308 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.922125 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:36.925900 26405 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:30:36.931167 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:36.934602 26251 raft_consensus.cc:2396] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260924 14:30:36.935036 26255 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:36.891434 29794 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:36.942757 26254 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:36.923694 29661 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:30:36.919759 29822 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:36.959775 26253 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260924 14:30:36.945729 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:36.922683 29288 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:36.948531 26258 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:36.971196 26256 raft_consensus.cc:2379] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:36.974749 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 318996us
I20260924 14:30:36.977751 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:36.978984 26255 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:36.944130 29778 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.950724 29886 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.922135 29615 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260924 14:30:36.953240 29540 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:36.936124 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 322587us
I20260924 14:30:36.945494 29661 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:36.955137 29438 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:36.967897 29845 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:36.948601 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:36.927924 29832 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:36.982520 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:37.009745 26251 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:36.984841 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:36.985708 26250 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:36.987833 26405 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:36.976432 29679 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.997596 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.017335 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:37.018625 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.013271 26255 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:37.022553 26254 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:36.988516 29765 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.006654 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:37.028383 26403 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:36.982438 29560 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:37.011075 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:37.030593 26401 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:37.029506 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.031579 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:36.974805 29288 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:37.033787 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.035181 26256 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:37.036577 26405 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:37.022322 29571 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:36.998601 29661 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:30:37.000234 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 189420us
I20260924 14:30:37.044736 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:36.988036 29778 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.041599 26251 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:36.985891 29822 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.051998 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.023653 29888 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:37.022600 29679 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:36.999145 29790 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.035548 29700 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:37.037591 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.035640 29850 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:30:37.035703 29788 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:37.027199 29466 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260924 14:30:36.994912 29886 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.010363 29540 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:37.046814 29830 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:30:37.093677 29794 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:37.131699 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.129563 29824 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:37.132879 29876 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.285s	user 0.003s	sys 0.001s
I20260924 14:30:37.133160 29890 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260924 14:30:37.099711 29849 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:37.142251 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 56955.6us
I20260924 14:30:37.143843 29830 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:37.143952 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:37.144460 29830 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.145056 29679 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:37.145165 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.145996 26324 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:37.146462 29830 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:37.146857 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:37.148353 29540 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:37.127200 29764 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.149503 29796 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:37.149439 29764 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.116928 29629 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:37.151702 29629 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:37.152194 29629 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:37.067938 29571 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.155786 29571 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:37.155946 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.156827 26330 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:37.157694 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.158723 27182 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:37.158982 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.161240 26295 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:37.162405 29466 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:37.163050 29890 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:37.165771 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.100768 29778 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:37.104936 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.084547 29888 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.034041 29765 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.127081 29788 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:37.109692 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.111159 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:37.111734 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 80227.1us
I20260924 14:30:37.111882 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:37.113912 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.119975 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:37.122234 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.186436 26327 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:37.186509 26331 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:37.187438 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:37.188081 26331 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:37.123178 26403 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:37.114770 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260924 14:30:37.189078 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 183306us
I20260924 14:30:37.128136 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.135141 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.094969 29818 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
W20260924 14:30:37.193889 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 82091.6us
I20260924 14:30:37.135799 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.136956 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:37.194790 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:37.193495 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 82413.9us
I20260924 14:30:37.138746 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:37.147408 26324 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:37.140775 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:37.196517 26323 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:37.056301 29814 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260924 14:30:37.107846 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.197590 26255 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:37.198829 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:37.126680 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.193423 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.127115 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:37.138566 26325 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:37.146853 29288 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:37.195109 29830 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:37.156843 29849 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.203540 26404 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:37.233307 26221 leader_election.cc:398] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:30:37.248528 26221 leader_election.cc:401] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:30:37.230659 29679 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260924 14:30:37.250309 29820 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:37.250733 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:37.235031 29726 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.251854 29790 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:37.252048 29726 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:37.254346 29726 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:37.235260 29902 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:37.255541 29902 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.256772 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:37.257424 26405 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:37.257790 29902 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:37.235668 29764 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:37.251374 26404 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 9
I20260924 14:30:37.195767 29466 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.193809 29661 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.205854 29830 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:37.227977 29778 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:37.238842 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.243149 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.229766 29886 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:37.244424 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.274685 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.199762 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:37.203051 26332 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
W20260924 14:30:37.233697 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 109450us
I20260924 14:30:37.278304 29763 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:37.278726 29763 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:37.279393 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 140783us
I20260924 14:30:37.279665 29763 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.282299 29571 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:37.282400 29663 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:37.283062 29663 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:37.282819 29571 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:37.282724 29827 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.284092 29827 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:37.284845 29571 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:37.285974 26255 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:37.287765 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.288023 29593 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:37.288846 26253 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:37.290568 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.291365 26258 raft_consensus.cc:2396] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:37.293630 29679 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8.
I20260924 14:30:37.293964 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.294520 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:37.292986 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 102954us
W20260924 14:30:37.287886 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 89153.7us
I20260924 14:30:37.296917 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:37.297744 26331 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:37.275352 27192 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:30:37.275620 26399 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:37.294638 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.294701 26249 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:37.292795 29814 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
W20260924 14:30:37.321763 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45717.4us
I20260924 14:30:37.316242 26327 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:37.420500 26398 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:37.274518 27189 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:37.233040 29700 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:30:37.250730 29820 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:37.428162 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 109752us
I20260924 14:30:37.429502 26330 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:37.229871 29765 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:37.232681 29822 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:37.231101 29888 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:37.277046 29830 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:30:37.434996 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 112319us
W20260924 14:30:37.436859 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 141211us
I20260924 14:30:37.438683 26250 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:37.439379 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:37.447734 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 150896us
I20260924 14:30:37.448372 26404 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:37.451184 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.420536 29814 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:37.426493 29778 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:37.429468 29679 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:37.273785 29466 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.481299 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:37.481325 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:37.482479 26323 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:37.461788 26324 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:37.429190 29886 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.483479 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:37.506219 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 57631.8us
I20260924 14:30:37.507213 27126 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:37.484896 26224 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:37.508749 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.509677 26403 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:37.447835 29700 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:37.455753 29676 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:37.517110 29801 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:30:37.518533 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 64998.5us
I20260924 14:30:37.457708 29888 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:37.505280 29894 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260924 14:30:37.480947 29903 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:37.536670 26224 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:37.484642 29615 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:37.550199 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:37.536723 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 303594us
I20260924 14:30:37.551290 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.515858 29902 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:37.556730 26405 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:37.552654 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:37.524637 29540 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
W20260924 14:30:37.534246 29842 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260924 14:30:37.531682 29794 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.561214 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:37.484359 29822 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260924 14:30:37.564342 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.452129 29788 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.560565 29593 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.578516 29629 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.545380 29676 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.558192 29532 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:37.626776 29532 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:37.627250 29532 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:30:37.634035 29906 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260924 14:30:37.559139 29824 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:37.513686 29886 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.576897 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:37.556496 29185 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260924 14:30:37.524032 29758 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260924 14:30:37.555537 29663 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.644474 29811 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.561172 29902 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:37.644948 29621 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.645220 29902 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:37.568392 29764 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:37.526513 29679 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.557878 29905 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:37.534801 29820 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.583511 29778 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.587604 29700 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:37.587903 29661 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:37.564159 29890 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.576619 29540 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:37.654598 29905 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:37.655369 29540 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.655988 29905 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.589476 27190 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:37.579282 29593 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.590463 29849 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:37.555655 29888 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.661688 29888 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:37.662308 29888 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:37.662781 29888 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:37.580158 29794 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.664866 29794 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:37.665447 29794 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:37.665566 29830 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:37.665898 29794 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:37.666568 29679 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:37.594712 29587 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:37.600311 29900 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260924 14:30:37.667712 29587 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.602703 29801 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:37.668571 29801 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:37.668943 29801 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:37.669677 29587 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:37.670372 29890 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.606292 29676 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:37.607715 29629 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:37.578536 29466 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:37.611554 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 70898.7us
I20260924 14:30:37.672384 29890 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:37.620679 29763 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:37.623013 29899 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.429s	user 0.003s	sys 0.000s
I20260924 14:30:37.635411 26328 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:37.641247 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:37.644570 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 76916.9us
I20260924 14:30:37.485199 29904 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260924 14:30:37.589061 29790 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:37.572758 29288 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.648823 26401 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:37.645537 29621 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.649912 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:37.650595 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 98020us
I20260924 14:30:37.682508 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:37.645179 29811 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:37.652293 29682 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:37.679577 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:37.707800 26221 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:37.718484 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1351) took 1116 ms. Trace:
W20260924 14:30:37.728801 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 49360.3us
I20260924 14:30:37.719009 26397 rpcz_store.cc:276] 0924 14:30:36.601836 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:36.602039 (+   203us) service_pool.cc:227] Handling call
0924 14:30:36.704613 (+102574us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:36.705007 (+   394us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:36.705035 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:36.705502 (+   467us) log.cc:842] Serialized 26 byte log entry
0924 14:30:37.050563 (+345061us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:37.050598 (+    35us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:37.050666 (+    68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:37.578591 (+527925us) raft_consensus.cc:1671] finished
0924 14:30:37.578609 (+    18us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:37.706894 (+128285us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54002fbcd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:37.718393 (+ 11499us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":4226,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6683,"spinlock_wait_cycles":230944640,"thread_start_us":344426,"threads_started":1,"wal-append.queue_time_us":457060}
I20260924 14:30:37.689270 29790 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:37.730949 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:37.731763 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.736390 26401 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:37.724285 29925 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:37.618947 29814 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.692490 29629 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:37.737826 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.738322 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.706248 29288 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.743101 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.752068 26255 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:30:37.743218 29822 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:37.743779 29818 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:37.744271 26249 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260924 14:30:37.732175 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 123177us
I20260924 14:30:37.738835 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:37.755420 27126 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:37.756278 26250 raft_consensus.cc:2354] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11.
W20260924 14:30:37.753897 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 389541us
I20260924 14:30:37.757735 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.746836 29593 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:37.743393 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.711279 29682 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:37.728848 29679 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:37.751189 26254 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:37.760550 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.776389 26258 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:37.764386 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.777282 26255 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:37.728144 29915 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:37.739579 29790 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:30:37.772415 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 289056us
I20260924 14:30:37.778755 27125 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:37.782667 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.783912 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.790051 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:37.785291 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.762775 26298 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:37.765312 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.785298 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1363) took 1042 ms. Trace:
I20260924 14:30:37.797932 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.785522 26253 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:37.796684 26406 rpcz_store.cc:276] 0924 14:30:36.742886 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:36.763371 (+ 20485us) service_pool.cc:227] Handling call
0924 14:30:36.788192 (+ 24821us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:36.788435 (+   243us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:36.788463 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:36.816281 (+ 27818us) log.cc:842] Serialized 26 byte log entry
0924 14:30:37.034550 (+218269us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:37.034607 (+    57us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:37.034673 (+    66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:37.622635 (+587962us) spinlock_profiling.cc:243] Waited 65.6 ms on lock 0x7b240032fe88. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b627ecf4 00007f09b627cdfb 00007f09b630b559 00007f09b630b1e5 00007f09b630a82d 00007f09b8fb946e 00007f09b8fd0c0c 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551
0924 14:30:37.775673 (+153038us) raft_consensus.cc:1671] finished
0924 14:30:37.775695 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:37.785228 (+  9533us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":150818688,"thread_start_us":194075,"threads_started":1,"wal-append.queue_time_us":364520}
I20260924 14:30:37.753669 29822 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:37.729193 29902 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:37.774747 29923 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260924 14:30:37.812326 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.804482 26249 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 10: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:37.763684 29834 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.798663 27186 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:37.810118 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.816958 27192 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260924 14:30:37.801153 26369 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:37.826005 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:37.774457 29682 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:37.832043 29788 raft_consensus.cc:2741] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260924 14:30:37.832887 29788 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:37.833308 29788 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:37.833688 29788 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:37.749897 29457 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260924 14:30:37.861235 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:37.865036 26258 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:30:37.659020 29532 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.799808 29814 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:37.867682 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 57147.6us
I20260924 14:30:37.872668 26401 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:37.785073 29551 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:30:37.867704 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:37.786185 29794 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:37.886147 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:37.731529 29888 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:37.809509 29849 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election won for term 7
W20260924 14:30:37.868526 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 307360us
I20260924 14:30:37.901362 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.887850 26397 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:37.873088 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:37.890084 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.890551 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.811622 29822 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:37.903622 26323 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:37.892288 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.808588 29762 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:37.905195 26371 leader_election.cc:398] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11.
I20260924 14:30:37.905560 26371 leader_election.cc:401] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260924 14:30:37.858199 29890 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:37.897955 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.898237 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.932765 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.907272 27190 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:37.932385 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.906618 26250 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:37.934963 26332 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:37.963021 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.962417 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.786741 29663 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260924 14:30:37.962466 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.933259 29890 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:37.934041 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.969122 26330 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:37.874246 29185 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.969764 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:37.964012 26329 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:37.872689 29921 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:37.973660 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.969570 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.980021 26398 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:37.984128 29185 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:37.904949 29621 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:37.986356 29185 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:37.986644 29288 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:37.899768 29778 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:37.798967 29915 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:37.987517 29778 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11.
I20260924 14:30:37.894666 29551 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:37.987856 29902 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:37.988214 29551 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:37.988788 29911 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260924 14:30:37.989094 29915 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:37.990483 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:37.991153 26324 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:38.000730 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:37.979462 27191 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:37.898859 29904 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:37.933287 29716 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260924 14:30:37.932176 29762 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:38.014518 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:38.014649 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.009763 27182 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:37.905737 29886 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:37.961660 29796 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260924 14:30:37.864176 29834 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:38.042359 29834 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:38.042879 29902 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:38.039743 27190 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:38.045262 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:38.046097 27190 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:38.046846 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:38.048063 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.049468 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:38.049567 29778 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:38.051322 29778 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:38.051990 29778 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:38.053880 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:37.963675 29679 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:38.054524 29818 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.063795 29818 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:38.066028 29818 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:38.055403 27190 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:38.033981 26397 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
W20260924 14:30:38.056171 29919 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260924 14:30:38.057417 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:38.176836 26406 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:38.056198 29891 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.177848 29891 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:38.108309 29921 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:38.180226 29891 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:38.181206 29921 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:38.052471 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.173755 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:37.972486 29890 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:38.182361 27182 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:38.183064 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:38.185191 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 213049us
W20260924 14:30:38.067281 29912 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260924 14:30:38.189116 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 136765us
I20260924 14:30:38.037024 29726 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:38.024741 29716 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:38.216180 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:38.175204 29762 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:38.223135 27126 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:38.063278 29679 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:38.236024 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:38.237104 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:38.203713 29891 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260924 14:30:38.238561 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 296192us
I20260924 14:30:38.246084 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:38.247300 26331 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:38.242163 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:38.040249 29904 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:38.209595 29811 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:38.219060 29726 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:38.223076 29716 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:38.261579 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:38.258484 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:38.260644 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:38.261272 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:38.265146 29668 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:30:38.266173 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:38.267437 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:38.267535 29907 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260924 14:30:38.269286 29914 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:38.269915 29914 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:38.270328 29914 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:38.270709 29801 consensus_meta.cc:287] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.002s
I20260924 14:30:38.239818 29593 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:38.271708 29593 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:38.263263 26331 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:38.272040 29593 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:38.239620 29764 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.275650 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:38.276298 26330 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:38.247434 29466 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.240003 29830 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:38.252234 29903 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:38.296030 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:38.261890 29726 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:38.298065 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45108.8us
W20260924 14:30:38.361934 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 463250us
I20260924 14:30:38.362277 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:38.287122 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:38.325090 29466 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:38.328747 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:38.329063 27190 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:38.330495 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:38.331171 29927 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260924 14:30:38.239452 29288 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260924 14:30:38.215394 29682 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:38.362782 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.271188 29801 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:38.372844 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:38.373761 27191 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:38.378598 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:38.250839 29822 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:38.394800 29822 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:38.395534 27188 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:38.376737 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:38.398669 27190 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:38.399030 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:38.401542 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:38.402694 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 848) took 1293 ms. Trace:
I20260924 14:30:38.403100 26328 rpcz_store.cc:276] 0924 14:30:37.109140 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.109453 (+   313us) service_pool.cc:227] Handling call
0924 14:30:38.402535 (+1293082us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b5400498754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:38.402637 (+   102us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1034753408}
I20260924 14:30:38.404085 29765 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:38.404662 26328 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:38.404788 29765 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:38.405365 29765 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:38.405699 29765 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:38.406512 26328 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:38.408480 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:38.390368 26406 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:38.351634 29903 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:38.327154 29532 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:38.248854 29788 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:38.434717 27189 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:38.411952 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.280257 29920 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.115s
I20260924 14:30:38.386783 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.431527 27190 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:38.419548 26330 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:38.416956 26325 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:38.329790 29764 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:38.443547 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57064.6us
I20260924 14:30:38.410794 29850 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:38.444492 26324 raft_consensus.cc:2354] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11.
I20260924 14:30:38.446960 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.443848 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:38.436074 29763 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.761s	user 0.008s	sys 0.003s
I20260924 14:30:38.420848 29903 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:38.439133 29661 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:38.456619 26402 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:38.467561 29796 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:38.459152 29834 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:38.448951 29700 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:30:38.494971 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 122240us
I20260924 14:30:38.452584 29920 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:38.499208 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:38.452219 29938 raft_consensus.cc:2741] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260924 14:30:38.508817 29764 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:38.510547 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:38.484272 29661 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:38.436936 29716 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:38.516512 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50259us
I20260924 14:30:38.512336 29466 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:38.521296 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:38.493527 29834 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:38.523597 27190 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:38.505227 29820 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:38.529209 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:38.531848 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:38.532876 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:38.527220 26295 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:38.501452 29920 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:38.542115 27123 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.542588 26397 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:30:38.523674 29661 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:38.544271 26399 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:30:38.542624 26398 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:38.545217 26329 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:38.524995 29834 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:38.546079 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:38.548058 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.548771 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:38.534149 29679 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:38.538879 29938 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:38.538450 29904 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:38.546797 29599 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:38.562489 26398 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:38.563822 29438 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260924 14:30:38.573128 29551 consensus_meta.cc:287] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.584s	user 0.009s	sys 0.002s
I20260924 14:30:38.573674 29551 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:38.574550 29765 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:38.575984 27185 raft_consensus.cc:2354] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6.
I20260924 14:30:38.550316 29822 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:38.581859 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.583076 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:38.538924 29593 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:38.587740 26399 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 11
I20260924 14:30:38.589982 29945 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:30:38.590864 29945 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:38.591871 27182 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:30:38.594864 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:38.591120 29551 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:38.562028 29466 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:38.626174 26332 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260924 14:30:38.626829 27194 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:38.624786 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:38.569218 29599 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:38.580528 29682 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:38.642874 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61111.6us
I20260924 14:30:38.661327 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.663292 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:38.664229 26404 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:38.647958 29814 consensus_meta.cc:287] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.732s	user 0.007s	sys 0.004s
I20260924 14:30:38.583657 29801 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:38.592459 29764 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:38.675958 26224 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:38.647737 29466 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:38.659410 29758 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:38.646715 29796 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:38.596910 29288 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260924 14:30:38.660081 29599 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:38.569028 29914 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:38.686923 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:38.689842 26329 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:38.597292 29942 consensus_queue.cc:1046] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260924 14:30:38.670186 29682 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:30:38.670063 29886 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:38.697100 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46283.7us
I20260924 14:30:38.697950 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:38.658485 29827 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260924 14:30:38.667723 29788 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:38.675730 29438 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.683056 29466 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:38.702700 27194 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260924 14:30:38.703418 26331 raft_consensus.cc:2354] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11.
I20260924 14:30:38.577608 29938 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:38.708153 29921 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:38.709187 29679 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.711620 29532 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.712073 29947 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.144s
I20260924 14:30:38.712111 29695 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:38.715358 29914 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:38.717420 29788 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:38.717624 29599 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:38.705013 29438 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:38.718376 29540 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:38.741284 29540 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:38.736205 29921 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:38.736493 29679 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:38.744005 29629 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.990s	user 0.012s	sys 0.000s
I20260924 14:30:38.742487 29540 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:38.744485 29629 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:38.745626 29679 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:38.720345 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:38.723819 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:38.746492 27194 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 11
I20260924 14:30:38.726848 26255 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:38.727578 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:38.748345 26254 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
W20260924 14:30:38.748726 26297 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.640s (SENT)
I20260924 14:30:38.729350 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:38.740897 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:38.754703 26399 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:38.731045 29794 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.731530 29716 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:38.720209 29593 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:38.729647 29796 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:38.757121 29948 log.cc:924] Time spent T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260924 14:30:38.757547 29955 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.772810 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:38.746712 26251 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:38.774389 26397 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:38.775007 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:38.758859 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 57540.7us
I20260924 14:30:38.737630 26404 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:30:38.776912 27126 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:38.695834 29822 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260924 14:30:38.567739 29920 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:38.784312 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:38.785537 26249 raft_consensus.cc:2379] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 4: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:38.763780 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:38.764514 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:38.769565 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:38.743291 29921 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:38.758121 29887 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.684921 29764 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:38.749956 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1133) took 1015 ms. Trace:
I20260924 14:30:38.741080 29599 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:38.791435 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:38.795786 26254 raft_consensus.cc:2379] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:38.798795 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:38.800916 26250 rpcz_store.cc:276] 0924 14:30:37.734927 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.756866 (+ 21939us) service_pool.cc:227] Handling call
0924 14:30:38.749802 (+992936us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400381dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:38.749902 (+   100us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":268194048}
I20260924 14:30:38.738003 29532 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:38.800614 29801 consensus_queue.cc:1159] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e09fbebe29a49d291d23f9d395fe335" responder_term: 8 status { last_received { term: 7 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:30:38.703841 29886 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:38.812004 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.813664 26404 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:38.735415 29661 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:38.777312 29933 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:38.739187 29695 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:38.803634 27194 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:38.824071 26397 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:38.843014 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:38.765625 29794 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:38.838393 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47085us
I20260924 14:30:38.845124 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:38.801777 29288 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:30:38.810674 26329 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:38.849035 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:38.850461 26249 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:38.856078 27126 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:38.851991 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:38.882402 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:38.882402 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:38.883080 26329 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:38.836452 29765 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:38.740588 29945 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:38.817209 29921 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:38.900389 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:38.907817 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1135) took 1143 ms. Trace:
I20260924 14:30:38.908033 26250 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:38.817224 29679 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:38.786219 29955 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:38.898957 27125 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:38.850099 29960 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:38.886322 29676 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:38.886829 29888 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:38.885107 29633 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:38.910028 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:38.898269 29938 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:38.812188 29887 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:38.896512 29942 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:38.889348 29288 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:38.931344 26329 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:38.921555 29957 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:38.983930 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 135439us
I20260924 14:30:38.906032 29914 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:38.947697 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:38.845050 29830 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
W20260924 14:30:38.983434 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 103738us
I20260924 14:30:38.945122 29682 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:38.908408 26253 rpcz_store.cc:276] 0924 14:30:37.755792 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.799081 (+ 43289us) service_pool.cc:227] Handling call
0924 14:30:37.861297 (+ 62216us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:37.861491 (+   194us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:30:37.898151 (+ 36660us) log.cc:842] Serialized 14 byte log entry
0924 14:30:38.591100 (+692949us) spinlock_profiling.cc:243] Waited 9.47 ms on lock 0x7b180036ad80. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09b6261990 00007f09b6423864 00007f09b6423155 00007f09b93463d0 00007f09b9358576 00007f09b93690d7 00007f09b9369058
0924 14:30:38.898455 (+307355us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:38.898522 (+    67us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:38.898591 (+    69us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:38.898683 (+    92us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:38.898836 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":65527,"prepare.queue_time_us":44647,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":725490,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":19320,"spinlock_wait_cycles":21774208,"thread_start_us":307057,"threads_started":1}
I20260924 14:30:38.986378 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:38.994575 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:38.844283 29933 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:39.013151 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:39.014335 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 66724.5us
I20260924 14:30:38.928504 29679 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:39.015101 29942 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:38.984915 29956 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260924 14:30:38.952428 29676 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:39.000003 26250 raft_consensus.cc:2396] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:38.982724 29938 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:38.917358 29921 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:38.981583 29888 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:39.044121 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.019397 29777 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:38.942046 29629 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260924 14:30:38.984869 29925 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:39.044461 29822 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
W20260924 14:30:39.077845 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 94016.3us
I20260924 14:30:39.046829 27125 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:39.010823 29764 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:39.044852 29903 consensus_meta.cc:287] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260924 14:30:39.061708 29532 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:39.019049 29540 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:39.026659 29682 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:39.089699 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:39.051007 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:39.073907 29790 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.219s	user 0.014s	sys 0.000s
I20260924 14:30:39.018741 29974 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:39.069957 29777 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:39.114703 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 64006us
I20260924 14:30:39.135857 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:39.137107 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:39.128468 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:39.009256 29957 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.140640 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:39.140161 26398 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:30:39.069837 29938 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:39.085242 29822 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:39.130940 29682 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:39.079268 29925 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:39.104756 29890 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.897s	user 0.004s	sys 0.008s
I20260924 14:30:39.133443 29974 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:39.133301 29790 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:39.130395 29886 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:39.142825 29764 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:39.141925 29661 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:39.142359 29695 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:39.159610 29794 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:39.105911 29903 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:39.189399 29890 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.133488 29540 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:39.177361 29974 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:39.203747 26327 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: []
I20260924 14:30:39.181237 29830 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:39.189148 29886 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:39.215607 29887 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:39.139595 29288 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:39.228489 27186 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:39.204458 29695 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:39.241340 27181 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:39.213760 29532 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:39.217408 29629 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 3 FOLLOWER]: Advancing to term 4
W20260924 14:30:39.257966 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 120923us
W20260924 14:30:39.226418 29762 consensus_meta.cc:287] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.986s	user 0.009s	sys 0.004s
I20260924 14:30:39.188854 29764 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:39.261857 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:39.164695 29888 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.145105 29942 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.222275 29830 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:39.168821 29676 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:39.222783 29886 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:39.271308 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:39.219674 29967 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:39.171864 29925 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.228025 29971 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:39.260291 29955 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:39.288902 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:39.278190 29695 raft_consensus.cc:2741] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260924 14:30:39.208328 29777 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.265640 29532 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:39.296778 29466 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.572s	user 0.005s	sys 0.005s
I20260924 14:30:39.297057 29764 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:39.297479 29970 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:39.336205 29914 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:39.240877 29822 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.299510 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:39.302510 29938 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:39.303326 29820 consensus_meta.cc:287] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.762s	user 0.009s	sys 0.003s
I20260924 14:30:39.281560 29679 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:39.322799 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 276207us
I20260924 14:30:39.369043 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:39.370329 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:39.370452 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1241) took 1126 ms. Trace:
W20260924 14:30:39.281472 29975 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260924 14:30:39.371853 29914 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:39.373163 29972 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260924 14:30:39.375015 29976 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
W20260924 14:30:39.364746 29726 consensus_meta.cc:287] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.980s	user 0.007s	sys 0.007s
W20260924 14:30:39.376750 29961 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260924 14:30:39.376214 29726 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:39.377942 29903 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:39.378727 29903 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:39.378834 29726 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:39.285533 29830 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:39.287949 29886 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:39.285169 29921 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:39.337764 29794 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260924 14:30:39.340332 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 257372us
I20260924 14:30:39.288362 29967 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:39.305269 29887 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:39.274106 29762 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:39.333935 29968 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260924 14:30:39.298012 29966 consensus_queue.cc:1046] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260924 14:30:39.348680 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:39.370901 26327 rpcz_store.cc:276] 0924 14:30:38.243860 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.243971 (+   111us) service_pool.cc:227] Handling call
0924 14:30:38.244270 (+   299us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:39.222848 (+978578us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:39.370242 (+147394us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b540001ead4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:39.370390 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2194469760}
I20260924 14:30:39.351809 29764 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:39.334182 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 45924.6us
I20260924 14:30:39.388435 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 598) took 1614 ms. Trace:
I20260924 14:30:39.430150 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:39.430038 27188 rpcz_store.cc:276] 0924 14:30:37.774017 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.785439 (+ 11422us) service_pool.cc:227] Handling call
0924 14:30:37.818843 (+ 33404us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:38.396008 (+577165us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:39.388192 (+992184us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b5400260a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:39.388380 (+   188us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1325705472}
I20260924 14:30:39.382576 26405 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:39.410270 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:39.418608 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.012s	sys 0.008s
W20260924 14:30:39.432034 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.013s	sys 0.008s
I20260924 14:30:39.432060 27186 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:39.432638 26250 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260924 14:30:39.352384 29834 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.002s
W20260924 14:30:39.364202 29964 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.199s	user 0.000s	sys 0.002s
W20260924 14:30:39.363979 29963 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.253s	user 0.001s	sys 0.001s
W20260924 14:30:39.385176 29931 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 1.120s	user 0.002s	sys 0.000s
I20260924 14:30:39.373054 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 628) took 1484 ms. Trace:
I20260924 14:30:39.446131 26326 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:39.433023 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:39.387459 29845 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:39.356426 29695 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:39.440532 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.382292 29814 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:39.448418 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:39.448788 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.466089 26255 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:39.361214 29540 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:39.419399 29887 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:39.446354 27181 rpcz_store.cc:276] 0924 14:30:37.888434 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.888767 (+   333us) service_pool.cc:227] Handling call
0924 14:30:39.372854 (+1484087us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540000d554. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:39.372986 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3001587456}
I20260924 14:30:39.343533 29466 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.428912 29790 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:39.379235 29903 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.458140 27186 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:39.465001 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:39.418677 29967 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:39.410915 29970 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:39.449957 29633 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:39.466228 26251 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:39.495878 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.496762 26251 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:39.497824 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.498543 26251 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:39.473054 26253 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:39.465761 29888 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:39.381436 29925 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:39.440124 29794 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:30:39.460676 27194 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:39.472222 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.466784 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:39.524590 26399 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 11: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:39.446784 29957 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:39.492542 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:39.494359 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 120255us
I20260924 14:30:39.502158 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.527501 26250 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:39.527897 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:39.528795 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:39.529059 26295 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:39.529512 26250 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:39.524590 26249 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:39.505971 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.480326 29887 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
W20260924 14:30:39.501072 29979 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260924 14:30:39.554203 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:39.554527 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 596) took 1789 ms. Trace:
W20260924 14:30:39.555250 29978 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260924 14:30:39.555780 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 676) took 1305 ms. Trace:
I20260924 14:30:39.471755 29695 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:39.440141 29834 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:39.445356 29764 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:39.480993 29890 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:39.533051 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 115348us
I20260924 14:30:39.533192 26253 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260924 14:30:39.519930 29888 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.531840 29794 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:39.554896 27189 rpcz_store.cc:276] 0924 14:30:37.765005 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.779789 (+ 14784us) service_pool.cc:227] Handling call
0924 14:30:37.820429 (+ 40640us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:38.435319 (+614890us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:39.554263 (+1118944us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b540065f754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:39.554470 (+   207us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1412134784}
I20260924 14:30:39.560041 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.560518 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:39.563325 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.564469 26250 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
W20260924 14:30:39.471413 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 476906us
I20260924 14:30:39.556231 26325 rpcz_store.cc:276] 0924 14:30:38.250849 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.267135 (+ 16286us) service_pool.cc:227] Handling call
0924 14:30:39.555601 (+1288466us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54007ce7d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:39.555719 (+   118us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":342344576}
I20260924 14:30:39.574707 26255 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:39.602110 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:39.548365 29984 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.265s	user 0.003s	sys 0.000s
I20260924 14:30:39.608215 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.610682 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1136) took 1721 ms. Trace:
I20260924 14:30:39.509346 29633 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:39.616958 26397 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:39.617614 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.626102 26257 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
W20260924 14:30:39.600834 29960 consensus_meta.cc:287] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.652s	user 0.019s	sys 0.000s
I20260924 14:30:39.623723 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1039) took 1895 ms. Trace:
I20260924 14:30:39.635937 29970 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:39.636029 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:39.648350 26330 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:39.640282 26403 rpcz_store.cc:276] 0924 14:30:37.728357 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.729854 (+  1497us) service_pool.cc:227] Handling call
0924 14:30:37.744251 (+ 14397us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:37.754438 (+ 10187us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:37.754466 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:37.759915 (+  5449us) log.cc:842] Serialized 26 byte log entry
0924 14:30:38.019570 (+259655us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:38.019622 (+    52us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:38.019712 (+    90us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:39.384346 (+1364634us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:39.606833 (+222487us) raft_consensus.cc:1671] finished
0924 14:30:39.606865 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:39.623610 (+ 16745us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4728,"raft.queue_time_us":19342,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":5581,"spinlock_wait_cycles":21024128,"thread_start_us":200264,"threads_started":1,"wal-append.queue_time_us":466173}
I20260924 14:30:39.650242 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:39.640746 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.653712 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.657608 26250 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:39.611268 29966 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:39.618278 29794 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:39.607676 29790 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:30:39.665091 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:39.665997 27181 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:39.665820 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.678614 27189 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:39.619871 26258 rpcz_store.cc:276] 0924 14:30:37.888947 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.889153 (+   206us) service_pool.cc:227] Handling call
0924 14:30:37.961946 (+ 72793us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:38.645387 (+683441us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:38.645420 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:38.672895 (+ 27475us) log.cc:842] Serialized 26 byte log entry
0924 14:30:38.774738 (+101843us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:38.774795 (+    57us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:38.774908 (+   113us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:39.610397 (+835489us) raft_consensus.cc:1671] finished
0924 14:30:39.610428 (+    31us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:39.610589 (+   161us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":689300,"raft.queue_time_us":693805,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":5923,"thread_start_us":80895,"threads_started":1,"wal-append.queue_time_us":229581}
I20260924 14:30:39.674482 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.681152 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.686450 27186 raft_consensus.cc:2354] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11.
I20260924 14:30:39.692556 26253 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:30:39.614220 29938 consensus_queue.cc:1046] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260924 14:30:39.695703 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:39.693929 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 200299us
I20260924 14:30:39.649086 29960 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:39.703724 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:39.708452 29633 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:39.721230 29633 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:39.721599 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.704819 30002 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:39.724011 30002 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:39.724598 30002 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:39.725456 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1381) took 1196 ms. Trace:
I20260924 14:30:39.726100 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1237) took 1489 ms. Trace:
I20260924 14:30:39.726501 26324 rpcz_store.cc:276] 0924 14:30:38.236631 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.236771 (+   140us) service_pool.cc:227] Handling call
0924 14:30:39.725925 (+1489154us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54005bb3d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:39.726047 (+   122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":464561920}
I20260924 14:30:39.728083 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 607) took 1351 ms. Trace:
I20260924 14:30:39.728677 27192 rpcz_store.cc:276] 0924 14:30:38.376572 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.400093 (+ 23521us) service_pool.cc:227] Handling call
0924 14:30:38.400381 (+   288us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:38.745316 (+344935us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:38.745348 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:38.745387 (+    39us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:38.745420 (+    33us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:39.727703 (+982283us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b540014ffd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:39.727791 (+    88us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:39.728019 (+   228us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":502,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":792533248}
I20260924 14:30:39.731559 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1374) took 1926 ms. Trace:
I20260924 14:30:39.701918 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:39.725890 26405 rpcz_store.cc:276] 0924 14:30:38.528843 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.528970 (+   127us) service_pool.cc:227] Handling call
0924 14:30:39.725296 (+1196326us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b54001c1954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:39.725397 (+   101us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1945040768}
I20260924 14:30:39.705549 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.736245 26257 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:39.706686 29952 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260924 14:30:39.683586 29726 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:39.707391 29695 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:39.708390 29762 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:39.762065 27185 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 626) took 1901 ms (client timeout 1656 ms). Trace:
W20260924 14:30:39.762537 27185 rpcz_store.cc:269] 0924 14:30:37.860153 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.860418 (+   265us) service_pool.cc:227] Handling call
0924 14:30:39.761828 (+1901410us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b5400326254. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:39.761983 (+   155us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1382110592}
I20260924 14:30:39.763839 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:39.764999 30006 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.054s	user 0.002s	sys 0.000s
I20260924 14:30:39.765159 27186 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:39.765558 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.714252 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.708957 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1326) took 2672 ms. Trace:
I20260924 14:30:39.716173 30000 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:39.766942 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.714390 29888 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:39.721666 29633 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:39.717976 29903 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:39.693038 29966 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:39.732020 26401 rpcz_store.cc:276] 0924 14:30:37.805217 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.821417 (+ 16200us) service_pool.cc:227] Handling call
0924 14:30:37.821624 (+   207us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:37.895020 (+ 73396us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:39.731250 (+1836230us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b5400475c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:39.731489 (+   239us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":116807040}
I20260924 14:30:39.674784 29890 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:39.672354 29532 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:39.718601 29999 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:30:39.723302 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.663080 29925 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:39.712401 30003 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260924 14:30:39.761744 29726 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:39.767931 27192 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:39.768190 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 681) took 1228 ms. Trace:
I20260924 14:30:39.796314 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.798925 26332 rpcz_store.cc:276] 0924 14:30:38.539777 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.540932 (+  1155us) service_pool.cc:227] Handling call
0924 14:30:38.541280 (+   348us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:38.651392 (+110112us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:39.767950 (+1116558us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b54007d3cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:39.768130 (+   180us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":189375360}
I20260924 14:30:39.805333 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 610) took 1053 ms. Trace:
I20260924 14:30:39.768296 29633 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:39.796259 29925 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.829381 29764 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:39.830255 29466 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:39.782064 29890 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:39.832566 29890 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:39.833253 29629 consensus_meta.cc:287] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.009s
I20260924 14:30:39.833885 29629 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260924 14:30:39.834678 27191 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 4. Ops: []
I20260924 14:30:39.805887 27182 rpcz_store.cc:276] 0924 14:30:38.751801 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.751946 (+   145us) service_pool.cc:227] Handling call
0924 14:30:38.764689 (+ 12743us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:38.782887 (+ 18198us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:38.782935 (+    48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:38.798132 (+ 15197us) log.cc:842] Serialized 26 byte log entry
0924 14:30:39.104858 (+306726us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:30:39.104911 (+    53us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:39.105002 (+    91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:39.375062 (+270060us) spinlock_profiling.cc:243] Waited 4.96 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:39.797977 (+422915us) raft_consensus.cc:1671] finished
0924 14:30:39.798009 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:39.805248 (+  7239us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":12146,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":17328,"spinlock_wait_cycles":11419008,"thread_start_us":217175,"threads_started":1,"wal-append.queue_time_us":324980}
I20260924 14:30:39.767410 29438 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:39.784420 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.766453 26256 rpcz_store.cc:276] 0924 14:30:37.036387 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.036515 (+   128us) service_pool.cc:227] Handling call
0924 14:30:37.067404 (+ 30889us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:37.181923 (+114519us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:37.181974 (+    51us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:37.200207 (+ 18233us) log.cc:842] Serialized 26 byte log entry
0924 14:30:37.716145 (+515938us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b18003f5500. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:30:37.804402 (+ 88257us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:37.804441 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:39.708518 (+1904077us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54007823d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:39.708603 (+    85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:39.708663 (+    60us) raft_consensus.cc:1671] finished
0924 14:30:39.708681 (+    18us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:39.708861 (+   180us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":54746,"raft.queue_time_us":1103,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":7588,"spinlock_wait_cycles":282589312,"thread_start_us":88060,"threads_started":1,"wal-append.queue_time_us":767269,"wal-append.run_cpu_time_us":1785,"wal-append.run_wall_time_us":1064463}
I20260924 14:30:39.844041 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.844955 29726 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:39.767431 30000 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:39.847514 30000 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:39.769979 29966 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:39.837705 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 303001us
I20260924 14:30:39.801091 29778 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260924 14:30:39.829542 29925 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:39.801779 29952 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:39.859167 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 383737us
I20260924 14:30:39.748356 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:39.882225 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:39.853933 29438 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:39.862651 30000 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:39.932662 29794 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:39.934051 30012 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260924 14:30:39.946148 30002 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:39.936415 29845 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:39.950827 27181 raft_consensus.cc:2396] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260924 14:30:39.862519 29952 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:39.872293 29890 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260924 14:30:39.952941 29952 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:39.947706 30000 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:39.953847 29887 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:39.955013 30000 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:39.955500 26250 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:39.951884 30002 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:39.922640 29989 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:39.956882 29989 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:39.960287 29989 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:39.965493 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 289563us
I20260924 14:30:39.927963 29921 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:39.999121 27190 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260924 14:30:39.992655 27126 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:40.001066 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:40.001801 29886 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.002s
I20260924 14:30:40.001924 26332 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:40.002246 29886 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:39.989681 30002 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:39.999035 29764 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:40.004038 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:39.998648 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:40.007280 26401 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:40.003561 30002 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.019240 26324 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:40.004388 29764 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:40.015708 26257 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:39.990042 29834 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:40.036769 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.016229 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:40.023748 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.026039 29886 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:40.027248 29794 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:40.022220 29967 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.002s
I20260924 14:30:40.028517 26253 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:40.039212 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:39.989849 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:40.037603 26324 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:40.042524 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.052438 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.001969 29999 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:30:40.028370 29682 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.859s	user 0.020s	sys 0.004s
I20260924 14:30:40.058475 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:40.036929 29830 consensus_meta.cc:287] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.613s	user 0.005s	sys 0.006s
I20260924 14:30:40.021006 29629 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:30:40.047164 29501 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
I20260924 14:30:39.758266 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:40.064558 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.072621 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.064679 26330 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:40.040345 30000 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:39.976116 29438 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.081781 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:40.070725 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 312530us
I20260924 14:30:40.084970 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.099574 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:40.101824 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:40.101971 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.028632 29820 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260924 14:30:40.062382 29834 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:40.108800 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.081348 29794 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:40.119925 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.121510 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.122078 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.122735 26401 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:40.122346 26327 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:40.124511 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.126247 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.059021 29967 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.105239 29920 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:40.106307 29886 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:30:40.123513 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:40.145484 26330 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:40.064595 29682 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:40.145555 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.147787 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.118786 29834 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:40.146994 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:40.078394 29830 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.147091 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:40.156276 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:40.158015 26251 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:40.063496 29999 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:40.098443 30000 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:40.158139 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:40.165853 27192 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260924 14:30:40.166414 27192 pending_rounds.cc:76] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 4
I20260924 14:30:40.166666 27192 pending_rounds.cc:97] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260924 14:30:40.166934 27192 raft_consensus.cc:2866] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:40.167950 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.130096 29887 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:40.145375 29794 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:40.112067 29633 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.170549 26249 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:40.173264 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.174295 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:40.171939 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.006s	sys 0.012s
W20260924 14:30:40.175750 29954 log.cc:924] Time spent T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.447s	user 0.003s	sys 0.000s
W20260924 14:30:40.180656 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.006s	sys 0.012s
W20260924 14:30:40.174042 29974 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.946s	user 0.017s	sys 0.006s
W20260924 14:30:40.182849 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 532696us
I20260924 14:30:40.188695 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:40.173341 29794 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:40.193296 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:40.174144 29834 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:40.197139 26401 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:30:40.197721 26328 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.005s	sys 0.012s
W20260924 14:30:40.201471 26328 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.006s	sys 0.012s
I20260924 14:30:40.202481 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 678) took 1951 ms. Trace:
I20260924 14:30:40.186882 29814 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:40.219303 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:40.241369 26402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1199) took 2340 ms (client timeout 1538 ms). Trace:
W20260924 14:30:40.242034 26402 rpcz_store.cc:269] 0924 14:30:37.900584 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.901017 (+   433us) service_pool.cc:227] Handling call
0924 14:30:40.241005 (+2339988us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b5400290754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:40.241230 (+   225us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1274401536}
I20260924 14:30:40.238862 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:40.196724 29917 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:40.196380 29794 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:40.260908 26401 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:40.202976 26328 rpcz_store.cc:276] 0924 14:30:38.251383 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.407241 (+155858us) service_pool.cc:227] Handling call
0924 14:30:38.411513 (+  4272us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:38.436694 (+ 25181us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:38.436738 (+    44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:38.458859 (+ 22121us) log.cc:842] Serialized 26 byte log entry
0924 14:30:38.572732 (+113873us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b180021ca60. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:30:39.553703 (+980971us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b180021ca60. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b8fdd82c 00007f09b64089dc 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411
0924 14:30:39.553863 (+   160us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:39.600492 (+ 46629us) log.cc:842] Serialized 14 byte log entry
0924 14:30:40.202140 (+601648us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:40.202223 (+    83us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:40.202267 (+    44us) raft_consensus.cc:1671] finished
0924 14:30:40.202278 (+    11us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:40.202419 (+   141us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":48513,"prepare.queue_time_us":63713,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":8785,"raft.queue_time_us":5746,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":5325,"spinlock_wait_cycles":8534144,"thread_start_us":629703,"threads_started":2,"wal-append.queue_time_us":30250,"wal-append.run_cpu_time_us":2096,"wal-append.run_wall_time_us":1018075}
I20260924 14:30:40.207733 29915 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:40.272330 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:40.270695 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:40.273154 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:40.208015 29764 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:40.276649 26324 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:40.245666 29887 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:40.277962 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:40.280030 26327 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:40.281104 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:40.285686 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:40.266808 29917 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:40.402518 29888 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:30:40.278702 29850 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.011s	sys 0.013s
I20260924 14:30:40.192251 29974 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.245796 29814 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:40.419858 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:40.420830 27182 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:30:40.422732 26371 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.011s	sys 0.032s
W20260924 14:30:40.421708 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 232736us
W20260924 14:30:40.425650 26371 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.011s	sys 0.032s
W20260924 14:30:40.427829 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 675313us
I20260924 14:30:40.409507 29917 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:40.413794 29954 log.cc:924] Time spent T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.143s	user 0.004s	sys 0.000s
I20260924 14:30:40.439857 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:40.409646 29532 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:40.443969 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:40.410598 29820 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:40.428988 26405 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:30:40.433246 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:40.456053 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:40.457185 27182 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:40.436107 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:40.437021 27188 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:40.413009 29438 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:40.284205 29970 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.150s
I20260924 14:30:40.456875 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:40.420795 29850 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.011s	sys 0.014s
W20260924 14:30:40.502951 30023 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260924 14:30:40.504601 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 670) took 2258 ms. Trace:
I20260924 14:30:40.505067 26323 rpcz_store.cc:276] 0924 14:30:38.246640 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.391744 (+145104us) service_pool.cc:227] Handling call
0924 14:30:38.437568 (+ 45824us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:38.442464 (+  4896us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:38.442496 (+    32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:38.454194 (+ 11698us) log.cc:842] Serialized 26 byte log entry
0924 14:30:38.674169 (+219975us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:38.700264 (+ 26095us) log.cc:842] Serialized 14 byte log entry
0924 14:30:38.950364 (+250100us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:38.950470 (+   106us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:40.162009 (+1211539us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:40.504175 (+342166us) raft_consensus.cc:1671] finished
0924 14:30:40.504324 (+   149us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:40.504498 (+   174us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":118563,"prepare.queue_time_us":26530,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":81131,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":29750,"spinlock_wait_cycles":72289536,"thread_start_us":348817,"threads_started":2,"wal-append.queue_time_us":231783}
W20260924 14:30:40.461014 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 180004us
I20260924 14:30:40.473151 29820 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:40.509490 26224 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:40.510037 29820 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:40.425654 29682 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:40.447652 29945 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.011s	sys 0.030s
W20260924 14:30:40.511372 29945 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.012s	sys 0.031s
I20260924 14:30:40.453959 29952 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:40.449558 30000 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:40.425546 29888 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:40.478709 26330 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:40.492029 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:40.394251 29887 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:40.510883 29682 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:40.471292 29733 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:40.501857 29764 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:40.513923 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 237425us
I20260924 14:30:40.503034 30043 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260924 14:30:40.508788 30015 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.317s
I20260924 14:30:40.448187 29830 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:40.434437 29967 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:40.527606 29587 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:40.527798 29967 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:40.447425 29845 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:40.528177 29587 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.528689 29887 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:40.472446 29532 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:40.509555 29932 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.326s
I20260924 14:30:40.515962 30025 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260924 14:30:40.528927 29845 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:40.532194 30015 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:40.532683 30015 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:40.533604 29845 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:40.534649 30015 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:40.535238 29888 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:40.523159 26369 leader_election.cc:398] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11.
W20260924 14:30:40.512926 30017 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.401s	user 0.002s	sys 0.000s
I20260924 14:30:40.525700 29814 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:40.526160 29700 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260924 14:30:40.516433 29886 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:40.539260 26369 leader_election.cc:401] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260924 14:30:40.545451 26328 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:40.561398 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:40.566180 26256 raft_consensus.cc:2396] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260924 14:30:40.515436 29633 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:40.519604 29733 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:40.520535 29764 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.573137 29733 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:40.574409 29764 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:40.576509 29532 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:40.530241 29587 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:40.585479 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:40.520041 30000 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:40.515836 29999 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:40.591635 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 72247us
I20260924 14:30:40.515388 29952 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:40.520390 29682 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:40.622095 30000 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:40.600711 30024 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:40.617681 29700 consensus_queue.cc:1046] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260924 14:30:40.687901 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 469322us
I20260924 14:30:40.692555 29952 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:40.694504 29700 raft_consensus.cc:3032] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:40.694617 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 885) took 2975 ms. Trace:
I20260924 14:30:40.694975 29700 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:40.696694 29974 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:40.694543 29794 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.699512 26258 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:40.695273 29983 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:40.706444 30028 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260924 14:30:40.698935 30024 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:40.761880 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.763422 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.770753 26330 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:40.764451 26324 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:40.764355 29983 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:40.758823 29682 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:40.695081 26252 rpcz_store.cc:276] 0924 14:30:37.719395 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:37.731154 (+ 11759us) service_pool.cc:227] Handling call
0924 14:30:37.750008 (+ 18854us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:37.818740 (+ 68732us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:37.818771 (+    31us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:30:37.892863 (+ 74092us) log.cc:842] Serialized 52 byte log entry
0924 14:30:38.245024 (+352161us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:38.264365 (+ 19341us) log.cc:842] Serialized 14 byte log entry
0924 14:30:38.673799 (+409434us) log.cc:842] Serialized 14 byte log entry
0924 14:30:38.712488 (+ 38689us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:40.694185 (+1981697us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b540015e5d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:40.694237 (+    52us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:40.694374 (+   137us) raft_consensus.cc:1671] finished
0924 14:30:40.694388 (+    14us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:40.694547 (+   159us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":487974,"prepare.queue_time_us":22680,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":3797,"raft.queue_time_us":479327,"raft.run_cpu_time_us":457,"raft.run_wall_time_us":41215,"spinlock_wait_cycles":83287424,"thread_start_us":257396,"threads_started":1,"wal-append.queue_time_us":496185,"wal-append.run_cpu_time_us":3601,"wal-append.run_wall_time_us":1251989}
I20260924 14:30:40.770742 26328 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:40.780586 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:40.773425 29920 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.625s	user 0.012s	sys 0.000s
I20260924 14:30:40.790628 27189 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:30:40.792300 27182 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
W20260924 14:30:40.779827 29849 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.670s	user 0.023s	sys 0.032s
W20260924 14:30:40.797107 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 287665us
W20260924 14:30:40.754263 29991 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.011s	sys 0.037s
I20260924 14:30:40.798230 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:40.798262 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:40.799546 26252 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:40.802114 27125 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:40.797031 29849 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.729s	user 0.024s	sys 0.033s
I20260924 14:30:40.811169 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.811169 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.813226 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.835568 26323 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:40.829741 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:40.796581 29824 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:40.840919 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:40.835816 26324 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:40.755674 29820 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:40.841770 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:40.845973 26252 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:30:40.800740 29991 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.011s	sys 0.037s
I20260924 14:30:40.848420 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.834105 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:40.857399 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.805701 29532 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260924 14:30:40.853704 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 361748us
I20260924 14:30:40.856283 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.834692 29890 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:40.864392 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:40.856475 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:40.840354 29682 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:30:40.864820 26327 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:40.840711 29849 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:40.850649 29887 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:40.785660 29700 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:40.871147 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:40.791827 30024 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:40.862008 29532 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:40.893718 27189 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260924 14:30:40.896126 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:40.856956 29726 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.027s
I20260924 14:30:40.898705 27123 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:40.855849 29824 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:40.904424 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.895290 29733 consensus_queue.cc:1046] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260924 14:30:40.906160 29733 raft_consensus.cc:3032] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:40.906571 29733 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:40.907428 29733 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.908777 29733 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:40.911268 30013 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:40.911867 30013 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:40.912509 26252 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:40.912710 30013 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.916232 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:40.917127 29845 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:30:40.917654 29845 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:40.918569 29845 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:40.920567 26403 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:40.867004 29682 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:40.894692 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:40.868922 29794 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:40.912747 30024 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:40.920252 29726 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260924 14:30:40.897234 29532 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:40.934551 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:40.929698 26369 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:40.925352 29700 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:41.015964 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:40.921043 29830 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:41.009025 30019 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:41.009308 30044 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:40.935269 26329 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
W20260924 14:30:40.930891 30042 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260924 14:30:41.019959 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 90337.9us
I20260924 14:30:40.865825 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:41.083636 27185 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:30:41.021502 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 125467us
I20260924 14:30:40.890580 29970 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260924 14:30:40.929114 29682 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:30:41.027169 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.050s	sys 0.068s
I20260924 14:30:41.084616 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:41.161875 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.052s	sys 0.068s
W20260924 14:30:41.163019 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 704130us
I20260924 14:30:41.164420 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:41.168566 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:41.173903 26296 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.000s	sys 0.021s
W20260924 14:30:41.174479 26296 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.000s	sys 0.022s
I20260924 14:30:41.175908 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:41.112550 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:41.226341 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:40.932865 30043 consensus_queue.cc:1046] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.169s
I20260924 14:30:41.227149 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:41.228394 27189 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:30:41.238152 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 125723us
I20260924 14:30:41.079751 29827 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:41.242686 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:41.249869 26405 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:30:41.251906 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:41.103331 29796 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:41.102607 30024 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:41.104341 29905 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:41.267817 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:41.252827 26400 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:30:41.262916 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:41.100172 29726 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:41.100553 29983 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:41.238327 29768 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.027s	sys 0.064s
I20260924 14:30:41.388738 26369 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:41.385643 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:41.272210 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1146) took 2550 ms. Trace:
W20260924 14:30:41.389788 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 305264us
I20260924 14:30:41.389705 26324 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:41.078539 29830 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:41.227041 30051 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:41.250559 29834 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:41.233193 29788 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:41.395527 26254 rpcz_store.cc:276] 0924 14:30:38.722112 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.821494 (+ 99382us) service_pool.cc:227] Handling call
0924 14:30:38.847615 (+ 26121us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:39.075753 (+228138us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:39.075788 (+    35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:39.075840 (+    52us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:39.075868 (+    28us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:41.271936 (+2196068us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54004538d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:41.272002 (+    66us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:41.272151 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":524178688}
I20260924 14:30:41.399737 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:41.424149 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1054) took 1228 ms. Trace:
I20260924 14:30:41.268276 29796 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:41.425717 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:41.267853 30024 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:41.432485 27192 rpcz_store.cc:276] 0924 14:30:40.195813 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.196270 (+   457us) service_pool.cc:227] Handling call
0924 14:30:40.201181 (+  4911us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:40.237866 (+ 36685us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:40.237904 (+    38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:40.253157 (+ 15253us) log.cc:842] Serialized 26 byte log entry
0924 14:30:40.558776 (+305619us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:40.558838 (+    62us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:40.558935 (+    97us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:41.423838 (+864903us) raft_consensus.cc:1671] finished
0924 14:30:41.423867 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:41.424071 (+   204us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":6876,"raft.run_cpu_time_us":6492,"raft.run_wall_time_us":15733,"thread_start_us":178866,"threads_started":1,"wal-append.queue_time_us":518584}
W20260924 14:30:41.434194 26402 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.030s	sys 0.043s
W20260924 14:30:41.445315 26402 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.030s	sys 0.043s
I20260924 14:30:41.441385 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:41.389659 29820 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:41.081218 29849 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:41.082944 30019 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:41.453507 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1072) took 1026 ms. Trace:
W20260924 14:30:41.404747 29850 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.003s	sys 0.010s
I20260924 14:30:41.290657 29991 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:41.402921 29768 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.030s	sys 0.064s
I20260924 14:30:41.413956 29983 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260924 14:30:41.462574 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 294112us
I20260924 14:30:41.453912 26402 rpcz_store.cc:276] 0924 14:30:40.426636 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.426800 (+   164us) service_pool.cc:227] Handling call
0924 14:30:40.444078 (+ 17278us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:40.444286 (+   208us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:30:40.894889 (+450603us) log.cc:842] Serialized 14 byte log entry
0924 14:30:41.453125 (+558236us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:41.453178 (+    53us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:41.453229 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:41.453303 (+    74us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:41.453424 (+   121us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7285,"thread_start_us":931422,"threads_started":2}
I20260924 14:30:41.463166 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:41.472816 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:41.463685 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1338) took 2015 ms. Trace:
I20260924 14:30:41.423350 29886 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:41.473888 26251 rpcz_store.cc:276] 0924 14:30:39.448657 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.499349 (+ 50692us) service_pool.cc:227] Handling call
0924 14:30:41.463477 (+1964128us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b540002b554. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:41.463626 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1431451392}
I20260924 14:30:41.475792 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1165) took 1312 ms. Trace:
I20260924 14:30:41.273313 29827 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:41.404551 30055 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:41.101251 30044 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:41.481123 26323 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.015s	sys 0.047s
I20260924 14:30:41.484153 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:41.449081 30024 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:41.443208 29796 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:30:41.461170 29850 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.004s	sys 0.011s
I20260924 14:30:41.476162 26251 rpcz_store.cc:276] 0924 14:30:40.163681 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.474908 (+1311227us) service_pool.cc:227] Handling call
0924 14:30:41.475201 (+   293us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:41.475407 (+   206us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:41.475430 (+    23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:41.475534 (+   104us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:41.475571 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:41.475630 (+    59us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:41.475736 (+   106us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:41.487502 26325 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:41.432826 30059 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:41.278218 29905 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:41.408320 29726 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:41.447858 29788 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:41.392863 30047 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.128s
I20260924 14:30:41.431506 30051 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:41.525101 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:41.481388 26295 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.993s	user 0.016s	sys 0.046s
I20260924 14:30:41.452735 30050 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:30:41.609781 26295 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.017s	sys 0.046s
I20260924 14:30:41.610149 26324 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:41.611739 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:41.465171 29820 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:41.589784 26297 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.094s (SENT) [suppressed 9 similar messages]
W20260924 14:30:41.596714 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 112682us
I20260924 14:30:41.598172 30059 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:41.478397 29917 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.965s	user 0.012s	sys 0.001s
I20260924 14:30:41.478096 29983 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:41.590631 26323 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.015s	sys 0.047s
I20260924 14:30:41.617986 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:41.533435 29677 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:41.608124 30054 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:41.476302 29768 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:30:41.464450 29834 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.144s
I20260924 14:30:41.653044 29921 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:41.653410 29934 raft_consensus.cc:3032] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:30:41.653996 29934 raft_consensus.cc:729] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:41.654636 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 668) took 3408 ms. Trace:
I20260924 14:30:41.531692 30055 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260924 14:30:41.632275 30066 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260924 14:30:41.636109 30063 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260924 14:30:41.637107 29827 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:41.648988 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:41.653595 29921 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:41.465976 29991 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:41.658706 29934 consensus_queue.cc:260] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:41.658787 29551 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260924 14:30:41.655095 26326 rpcz_store.cc:276] 0924 14:30:38.245958 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.246088 (+   130us) service_pool.cc:227] Handling call
0924 14:30:38.260019 (+ 13931us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:39.446552 (+1186533us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:41.654383 (+2207831us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540012e654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:41.654574 (+   191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2727343360}
I20260924 14:30:41.643108 29983 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:41.644454 29850 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:41.676299 26324 raft_consensus.cc:2396] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260924 14:30:41.675284 29905 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:41.651603 30050 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:41.706964 26250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 902) took 2048 ms (client timeout 2596 ms). Trace:
W20260924 14:30:41.707458 26250 rpcz_store.cc:269] 0924 14:30:39.655299 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.658748 (+  3449us) service_pool.cc:227] Handling call
0924 14:30:41.703267 (+2044519us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54003827d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:41.703427 (+   160us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":595048192}
I20260924 14:30:41.707839 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1076) took 1268 ms. Trace:
I20260924 14:30:41.708168 30059 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:41.709085 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1169) took 1146 ms. Trace:
I20260924 14:30:41.708290 26399 rpcz_store.cc:276] 0924 14:30:40.439057 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.439206 (+   149us) service_pool.cc:227] Handling call
0924 14:30:40.471741 (+ 32535us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:40.769282 (+297541us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:40.769317 (+    35us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:30:40.788042 (+ 18725us) log.cc:842] Serialized 52 byte log entry
0924 14:30:41.341034 (+552992us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:41.341090 (+    56us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:41.341197 (+   107us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:41.707536 (+366339us) raft_consensus.cc:1671] finished
0924 14:30:41.707559 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:41.707717 (+   158us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":53716,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":768604,"threads_started":2,"wal-append.queue_time_us":704421}
I20260924 14:30:41.686157 29915 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:41.665254 29830 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:41.689742 30038 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.394s	user 0.000s	sys 0.001s
I20260924 14:30:41.624279 29820 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:41.643376 29917 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:41.701231 29700 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.001s
I20260924 14:30:41.697664 29850 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:41.739734 29850 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:41.741196 29850 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:41.704345 29733 consensus_meta.cc:287] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.795s	user 0.012s	sys 0.001s
I20260924 14:30:41.709450 26250 rpcz_store.cc:276] 0924 14:30:40.562306 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.708310 (+1146004us) service_pool.cc:227] Handling call
0924 14:30:41.708640 (+   330us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:41.708813 (+   173us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:41.708836 (+    23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:41.708873 (+    37us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:41.708899 (+    26us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:41.708946 (+    47us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:41.709037 (+    91us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:41.646639 29677 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:41.745592 29677 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:41.737790 29915 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:41.698242 30000 consensus_meta.cc:287] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.044s	user 0.010s	sys 0.004s
I20260924 14:30:41.751593 29700 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:41.747859 29830 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:41.752355 29726 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:41.752827 30044 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:41.759642 26330 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:41.699573 30019 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:41.762176 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 532318us
I20260924 14:30:41.762532 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1170) took 1084 ms. Trace:
I20260924 14:30:41.747673 29934 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:41.742975 29788 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:41.769752 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:41.766420 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.019s	sys 0.015s
I20260924 14:30:41.769534 26250 rpcz_store.cc:276] 0924 14:30:40.678335 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.742632 (+1064297us) service_pool.cc:227] Handling call
0924 14:30:41.761845 (+ 19213us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:41.762064 (+   219us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:41.762100 (+    36us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:41.762156 (+    56us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:41.762225 (+    69us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:41.762298 (+    73us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:41.762458 (+   160us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:41.751603 29991 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:41.763315 30000 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11.
W20260924 14:30:41.780412 30009 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.862s	user 0.005s	sys 0.012s
I20260924 14:30:41.777343 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:41.783411 29888 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:41.785079 29925 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260924 14:30:41.786609 26253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 905) took 2126 ms (client timeout 2166 ms). Trace:
W20260924 14:30:41.812083 26253 rpcz_store.cc:269] 0924 14:30:39.660555 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.722966 (+ 62411us) service_pool.cc:227] Handling call
0924 14:30:41.786409 (+2063443us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54004fae54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:41.786537 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":586844032}
I20260924 14:30:41.764305 29700 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:41.789467 30053 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.369s	user 0.000s	sys 0.002s
I20260924 14:30:41.790520 29915 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:41.794476 30010 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 2.071s	user 0.003s	sys 0.000s
I20260924 14:30:41.765419 30050 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:41.803333 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:41.807981 29888 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:41.799696 30070 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:41.823925 29682 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.596s	user 0.006s	sys 0.006s
I20260924 14:30:41.824193 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1063) took 2385 ms. Trace:
I20260924 14:30:41.829761 26398 rpcz_store.cc:276] 0924 14:30:39.438460 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.438605 (+   145us) service_pool.cc:227] Handling call
0924 14:30:39.461400 (+ 22795us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:39.461678 (+   278us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:39.461713 (+    35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:39.488879 (+ 27166us) log.cc:842] Serialized 26 byte log entry
0924 14:30:39.599123 (+110244us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:39.599185 (+    62us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:39.599421 (+   236us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:41.790054 (+2190633us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:41.823836 (+ 33782us) raft_consensus.cc:1671] finished
0924 14:30:41.823858 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:41.824110 (+   252us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":75127168,"thread_start_us":68897,"threads_started":1,"wal-append.queue_time_us":226211}
I20260924 14:30:41.765390 29830 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:41.833204 29830 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:41.822306 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:41.810693 29886 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:30:41.821584 29887 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.935s	user 0.007s	sys 0.006s
I20260924 14:30:41.838829 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:41.840684 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:41.826515 29888 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:41.842605 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:41.838063 29925 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:41.837487 30051 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:41.821166 29796 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:41.839890 29887 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:30:41.827154 26256 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.019s	sys 0.013s
W20260924 14:30:41.838166 26255 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.012s	sys 0.044s
W20260924 14:30:41.872134 26255 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.012s	sys 0.044s
W20260924 14:30:41.873075 26331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 673) took 3612 ms (client timeout 2197 ms). Trace:
I20260924 14:30:41.826804 30050 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:41.924569 26256 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.019s	sys 0.014s
W20260924 14:30:41.933507 26331 rpcz_store.cc:269] 0924 14:30:38.250101 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:38.286742 (+ 36641us) service_pool.cc:227] Handling call
0924 14:30:41.862394 (+3575652us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54007d37d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:41.862542 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":781313792}
W20260924 14:30:41.955701 26328 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.011s	sys 0.014s
W20260924 14:30:41.956172 26328 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.011s	sys 0.014s
I20260924 14:30:41.837386 29682 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:41.958170 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1160) took 2020 ms. Trace:
I20260924 14:30:41.957234 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1161) took 2002 ms. Trace:
I20260924 14:30:41.973124 27182 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260924 14:30:41.859542 29778 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:41.851637 30069 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:41.859720 29788 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:41.859597 30000 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:41.859231 29925 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:41.972863 26256 rpcz_store.cc:276] 0924 14:30:39.938149 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.601527 (+663378us) service_pool.cc:227] Handling call
0924 14:30:40.641574 (+ 40047us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:40.641866 (+   292us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:30:40.925677 (+283811us) log.cc:842] Serialized 14 byte log entry
0924 14:30:41.957752 (+1032075us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:41.957821 (+    69us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:41.957890 (+    69us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:41.957973 (+    83us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:41.958108 (+   135us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3978,"prepare.queue_time_us":893286,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":1280942,"threads_started":2}
I20260924 14:30:41.864403 29726 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:41.839169 29921 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:41.973399 26255 rpcz_store.cc:276] 0924 14:30:39.954401 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.598730 (+644329us) service_pool.cc:227] Handling call
0924 14:30:40.599199 (+   469us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:40.685213 (+ 86014us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:40.685258 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:40.706792 (+ 21534us) log.cc:842] Serialized 26 byte log entry
0924 14:30:41.226521 (+519729us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:41.951258 (+724737us) log.cc:842] Serialized 14 byte log entry
0924 14:30:41.956723 (+  5465us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:41.956807 (+    84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:41.956873 (+    66us) raft_consensus.cc:1671] finished
0924 14:30:41.956890 (+    17us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:41.957036 (+   146us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5212,"prepare.queue_time_us":313408,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":88271,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":11779,"thread_start_us":1132984,"threads_started":2,"wal-append.queue_time_us":692274}
I20260924 14:30:41.995003 30062 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:41.995311 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:41.841086 30070 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:41.996285 26255 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:41.997376 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:41.997779 30070 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:41.992268 30069 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:41.995635 30062 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:41.999294 27125 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:42.002370 30062 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:42.002792 30069 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:41.972309 29887 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:41.971074 29886 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:41.866257 30051 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.126s
I20260924 14:30:41.970557 30054 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:41.974262 29716 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:41.988361 30043 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:42.012673 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:41.990096 29915 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:41.989380 29820 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:42.020656 26400 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 3. Current term is 4. Ops: []
I20260924 14:30:42.003407 30069 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:42.023855 26256 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:42.024556 29677 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:42.025686 30050 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:42.029322 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:42.026859 29915 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:42.038939 30067 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260924 14:30:42.038888 29677 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:42.040225 29915 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:42.037465 29700 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:42.040947 29677 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:42.036271 30043 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:42.004484 29682 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:42.042622 30043 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:42.009757 29917 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:42.037451 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:42.007579 30073 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:30:41.987534 29778 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:42.044728 30073 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:42.045066 26399 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:42.027618 29716 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:42.049867 26400 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260924 14:30:42.051964 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:42.046365 29778 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:41.993216 30000 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:42.004899 29726 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:42.003882 29788 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:42.030244 29887 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:42.082222 26400 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260924 14:30:42.064814 29921 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:42.085181 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:42.087031 26256 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:42.036140 30069 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:42.047523 29925 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:42.022989 30054 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:42.083786 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:42.045995 30073 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:42.152631 26400 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:42.166188 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 622) took 1332 ms. Trace:
I20260924 14:30:42.087182 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:42.226280 27182 rpcz_store.cc:276] 0924 14:30:40.833913 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.892578 (+ 58665us) service_pool.cc:227] Handling call
0924 14:30:40.922163 (+ 29585us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:41.973607 (+1051444us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:42.165843 (+192236us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540014ffd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:42.166103 (+   260us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2416409344}
W20260924 14:30:42.224788 26329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1255) took 1411 ms (client timeout 1632 ms). Trace:
I20260924 14:30:42.053850 29682 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:42.247839 29682 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:42.248332 29682 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:42.249773 29888 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:42.251276 29824 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:42.253643 29933 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:42.058281 29532 consensus_meta.cc:287] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.041s	user 0.011s	sys 0.000s
I20260924 14:30:42.117174 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:42.066272 29716 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:42.234596 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:42.055371 29677 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:42.241453 26329 rpcz_store.cc:269] 0924 14:30:40.813763 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.813910 (+   147us) service_pool.cc:227] Handling call
0924 14:30:42.181238 (+1367328us) spinlock_profiling.cc:243] Waited 77 ms on lock 0x7b5400505854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:42.224685 (+ 43447us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":177042688}
I20260924 14:30:42.129472 30051 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:42.209043 30024 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.606s	user 0.006s	sys 0.006s
I20260924 14:30:42.825381 26251 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:42.458842 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.039s	sys 0.014s
I20260924 14:30:42.086481 29921 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:42.831110 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 642) took 3119 ms. Trace:
W20260924 14:30:42.826385 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.342s	user 0.041s	sys 0.024s
W20260924 14:30:42.461154 26222 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.011s	sys 0.033s
W20260924 14:30:42.833510 26222 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.441s	user 0.011s	sys 0.037s
W20260924 14:30:42.206348 29983 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.006s
W20260924 14:30:42.273980 26249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1345) took 2733 ms (client timeout 2094 ms). Trace:
I20260924 14:30:42.084868 29996 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:42.832698 27186 rpcz_store.cc:276] 0924 14:30:39.705650 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.732899 (+ 27249us) service_pool.cc:227] Handling call
0924 14:30:39.764946 (+ 32047us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:39.765619 (+   673us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:42.824752 (+3059133us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b5400326250. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:42.824914 (+   162us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":72877312}
I20260924 14:30:42.229450 30069 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:42.154268 29788 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:42.116060 29796 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:42.839123 26249 rpcz_store.cc:269] 0924 14:30:39.537316 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.538198 (+   882us) service_pool.cc:227] Handling call
0924 14:30:42.270400 (+2732202us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b54002a1a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:42.270562 (+   162us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1415285120}
I20260924 14:30:42.873157 29996 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:42.838093 27124 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.005s	sys 0.019s
W20260924 14:30:42.874550 27124 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.412s	user 0.006s	sys 0.020s
W20260924 14:30:42.872212 26256 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.647s	user 0.010s	sys 0.000s
I20260924 14:30:42.875684 26256 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:30:42.254894 29888 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:42.247332 30060 raft_consensus.cc:3032] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:42.087554 29887 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:42.254814 29532 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:42.275684 29934 consensus_meta.cc:287] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.004s
I20260924 14:30:42.876796 27125 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:42.885418 30076 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:42.882364 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:42.883734 26252 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.412s	user 0.030s	sys 0.019s
I20260924 14:30:42.274102 29933 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:42.830374 29925 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:42.838975 29677 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:42.269521 29726 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:42.300475 30002 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:42.481980 30000 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:42.891007 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:42.892274 27185 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260924 14:30:42.469018 29682 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:42.898299 26253 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:42.827026 30024 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:42.899233 27190 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.236s	user 0.000s	sys 0.050s
I20260924 14:30:42.880975 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:42.904167 26256 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:30:42.922572 27189 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:42.922706 26256 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:42.923930 27190 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.261s	user 0.001s	sys 0.050s
W20260924 14:30:42.924348 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.302s	user 0.021s	sys 0.025s
W20260924 14:30:42.924744 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.302s	user 0.021s	sys 0.025s
W20260924 14:30:42.904820 29845 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.351s	user 0.017s	sys 0.032s
W20260924 14:30:42.905658 26323 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.262s	user 0.010s	sys 0.034s
W20260924 14:30:42.906370 26252 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.435s	user 0.031s	sys 0.019s
I20260924 14:30:42.925563 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:42.894107 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52166us
I20260924 14:30:42.908339 30054 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:42.906577 26324 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.197s	user 0.028s	sys 0.018s
I20260924 14:30:42.907709 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:42.927762 26324 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.218s	user 0.028s	sys 0.018s
I20260924 14:30:42.928073 27186 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:42.845569 29921 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:42.461283 29994 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:42.891623 30060 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:42.840217 30051 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:42.909721 26295 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.297s	user 0.020s	sys 0.025s
W20260924 14:30:42.951462 26295 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.020s	sys 0.025s
I20260924 14:30:42.951711 26398 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:30:42.951957 26405 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.511s	user 0.027s	sys 0.031s
I20260924 14:30:42.864231 29983 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:42.907189 29981 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 3.651s	user 0.021s	sys 0.032s
I20260924 14:30:42.926667 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:42.923688 26256 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:42.929086 26250 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.146s	user 0.027s	sys 0.011s
W20260924 14:30:42.930855 26326 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.027s	sys 0.023s
W20260924 14:30:42.930152 29845 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.389s	user 0.017s	sys 0.032s
I20260924 14:30:42.950166 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:42.929255 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:42.910794 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.443s	user 0.021s	sys 0.026s
W20260924 14:30:42.971531 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.504s	user 0.022s	sys 0.026s
I20260924 14:30:42.955509 30002 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:42.953120 29934 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:42.956122 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.484s	user 0.027s	sys 0.007s
W20260924 14:30:42.958092 26369 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.000s	sys 0.000s
W20260924 14:30:42.981508 26369 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.001s	sys 0.000s
I20260924 14:30:42.956642 30084 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
W20260924 14:30:42.981889 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 728830us
I20260924 14:30:42.909767 30076 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:42.985976 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:42.961018 26405 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.520s	user 0.027s	sys 0.031s
I20260924 14:30:42.963245 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:42.967733 26326 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.197s	user 0.027s	sys 0.023s
I20260924 14:30:42.967790 26256 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:30:42.980466 30013 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.714s	user 0.025s	sys 0.014s
W20260924 14:30:42.968058 26250 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.185s	user 0.028s	sys 0.011s
I20260924 14:30:42.974308 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:42.974949 26254 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.454s	user 0.008s	sys 0.040s
I20260924 14:30:42.957943 29957 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260924 14:30:42.978366 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:42.917912 29933 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:42.969201 29981 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 3.713s	user 0.022s	sys 0.032s
W20260924 14:30:42.982445 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.511s	user 0.027s	sys 0.008s
I20260924 14:30:42.971920 29994 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:42.984822 29934 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:42.931296 26323 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.289s	user 0.011s	sys 0.035s
I20260924 14:30:42.991147 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:42.994643 26254 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.474s	user 0.008s	sys 0.040s
I20260924 14:30:43.014025 26256 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260924 14:30:42.997382 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.014613 26256 pending_rounds.cc:76] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 4
I20260924 14:30:43.015183 26256 pending_rounds.cc:97] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260924 14:30:42.998039 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:43.005086 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 1.53325e+06us
I20260924 14:30:43.017174 26256 raft_consensus.cc:2866] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:42.996276 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1052) took 2968 ms. Trace:
I20260924 14:30:43.013336 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1164) took 2984 ms. Trace:
I20260924 14:30:42.995790 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.021334 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:42.982585 30043 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.008s	sys 0.026s
I20260924 14:30:42.981688 30054 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:42.981869 30000 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:43.021981 27186 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:42.994299 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1172) took 2315 ms. Trace:
I20260924 14:30:43.024142 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.019773 26254 rpcz_store.cc:276] 0924 14:30:40.028755 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.446933 (+1418178us) service_pool.cc:227] Handling call
0924 14:30:41.470992 (+ 24059us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:43.012838 (+1541846us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:30:43.012886 (+    48us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:43.012949 (+    63us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:43.012993 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:43.013100 (+   107us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:43.013258 (+   158us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":285947,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":16742,"thread_start_us":1491591,"threads_started":1}
I20260924 14:30:43.004670 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1257) took 2165 ms. Trace:
I20260924 14:30:43.046939 26401 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 5. Current term is 6. Ops: []
I20260924 14:30:43.018895 27190 rpcz_store.cc:276] 0924 14:30:40.028356 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.029883 (+  1527us) service_pool.cc:227] Handling call
0924 14:30:40.056800 (+ 26917us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:40.057040 (+   240us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:40.057063 (+    23us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:41.662089 (+1605026us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b2c001c06e8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:30:41.662662 (+   573us) log.cc:842] Serialized 26 byte log entry
0924 14:30:42.924522 (+1261860us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:42.924577 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:42.925210 (+   633us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:42.993148 (+ 67938us) raft_consensus.cc:1671] finished
0924 14:30:42.993174 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:42.996210 (+  3036us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":74415360,"thread_start_us":1261366,"threads_started":1,"wal-append.queue_time_us":1328477}
I20260924 14:30:43.049499 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:43.015609 30013 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.774s	user 0.025s	sys 0.015s
I20260924 14:30:43.044984 26250 rpcz_store.cc:276] 0924 14:30:40.679040 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.776138 (+1097098us) service_pool.cc:227] Handling call
0924 14:30:41.782328 (+  6190us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:41.782545 (+   217us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:30:41.782863 (+   318us) log.cc:842] Serialized 14 byte log entry
0924 14:30:42.993879 (+1211016us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:42.993957 (+    78us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:42.994014 (+    57us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:42.994106 (+    92us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:42.994234 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":4960,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":2603,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":1210651,"threads_started":1}
I20260924 14:30:42.987794 29788 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:43.054383 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:43.050697 26298 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.439s	user 0.021s	sys 0.048s
W20260924 14:30:43.057158 26298 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.445s	user 0.022s	sys 0.048s
W20260924 14:30:43.057578 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 1.44615e+06us
W20260924 14:30:43.027794 30043 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.009s	sys 0.028s
I20260924 14:30:43.045218 26403 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:43.013155 30055 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:42.995090 27123 connection.cc:232] RPC call timeout handler was delayed by -1.4956133130008311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
W20260924 14:30:43.065500 27123 connection.cc:232] RPC call timeout handler was delayed by -1.4947084000004907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
I20260924 14:30:43.066066 27123 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:42.988044 29726 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:43.053429 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:43.051774 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:43.061657 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:42.996345 29830 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:43.001616 29933 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.003374 29994 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:43.019532 29938 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 3.303s	user 0.014s	sys 0.010s
I20260924 14:30:43.067708 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:42.993636 30077 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:30:42.982394 30002 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:42.994957 30084 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:43.067762 27182 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:43.070036 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:43.099705 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:43.069916 26298 connection.cc:232] RPC call timeout handler was delayed by -1.4468091620010455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
I20260924 14:30:43.100757 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.075538 29726 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:43.108095 29921 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:43.109658 29726 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.110018 30024 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:43.112016 29762 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:43.112764 29762 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:43.057847 30000 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:43.116022 30000 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:43.117257 26258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1158) took 3244 ms (client timeout 3276 ms). Trace:
W20260924 14:30:43.117808 26258 rpcz_store.cc:269] 0924 14:30:39.872712 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.154322 (+281610us) service_pool.cc:227] Handling call
0924 14:30:43.117039 (+2962717us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54000c0dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:43.117195 (+   156us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1246502144}
I20260924 14:30:43.119627 29996 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:43.064200 29957 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:43.120292 29957 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:43.120658 29957 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:43.050932 30054 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:43.123302 30054 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:43.101382 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.100217 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.069038 29788 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:43.124008 26401 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:43.102519 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:43.102735 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.124866 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.125705 26295 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:43.125793 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 694) took 1713 ms. Trace:
I20260924 14:30:43.065016 29532 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.065182 29887 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:43.127642 26404 raft_consensus.cc:2396] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260924 14:30:43.128507 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:43.098560 29938 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 3.382s	user 0.015s	sys 0.010s
I20260924 14:30:43.129722 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.133194 26400 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:43.129846 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.016748 30060 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:43.135761 26297 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.605s (SENT)
I20260924 14:30:43.139835 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.103986 30077 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:43.105780 29994 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:43.144471 27189 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:43.115298 29762 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:43.161080 29762 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:43.161561 29762 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:43.161940 29762 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:43.162007 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1406) took 1148 ms. Trace:
I20260924 14:30:43.162616 26399 rpcz_store.cc:276] 0924 14:30:42.014076 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:42.014223 (+   147us) service_pool.cc:227] Handling call
0924 14:30:43.161789 (+1147566us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b540036bad4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:43.161935 (+   146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":14731136}
I20260924 14:30:43.148202 29788 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.173236 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:43.177021 30002 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:43.144295 30077 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:30:43.181674 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 690) took 2166 ms. Trace:
I20260924 14:30:43.182471 29677 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:43.183018 29994 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:43.185357 29994 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:43.186081 26401 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:30:43.186764 26403 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
W20260924 14:30:43.187593 26328 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.231s	user 0.015s	sys 0.040s
W20260924 14:30:43.189426 26328 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.233s	user 0.015s	sys 0.040s
I20260924 14:30:43.151319 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.153187 29957 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:43.156677 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.131726 30054 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:43.158178 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 612) took 3501 ms. Trace:
I20260924 14:30:43.158459 26255 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:43.182204 26325 rpcz_store.cc:276] 0924 14:30:41.015680 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.384593 (+368913us) service_pool.cc:227] Handling call
0924 14:30:41.420966 (+ 36373us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:41.588143 (+167177us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:43.181422 (+1593279us) spinlock_profiling.cc:243] Waited 66.1 ms on lock 0x7b5400498254. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:43.181609 (+   187us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":152133504}
I20260924 14:30:43.147694 30084 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:43.069531 30055 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:43.138636 29682 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:43.193955 27191 rpcz_store.cc:276] 0924 14:30:39.656574 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.659286 (+  2712us) service_pool.cc:227] Handling call
0924 14:30:39.697080 (+ 37794us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:39.835126 (+138046us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:43.157921 (+3322795us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b5400071a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:43.158114 (+   193us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":315921792}
I20260924 14:30:43.199460 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.200045 26329 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 7: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:43.206310 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.201550 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.207633 26400 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 4: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:43.207314 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:43.188153 29849 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.388s	user 0.026s	sys 0.041s
I20260924 14:30:43.208794 26399 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:43.199481 30060 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:43.210587 26325 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:43.213613 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.202785 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 692) took 1816 ms. Trace:
I20260924 14:30:43.201269 29934 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.201633 30051 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:43.224097 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:43.212201 29849 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.412s	user 0.026s	sys 0.041s
I20260924 14:30:43.225899 29957 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.217155 29532 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:43.227530 30076 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.045799 26323 rpcz_store.cc:276] 0924 14:30:40.839872 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.842611 (+  2739us) service_pool.cc:227] Handling call
0924 14:30:40.842978 (+   367us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:40.843231 (+   253us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:30:41.641894 (+798663us) log.cc:842] Serialized 14 byte log entry
0924 14:30:43.004371 (+1362477us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:43.004417 (+    46us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:43.004466 (+    49us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:43.004526 (+    60us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:43.004623 (+    97us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":965589,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2115,"raft.queue_time_us":567,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7343,"thread_start_us":2143269,"threads_started":2}
I20260924 14:30:43.244513 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.231536 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:43.245337 26323 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:43.249145 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.250144 26323 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:43.222997 29983 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:43.238914 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:43.200126 30054 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:43.251456 26399 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:43.208878 26403 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:43.259577 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.236996 29902 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:43.264294 29902 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:43.266103 27188 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 631) took 3703 ms (client timeout 2193 ms). Trace:
W20260924 14:30:43.266793 27188 rpcz_store.cc:269] 0924 14:30:39.559617 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.559774 (+   157us) service_pool.cc:227] Handling call
0924 14:30:43.262677 (+3702903us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b54002008d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:43.262812 (+   135us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1887410560}
I20260924 14:30:43.221828 26328 rpcz_store.cc:276] 0924 14:30:41.386697 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.386848 (+   151us) service_pool.cc:227] Handling call
0924 14:30:41.387253 (+   405us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:41.387540 (+   287us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:30:41.956613 (+569073us) log.cc:842] Serialized 14 byte log entry
0924 14:30:43.191300 (+1234687us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:43.191353 (+    53us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:43.191396 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:43.192483 (+  1087us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:43.202712 (+ 10229us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":462516,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":2604,"raft.queue_time_us":1355,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5685,"thread_start_us":1772770,"threads_started":2}
I20260924 14:30:43.262187 30077 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:43.251394 29532 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:43.279316 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.280747 26325 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260924 14:30:43.293663 26323 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:43.296123 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.297086 26323 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
W20260924 14:30:43.300196 26398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1412) took 1244 ms (client timeout 1566 ms). Trace:
W20260924 14:30:43.289438 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.431s	user 0.039s	sys 0.048s
W20260924 14:30:43.300809 26398 rpcz_store.cc:269] 0924 14:30:42.055817 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:42.056071 (+   254us) service_pool.cc:227] Handling call
0924 14:30:43.299930 (+1243859us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b54001e4bd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:43.300121 (+   191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1665587584}
W20260924 14:30:43.301548 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.443s	user 0.039s	sys 0.048s
I20260924 14:30:43.300001 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:43.306094 29886 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:43.306769 29886 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:43.285137 29902 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:43.308764 29902 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:43.311231 30076 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:43.312948 29532 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.286101 30069 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:43.305337 26328 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:43.331758 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.317744 30077 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.328153 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1249) took 2564 ms. Trace:
I20260924 14:30:43.336488 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:43.341542 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:43.341974 26258 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:43.341306 26327 rpcz_store.cc:276] 0924 14:30:40.764046 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.764168 (+   122us) service_pool.cc:227] Handling call
0924 14:30:40.764499 (+   331us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:40.872888 (+108389us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:43.327862 (+2454974us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54005baed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:43.328080 (+   218us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":230066944}
W20260924 14:30:43.344826 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 442602us
I20260924 14:30:43.337929 26323 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:43.349900 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.350785 26256 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:43.353101 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:43.313625 29886 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:43.369333 27185 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:43.370576 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.373483 26325 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:43.371035 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.370802 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 695) took 1957 ms. Trace:
I20260924 14:30:43.376453 29677 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:43.373917 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:43.345682 29925 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:43.392839 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:43.393054 26327 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:43.398770 26256 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:43.390098 29983 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260924 14:30:43.397516 29994 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:43.401942 29994 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:43.404397 26369 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:43.416440 26249 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260924 14:30:43.335048 30076 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:43.418816 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.419404 26328 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:43.368420 30085 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:43.316388 29762 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:43.385376 30101 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:43.427177 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.428282 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.428892 26256 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:43.429910 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.430686 26256 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:43.445043 26369 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:43.403275 29788 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:43.449528 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:43.419565 30047 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:43.455324 26328 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:43.454463 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.453181 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.422380 29970 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:43.458824 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:43.461786 26331 raft_consensus.cc:2354] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9.
I20260924 14:30:43.428694 30101 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:43.493090 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.453706 30054 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:43.487591 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.462816 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:43.461328 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:43.488831 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:43.456508 30047 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:43.472436 29676 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 3.757s	user 0.032s	sys 0.059s
I20260924 14:30:43.424227 29985 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260924 14:30:43.447494 30085 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:43.460696 26256 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:43.445911 29983 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260924 14:30:43.459909 29951 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260924 14:30:43.459782 30077 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:43.535880 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1223) took 2126 ms. Trace:
I20260924 14:30:43.535934 26403 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:43.558458 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 492436us
W20260924 14:30:43.537120 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.011s	sys 0.024s
I20260924 14:30:43.126735 26326 rpcz_store.cc:276] 0924 14:30:41.412784 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.697579 (+284795us) service_pool.cc:227] Handling call
0924 14:30:41.700710 (+  3131us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:41.748012 (+ 47302us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:41.748045 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:41.748562 (+   517us) log.cc:842] Serialized 26 byte log entry
0924 14:30:42.989774 (+1241212us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:43.020939 (+ 31165us) log.cc:842] Serialized 14 byte log entry
0924 14:30:43.021211 (+   272us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:43.021303 (+    92us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:43.125581 (+104278us) raft_consensus.cc:1671] finished
0924 14:30:43.125605 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:43.125736 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":46897,"prepare.queue_time_us":33154,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":56564,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":31918,"spinlock_wait_cycles":64113920,"thread_start_us":1218485,"threads_started":1,"wal-append.queue_time_us":1308458}
W20260924 14:30:43.542500 26298 connection.cc:232] RPC call timeout handler was delayed by -1.1612103570005274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
W20260924 14:30:43.510260 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 376598us
W20260924 14:30:43.615679 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.011s	sys 0.033s
I20260924 14:30:43.510955 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:43.510967 26328 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:43.613775 26405 rpcz_store.cc:276] 0924 14:30:41.409339 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.409551 (+   212us) service_pool.cc:227] Handling call
0924 14:30:41.409951 (+   400us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:41.410219 (+   268us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:30:43.019398 (+1609179us) log.cc:842] Serialized 14 byte log entry
0924 14:30:43.535390 (+515992us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:43.535488 (+    98us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:43.535543 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:43.535685 (+   142us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:43.535817 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":154689,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":16712,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":17626,"thread_start_us":2075719,"threads_started":2,"wal-append.queue_time_us":558116}
W20260924 14:30:43.588992 30100 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260924 14:30:43.604116 26398 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:43.544878 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:43.605023 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.454924 29788 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:43.515228 26249 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:43.606267 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:43.602836 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 141597us
I20260924 14:30:43.615240 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:43.618806 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 601024us
I20260924 14:30:43.516567 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:43.601701 29676 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 3.886s	user 0.032s	sys 0.066s
I20260924 14:30:43.625483 26224 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:43.622915 26250 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:43.624609 26256 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 3.
I20260924 14:30:43.462774 29970 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:43.626919 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:43.589715 29994 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.540270 29996 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:43.646303 30000 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:43.646028 29996 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:43.646819 30000 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:43.648252 29996 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.581041 30069 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:43.650249 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44081.4us
I20260924 14:30:43.651262 29941 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:43.651500 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:43.652298 30084 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:43.652726 30084 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:43.653056 30084 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:43.653755 30109 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:43.653991 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:43.625840 26404 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:43.611337 30047 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:43.511430 30101 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:43.666908 30069 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:43.545004 30048 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:43.672043 27188 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260924 14:30:43.675040 27188 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
W20260924 14:30:43.614265 29942 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 3.898s	user 0.019s	sys 0.016s
I20260924 14:30:43.627954 29788 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:43.633725 29985 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:43.636127 29762 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.636667 29983 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:43.637079 30102 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260924 14:30:43.766314 30076 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:43.770409 30084 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:43.639205 30108 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260924 14:30:43.630642 30054 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:30:43.641187 30077 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:43.643927 30107 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260924 14:30:43.654491 30109 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260924 14:30:43.634920 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 124068us
W20260924 14:30:44.280925 26250 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.002s
W20260924 14:30:44.281100 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 662136us
I20260924 14:30:44.281426 26250 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:44.282680 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1076) took 1283 ms. Trace:
I20260924 14:30:43.656157 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:43.680522 27194 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:43.654771 29677 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:43.614603 29886 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:44.327584 29886 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:44.328605 29886 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:43.659402 29941 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:43.656059 30000 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:43.744499 29942 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 4.028s	user 0.019s	sys 0.017s
I20260924 14:30:43.628334 29970 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:43.770817 30085 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:43.637929 30098 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260924 14:30:43.744290 30099 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:44.372347 30099 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:43.783488 30101 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:43.793176 30111 raft_consensus.cc:3032] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
W20260924 14:30:43.800998 26330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 689) took 2785 ms (client timeout 3276 ms). Trace:
W20260924 14:30:43.865481 27181 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 644) took 4159 ms (client timeout 2972 ms). Trace:
I20260924 14:30:43.955224 30084 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:44.374543 26330 rpcz_store.cc:269] 0924 14:30:41.015256 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.015480 (+   224us) service_pool.cc:227] Handling call
0924 14:30:43.795317 (+2779837us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b540012e154. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:43.800916 (+  5599us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1708164352}
I20260924 14:30:43.778424 30054 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:44.372915 30099 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:43.788108 30077 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:44.383615 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:44.384574 26330 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:44.027158 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.385903 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:44.386638 26249 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:44.035252 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.387588 27191 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:44.388626 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.388881 30101 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:44.045904 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1407) took 2028 ms. Trace:
I20260924 14:30:44.389950 26402 rpcz_store.cc:276] 0924 14:30:42.017584 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:42.017733 (+   149us) service_pool.cc:227] Handling call
0924 14:30:42.021243 (+  3510us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:42.117693 (+ 96450us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:42.117728 (+    35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:42.117773 (+    45us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:42.117804 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.045583 (+1927779us) spinlock_profiling.cc:243] Waited 96.2 ms on lock 0x7b540023f854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:44.045646 (+    63us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:44.045828 (+   182us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":221322624}
I20260924 14:30:44.047739 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1398) took 4901 ms. Trace:
I20260924 14:30:44.054205 30025 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:30:44.077365 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:43.744311 29985 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:44.084481 30047 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:44.099475 27190 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:44.405007 30047 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:44.405864 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1072) took 1416 ms. Trace:
I20260924 14:30:44.406282 27190 rpcz_store.cc:276] 0924 14:30:42.989550 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:43.137584 (+148034us) service_pool.cc:227] Handling call
0924 14:30:44.099186 (+961602us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:44.403751 (+304565us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.405667 (+  1916us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b540073c650. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:44.405803 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2164901248}
W20260924 14:30:44.116832 27182 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.009s	sys 0.002s
W20260924 14:30:44.407648 27182 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.010s	sys 0.002s
I20260924 14:30:44.408417 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1075) took 1409 ms. Trace:
I20260924 14:30:44.408759 27182 rpcz_store.cc:276] 0924 14:30:42.998768 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:43.134254 (+135486us) service_pool.cc:227] Handling call
0924 14:30:43.134563 (+   309us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:43.134984 (+   421us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:30:43.613041 (+478057us) log.cc:842] Serialized 14 byte log entry
0924 14:30:44.408056 (+795015us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:44.408113 (+    57us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:44.408158 (+    45us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.408236 (+    78us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:44.408367 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":636685,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":1260946,"threads_started":2,"wal-append.queue_time_us":182518}
I20260924 14:30:44.410437 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:39.531488 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:44.412003 29985 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:44.413084 29985 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:44.135926 26079 catalog_manager.cc:5495] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 9, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 9 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
W20260924 14:30:44.138269 26077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 140) took 18773 ms (client timeout 15000 ms). Trace:
I20260924 14:30:44.389813 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:44.151473 26221 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:49618 dropped due to backpressure. The service queue is full; it has 50 items.
I20260924 14:30:44.151823 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.152923 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.158942 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:44.160189 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:44.160380 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:44.177908 29629 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:44.186793 26254 raft_consensus.cc:2396] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260924 14:30:44.202719 29902 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:44.203722 30076 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:44.194856 26258 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.852s	user 0.012s	sys 0.003s
I20260924 14:30:44.208424 29788 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:44.217275 26257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 907) took 4556 ms (client timeout 2614 ms). Trace:
I20260924 14:30:44.219324 29955 raft_consensus.cc:3032] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Stepping down as leader of term 3
I20260924 14:30:44.222681 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1163) took 4256 ms. Trace:
I20260924 14:30:44.442008 26252 rpcz_store.cc:276] 0924 14:30:39.965997 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.913654 (+947657us) service_pool.cc:227] Handling call
0924 14:30:40.913988 (+   334us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:41.448321 (+534333us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:30:42.925918 (+1477597us) log.cc:842] Serialized 14 byte log entry
0924 14:30:43.342665 (+416747us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:43.342719 (+    54us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:43.342774 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.221129 (+878355us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b18000865c0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9
0924 14:30:44.222393 (+  1264us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54006d2ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:44.222442 (+    49us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:44.222609 (+   167us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1303,"prepare.queue_time_us":91910,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":1421,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":5177,"spinlock_wait_cycles":1234846976,"thread_start_us":1869533,"threads_started":2,"wal-append.queue_time_us":565872}
I20260924 14:30:44.225855 30055 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:44.441210 26257 rpcz_store.cc:269] 0924 14:30:39.661190 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.737127 (+ 75937us) service_pool.cc:227] Handling call
0924 14:30:44.216910 (+4479783us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54000c17d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:44.217112 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":493226752}
I20260924 14:30:44.228328 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1168) took 3681 ms. Trace:
I20260924 14:30:44.447359 26251 rpcz_store.cc:276] 0924 14:30:40.547233 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.594969 (+1047736us) service_pool.cc:227] Handling call
0924 14:30:42.824996 (+1230027us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:42.837547 (+ 12551us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.228135 (+1390588us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b54002a1a50. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:44.228274 (+   139us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1551800576}
I20260924 14:30:44.230917 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1230) took 1151 ms. Trace:
I20260924 14:30:44.448822 26401 rpcz_store.cc:276] 0924 14:30:43.079629 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:43.124612 (+ 44983us) service_pool.cc:227] Handling call
0924 14:30:44.230672 (+1106060us) spinlock_profiling.cc:243] Waited 60.5 ms on lock 0x7b54003d59d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:44.230780 (+   108us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":139109120}
I20260924 14:30:44.449674 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 916) took 1077 ms. Trace:
W20260924 14:30:44.234521 30073 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.267s	user 0.012s	sys 0.004s
I20260924 14:30:44.240484 29777 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:44.283231 27186 rpcz_store.cc:276] 0924 14:30:42.999111 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:43.135945 (+136834us) service_pool.cc:227] Handling call
0924 14:30:43.136325 (+   380us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:43.136770 (+   445us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:43.136797 (+    27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:43.141173 (+  4376us) log.cc:842] Serialized 26 byte log entry
0924 14:30:44.197864 (+1056691us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b180031f6c0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:30:44.264337 (+ 66473us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:44.264378 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.264446 (+    68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:44.282330 (+ 17884us) raft_consensus.cc:1671] finished
0924 14:30:44.282357 (+    27us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:44.282605 (+   248us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20842,"raft.queue_time_us":1246,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3739,"spinlock_wait_cycles":81998720,"thread_start_us":66266,"threads_started":1,"wal-append.queue_time_us":1120172}
I20260924 14:30:44.285475 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:44.286965 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 630850us
I20260924 14:30:44.456080 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.458403 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.292070 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1061) took 1389 ms. Trace:
I20260924 14:30:44.459628 27185 rpcz_store.cc:276] 0924 14:30:42.903178 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:42.929834 (+ 26656us) service_pool.cc:227] Handling call
0924 14:30:42.966507 (+ 36673us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:43.372353 (+405846us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.291829 (+919476us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b540000ee54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:44.292015 (+   186us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":925917056}
I20260924 14:30:44.293448 30033 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:30:44.298760 29925 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:44.302132 29994 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:44.304535 26255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1177) took 2375 ms (client timeout 1966 ms). Trace:
W20260924 14:30:44.465087 26255 rpcz_store.cc:269] 0924 14:30:41.928642 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.997153 (+ 68511us) service_pool.cc:227] Handling call
0924 14:30:44.304356 (+2307203us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540015ead4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:44.304461 (+   105us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2641486080}
I20260924 14:30:44.465499 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 927) took 1007 ms. Trace:
I20260924 14:30:44.305668 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:44.306617 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:44.466668 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 160101us
I20260924 14:30:44.467018 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.468201 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.306850 29952 consensus_queue.cc:1046] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:30:44.309623 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:44.439172 29902 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:44.313616 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1174) took 2385 ms. Trace:
I20260924 14:30:44.320688 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1272) took 1191 ms. Trace:
I20260924 14:30:44.343541 29830 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:44.357486 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 639) took 4693 ms. Trace:
I20260924 14:30:44.363354 30085 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:44.354019 29970 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:44.368664 30048 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:44.369264 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:43.376726 26324 rpcz_store.cc:276] 0924 14:30:41.413184 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.708259 (+295075us) service_pool.cc:227] Handling call
0924 14:30:41.708640 (+   381us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:41.709408 (+   768us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:30:42.928104 (+1218696us) log.cc:842] Serialized 14 byte log entry
0924 14:30:43.370225 (+442121us) log.cc:842] Serialized 14 byte log entry
0924 14:30:43.370454 (+   229us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:43.370505 (+    51us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:43.370557 (+    52us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:43.370625 (+    68us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:43.370740 (+   115us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":263811,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":1868,"raft.queue_time_us":1710,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":4860,"thread_start_us":1637320,"threads_started":2}
I20260924 14:30:44.480010 27194 rpcz_store.cc:276] 0924 14:30:39.663770 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.664165 (+   395us) service_pool.cc:227] Handling call
0924 14:30:39.700135 (+ 35970us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:44.288781 (+4588646us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.357177 (+ 68396us) spinlock_profiling.cc:243] Waited 3.98 s on lock 0x7b54003b79d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:44.357379 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9153246080}
I20260924 14:30:44.490211 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.491173 27185 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:44.462764 30109 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:44.392033 26397 rpcz_store.cc:276] 0924 14:30:39.146796 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.154675 (+  7879us) service_pool.cc:227] Handling call
0924 14:30:39.171167 (+ 16492us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:39.625930 (+454763us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.047475 (+4421545us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b5400336654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:44.047667 (+   192us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1024699776}
I20260924 14:30:44.392422 26256 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 4.
I20260924 14:30:44.389787 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.371758 26223 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:55802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1356)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1352)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1355)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1190)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1348)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1365)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1353)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1192)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 935)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1369)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1359)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1349)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1358)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1351)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1357)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 917)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1185)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1186)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1187)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1189)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1350)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1354)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1360)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1363)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1367)
I20260924 14:30:44.393363 30025 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:44.401300 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 772326us
I20260924 14:30:44.403219 26330 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:44.375003 30111 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:44.417573 27189 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:44.500631 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.501624 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.501816 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.502252 26251 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:44.502965 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.503182 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:44.415023 26077 rpcz_store.cc:269] 0924 14:30:25.364989 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:25.365333 (+   344us) service_pool.cc:227] Handling call
0924 14:30:44.138040 (+18772707us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:30:33.751763 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:30:33.766708 (+ 14945us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:30:33.766751 (+    43us) write_op.cc:435] Acquired schema lock
0924 14:30:33.766764 (+    13us) tablet.cc:660] Decoding operations
0924 14:30:33.768573 (+  1809us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:30:33.768639 (+    66us) write_op.cc:641] Partition lock acquired for write op
0924 14:30:33.768662 (+    23us) tablet.cc:683] Acquiring locks for 21 operations
0924 14:30:33.770160 (+  1498us) tablet.cc:699] Row locks acquired
0924 14:30:33.770183 (+    23us) write_op.cc:260] PREPARE: finished
0924 14:30:33.770344 (+   161us) write_op.cc:270] Start()
0924 14:30:33.770473 (+   129us) write_op.cc:276] Timestamp: P: 1790260233770289 usec, L: 0
0924 14:30:33.770496 (+    23us) op_driver.cc:331] REPLICATION: starting
0924 14:30:33.887627 (+117131us) log.cc:842] Serialized 8364 byte log entry
0924 14:30:34.363251 (+475624us) op_driver.cc:444] REPLICATION: finished
0924 14:30:34.771234 (+407983us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b18000d4e00. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426764 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:30:35.079661 (+308427us) write_op.cc:301] APPLY: starting
0924 14:30:35.095066 (+ 15405us) tablet.cc:1365] starting BulkCheckPresence
0924 14:30:35.187627 (+ 92561us) tablet.cc:1368] finished BulkCheckPresence
0924 14:30:35.194314 (+  6687us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:30:35.257728 (+ 63414us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:30:35.258035 (+   307us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=21
0924 14:30:35.258081 (+    46us) write_op.cc:312] APPLY: finished
0924 14:30:35.258511 (+   430us) log.cc:842] Serialized 144 byte log entry
0924 14:30:35.409059 (+150548us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:35.430254 (+ 21195us) write_op.cc:454] Released schema lock
0924 14:30:35.430419 (+   165us) write_op.cc:341] FINISH: Updating metrics
0924 14:30:35.431047 (+   628us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:35.431089 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5494862,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":539813,"apply.run_cpu_time_us":10250,"apply.run_wall_time_us":369580,"num_ops":21,"prepare.queue_time_us":322005,"prepare.run_cpu_time_us":17385,"prepare.run_wall_time_us":302538,"raft.queue_time_us":188926,"raft.run_cpu_time_us":19275,"raft.run_wall_time_us":599423,"replication_time_us":552069,"spinlock_wait_cycles":28373120,"thread_start_us":864705,"threads_started":5,"wal-append.queue_time_us":188617,"wal-append.run_cpu_time_us":24500,"wal-append.run_wall_time_us":2219944}]]}
I20260924 14:30:44.505831 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:44.383623 30054 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:44.508596 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:44.418483 26326 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:44.435259 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.436486 29629 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:44.437127 26325 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260924 14:30:44.514810 27188 raft_consensus.cc:2396] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260924 14:30:44.435746 30084 raft_consensus.cc:2741] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260924 14:30:44.437681 30129 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260924 14:30:44.511555 29902 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:44.516753 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:44.440632 26258 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
W20260924 14:30:44.232789 27192 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.008s	sys 0.008s
W20260924 14:30:44.518575 27192 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.009s	sys 0.009s
I20260924 14:30:44.450129 26254 rpcz_store.cc:276] 0924 14:30:43.372581 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:43.392294 (+ 19713us) service_pool.cc:227] Handling call
0924 14:30:44.449477 (+1057183us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b5400716954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:44.449617 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1755508608}
I20260924 14:30:44.453022 26079 catalog_manager.cc:5495] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 7, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 7 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:44.492100 30048 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:44.446949 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.523787 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:44.465929 26250 rpcz_store.cc:276] 0924 14:30:43.458037 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.285174 (+827137us) service_pool.cc:227] Handling call
0924 14:30:44.465436 (+180262us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:30:44.311942 26329 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.023s	user 0.014s	sys 0.042s
W20260924 14:30:44.524678 26329 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.236s	user 0.014s	sys 0.042s
I20260924 14:30:44.525482 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1263) took 1599 ms. Trace:
W20260924 14:30:44.453205 30073 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.488s	user 0.012s	sys 0.004s
I20260924 14:30:44.440423 29886 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:44.470949 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:44.470955 30055 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260924 14:30:44.528844 26323 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:44.532567 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:44.533310 26405 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:44.475540 26331 rpcz_store.cc:276] 0924 14:30:43.130510 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:43.142031 (+ 11521us) service_pool.cc:227] Handling call
0924 14:30:44.320458 (+1178427us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b5400505854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:44.320619 (+   161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":617092864}
I20260924 14:30:44.477583 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:44.507761 29941 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:44.478972 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:30:44.536600 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 57720.8us
I20260924 14:30:44.483384 26401 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:44.374606 27181 rpcz_store.cc:269] 0924 14:30:39.706490 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:39.713903 (+  7413us) service_pool.cc:227] Handling call
0924 14:30:43.865171 (+4151268us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540073c654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:43.865383 (+   212us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":423542528}
I20260924 14:30:44.472409 29925 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
W20260924 14:30:44.495266 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 126101us
I20260924 14:30:44.540190 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.462463 29682 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:44.498086 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 919) took 1102 ms. Trace:
I20260924 14:30:44.498375 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.415946 26221 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:49618 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1356)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1352)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1355)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1190)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1348)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1365)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1353)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1192)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 935)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1369)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1359)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1349)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1358)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1351)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1357)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 917)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1185)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1186)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1187)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1189)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1350)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1354)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1360)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1363)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1367)
I20260924 14:30:44.499771 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.464685 30033 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:44.475335 30047 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:44.475870 29994 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:44.504014 26251 raft_consensus.cc:2362] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11. Re-sending same reply.
I20260924 14:30:44.546882 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.542186 26255 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:44.424098 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.511056 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:44.504326 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.511866 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:44.451557 30076 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:44.453945 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:44.519579 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.518595 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 910) took 1215 ms. Trace:
I20260924 14:30:44.552109 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.551916 26258 rpcz_store.cc:276] 0924 14:30:43.303657 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:43.336124 (+ 32467us) service_pool.cc:227] Handling call
0924 14:30:44.518536 (+1182412us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:44.523015 26326 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:44.516352 29902 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:44.476311 29762 consensus_queue.cc:1046] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260924 14:30:44.526232 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:44.528772 30054 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:44.446353 29955 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:44.497073 30109 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:44.524585 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:44.573904 26397 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:44.529464 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.525153 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.475095 26253 rpcz_store.cc:276] 0924 14:30:41.927769 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:41.927909 (+   140us) service_pool.cc:227] Handling call
0924 14:30:41.951045 (+ 23136us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:42.923528 (+972483us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.313359 (+1389831us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b540002ba54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:44.313553 (+   194us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2177840896}
I20260924 14:30:44.577059 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.541568 26256 rpcz_store.cc:276] 0924 14:30:43.395465 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.077039 (+681574us) service_pool.cc:227] Handling call
0924 14:30:44.497913 (+420874us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:44.577821 26253 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:44.578264 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.540179 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:44.545828 26405 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:44.545879 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:44.532666 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 93839.9us
I20260924 14:30:44.547801 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 925) took 1114 ms. Trace:
I20260924 14:30:44.579870 26404 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:44.580073 26255 rpcz_store.cc:276] 0924 14:30:43.433612 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.498093 (+1064481us) service_pool.cc:227] Handling call
0924 14:30:44.547738 (+ 49645us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:44.581460 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.582015 26255 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:44.551029 26400 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:44.484915 29830 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:44.584712 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 97053.6us
I20260924 14:30:44.586433 29830 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:44.587178 26249 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:44.573673 30109 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:44.558112 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:44.560483 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:44.579392 26256 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:44.584656 26323 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:44.589371 26250 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:44.561520 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.594693 26258 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:44.595554 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.561985 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.566928 29442 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:44.567699 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.524456 30026 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260924 14:30:44.568377 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.540268 30039 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260924 14:30:44.604039 26257 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:44.567857 26079 catalog_manager.cc:5495] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 3 to 6, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 6 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:44.548409 30128 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:44.614147 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.549432 30033 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:30:44.570150 29629 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:44.570345 27182 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260924 14:30:44.565339 29185 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:44.572201 29762 raft_consensus.cc:3032] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:44.565891 29989 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:44.540571 30084 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:44.550333 29952 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:44.547760 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:44.630982 26079 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 7, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 7 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:44.528893 29966 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260924 14:30:44.557693 26369 leader_election.cc:398] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9.
I20260924 14:30:44.636982 26369 leader_election.cc:401] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:30:44.589401 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.525928 26329 rpcz_store.cc:276] 0924 14:30:42.926514 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:43.263649 (+337135us) service_pool.cc:227] Handling call
0924 14:30:43.274935 (+ 11286us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:43.278528 (+  3593us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:43.278556 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:43.278945 (+   389us) log.cc:842] Serialized 26 byte log entry
0924 14:30:44.525168 (+1246223us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:44.525208 (+    40us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.525272 (+    64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:44.525316 (+    44us) raft_consensus.cc:1671] finished
0924 14:30:44.525325 (+     9us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:44.525431 (+   106us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":5184,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":773,"thread_start_us":1235909,"threads_started":1,"wal-append.queue_time_us":502364}
I20260924 14:30:44.593832 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.485203 30085 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:44.595099 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 655) took 1715 ms. Trace:
I20260924 14:30:44.601696 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:44.561528 30047 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:44.613131 29970 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:44.558769 30025 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:44.616668 27185 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:44.649883 27185 raft_consensus.cc:595] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:44.536827 29886 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:44.620419 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.667730 26250 raft_consensus.cc:2362] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9. Re-sending same reply.
I20260924 14:30:44.584825 29941 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:44.572207 29994 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:44.630426 26255 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:44.615839 29790 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:44.638689 26256 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:44.675129 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.675782 26256 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:44.676100 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.676612 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.613447 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.677284 26256 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:44.677569 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 929) took 1155 ms. Trace:
I20260924 14:30:44.677981 26251 rpcz_store.cc:276] 0924 14:30:43.522415 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.604035 (+1081620us) service_pool.cc:227] Handling call
0924 14:30:44.677507 (+ 73472us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:44.678079 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:44.644316 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 86786.6us
I20260924 14:30:44.627354 30026 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:30:44.682643 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.682950 30026 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:44.640403 27192 rpcz_store.cc:276] 0924 14:30:42.880102 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:42.880233 (+   131us) service_pool.cc:227] Handling call
0924 14:30:42.890393 (+ 10160us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:43.344235 (+453842us) raft_consensus.cc:1520] Early marking committed up to index 4
0924 14:30:43.612801 (+268566us) log.cc:842] Serialized 14 byte log entry
0924 14:30:44.540901 (+928100us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:44.548110 (+  7209us) log.cc:842] Serialized 26 byte log entry
0924 14:30:44.548286 (+   176us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:44.548318 (+    32us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.548388 (+    70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:44.594909 (+ 46521us) raft_consensus.cc:1671] finished
0924 14:30:44.594931 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:44.595047 (+   116us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":21878,"prepare.queue_time_us":427684,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":93527,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":11581,"thread_start_us":1577161,"threads_started":3,"wal-append.queue_time_us":182720}
W20260924 14:30:44.668280 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 66679.6us
I20260924 14:30:44.641700 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:44.625469 29762 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:44.676710 26250 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:44.699975 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.679346 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.691354 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.701720 26258 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:44.698414 26257 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:44.631210 29952 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:44.640198 30077 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 9
I20260924 14:30:44.575732 30111 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:44.628686 29814 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260924 14:30:44.675792 26079 catalog_manager.cc:5495] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:44.697393 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:44.660190 27126 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.701305 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.667013 30073 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:44.723588 26397 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:44.704007 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1190) took 1166 ms. Trace:
I20260924 14:30:44.724972 26257 rpcz_store.cc:276] 0924 14:30:43.538098 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.547480 (+1009382us) service_pool.cc:227] Handling call
0924 14:30:44.703944 (+156464us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:30:44.725584 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65540.9us
I20260924 14:30:44.701431 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:44.725672 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.726389 26405 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:44.704586 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.726836 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.706179 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.707392 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.728044 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.707548 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.702685 26256 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:44.729210 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.729502 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.730259 26256 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:44.730959 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:44.697688 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 56101.9us
W20260924 14:30:44.715054 26371 leader_election.cc:334] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.584s (SENT)
I20260924 14:30:44.672003 29999 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:44.629820 29955 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:44.737887 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 928) took 1247 ms. Trace:
I20260924 14:30:44.720884 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1240) took 1230 ms. Trace:
I20260924 14:30:44.738436 26258 rpcz_store.cc:276] 0924 14:30:43.490252 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.595289 (+1105037us) service_pool.cc:227] Handling call
0924 14:30:44.737610 (+142321us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54003827d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:44.737812 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":242313088}
I20260924 14:30:44.739025 26400 rpcz_store.cc:276] 0924 14:30:43.490807 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:43.519743 (+ 28936us) service_pool.cc:227] Handling call
0924 14:30:43.562269 (+ 42526us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:44.589965 (+1027696us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.720572 (+130607us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b54001e4bd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:44.720807 (+   235us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2273364864}
I20260924 14:30:44.740106 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.741117 26258 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:44.722261 26079 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 8, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 8 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:44.731819 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:44.732988 26256 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:44.703051 29970 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:44.726568 26251 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:44.626374 30128 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:44.742295 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 921) took 1337 ms. Trace:
I20260924 14:30:44.728266 26253 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:44.714928 29830 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:44.636021 29989 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:44.743283 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.744800 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:44.769418 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:44.745517 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:44.770741 26328 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:44.772056 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:44.772390 26298 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:44.772889 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:44.747087 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:44.736420 26255 raft_consensus.cc:2396] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260924 14:30:44.751219 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:44.773686 26324 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:44.753494 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.774717 26258 raft_consensus.cc:2354] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9.
I20260924 14:30:44.724781 29682 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:44.781141 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:44.782178 26403 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:44.746507 29999 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:44.666895 29716 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260924 14:30:44.752234 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:44.714457 29952 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:44.759779 26079 catalog_manager.cc:5495] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 4 to 6, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 6 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:44.636341 30084 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:44.762518 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.760449 26256 rpcz_store.cc:276] 0924 14:30:43.404890 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.730735 (+1325845us) service_pool.cc:227] Handling call
0924 14:30:44.742216 (+ 11481us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:44.765479 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:44.770057 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:44.734212 29185 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:44.777668 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.736770 29994 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:30:44.779191 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.735581 29983 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:30:44.775760 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.820235 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.820560 26258 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:44.821192 26253 raft_consensus.cc:2396] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:44.821437 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.738791 26250 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:44.822257 26327 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:44.782312 30111 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:44.755880 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:44.759709 30055 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:44.749435 29834 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:44.743263 30073 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:44.735288 30099 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:44.795397 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:44.794320 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:44.846469 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 52249.1us
I20260924 14:30:44.763363 29830 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:44.809514 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.771160 29998 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:44.766696 29790 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:44.769263 29629 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:44.813191 26329 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:30:44.814198 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 87424us
I20260924 14:30:44.814265 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.679966 29941 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:44.826563 26255 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:30:44.784108 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:44.790432 26112 debug-util.cc:398] Leaking SignalData structure 0x7b0800053c40 after lost signal to thread 23138
I20260924 14:30:44.827834 30071 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:44.836515 26258 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:44.838742 26250 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:44.831802 29788 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:44.787060 29970 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:44.804291 29999 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:44.805416 29914 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:44.867908 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1187) took 1418 ms. Trace:
I20260924 14:30:44.881711 26251 rpcz_store.cc:276] 0924 14:30:43.449643 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.839775 (+1390132us) service_pool.cc:227] Handling call
0924 14:30:44.867384 (+ 27609us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:44.867604 (+   220us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:44.867630 (+    26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:44.867673 (+    43us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:44.867704 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.867756 (+    52us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:44.867857 (+   101us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:44.882978 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.868335 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1185) took 1419 ms. Trace:
I20260924 14:30:44.884460 26255 rpcz_store.cc:276] 0924 14:30:43.449112 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.825865 (+1376753us) service_pool.cc:227] Handling call
0924 14:30:44.826144 (+   279us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:44.868081 (+ 41937us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.868266 (+   185us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:44.885102 29886 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:44.885529 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.886199 26255 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:44.838857 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1186) took 1389 ms. Trace:
I20260924 14:30:44.870086 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 917) took 1478 ms. Trace:
I20260924 14:30:44.887668 26253 rpcz_store.cc:276] 0924 14:30:43.449383 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.827661 (+1378278us) service_pool.cc:227] Handling call
0924 14:30:44.838309 (+ 10648us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:44.838515 (+   206us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:44.838540 (+    25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:44.838582 (+    42us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:44.838621 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.838676 (+    55us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:44.838796 (+   120us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:44.888069 26250 rpcz_store.cc:276] 0924 14:30:43.391234 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.822681 (+1431447us) service_pool.cc:227] Handling call
0924 14:30:44.823084 (+   403us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:44.869775 (+ 46691us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:44.870019 (+   244us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:30:44.889320 30099 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:44.889475 26253 pending_rounds.cc:76] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 4
I20260924 14:30:44.889783 26253 pending_rounds.cc:97] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260924 14:30:44.890007 26253 raft_consensus.cc:2866] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:30:44.796514 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 93139.6us
I20260924 14:30:44.767400 30046 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260924 14:30:44.871227 26254 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:44.854869 26256 raft_consensus.cc:2354] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
I20260924 14:30:44.858316 26328 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:44.806490 30076 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:44.758872 30081 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260924 14:30:44.796095 29682 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:44.854858 29830 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:44.864557 26257 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:44.765952 30047 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:44.921970 29814 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:44.922494 29814 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:44.923467 29814 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:44.924474 29682 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:44.925009 29814 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:44.776930 29762 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:44.926312 29762 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:44.929551 29830 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:44.933810 29442 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:44.885710 29886 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:44.933790 29830 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:44.934670 29886 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:44.850240 27192 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:44.938293 27192 raft_consensus.cc:595] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:44.938776 27192 raft_consensus.cc:603] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:44.763151 30026 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:44.869451 26112 debug-util.cc:398] Leaking SignalData structure 0x7b08002c9120 after lost signal to thread 26115
I20260924 14:30:44.865999 26398 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260924 14:30:44.945106 26112 debug-util.cc:398] Leaking SignalData structure 0x7b080013db60 after lost signal to thread 26282
I20260924 14:30:44.844079 26079 catalog_manager.cc:5495] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 6 to 7, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 7 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:44.857002 29994 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:44.906458 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.954967 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:44.955788 26256 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:44.871609 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.919628 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:44.920194 26324 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:44.920490 30134 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:44.920605 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:44.921180 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:44.934340 29442 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:44.894876 26251 raft_consensus.cc:2362] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10. Re-sending same reply.
W20260924 14:30:44.946723 26112 debug-util.cc:398] Leaking SignalData structure 0x7b080038fdc0 after lost signal to thread 26356
I20260924 14:30:44.949417 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:44.935163 29886 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:44.855199 29998 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:44.866919 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 114784us
W20260924 14:30:44.965883 26112 debug-util.cc:398] Leaking SignalData structure 0x7b0800379800 after lost signal to thread 26430
W20260924 14:30:44.992511 26112 debug-util.cc:398] Leaking SignalData structure 0x7b08001421e0 after lost signal to thread 27222
I20260924 14:30:44.974951 26258 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
W20260924 14:30:44.993855 26112 debug-util.cc:398] Leaking SignalData structure 0x7b0800294a00 after lost signal to thread 29921
W20260924 14:30:44.994266 26112 debug-util.cc:398] Leaking SignalData structure 0x7b08002d9fa0 after lost signal to thread 29933
I20260924 14:30:44.994819 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:44.995286 26112 debug-util.cc:398] Leaking SignalData structure 0x7b08003a9540 after lost signal to thread 30043
I20260924 14:30:44.995666 26258 raft_consensus.cc:2362] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11. Re-sending same reply.
W20260924 14:30:44.995980 26112 debug-util.cc:398] Leaking SignalData structure 0x7b08000d67a0 after lost signal to thread 30068
W20260924 14:30:44.996448 26112 debug-util.cc:398] Leaking SignalData structure 0x7b08000cd060 after lost signal to thread 30080
W20260924 14:30:44.996745 26112 debug-util.cc:398] Leaking SignalData structure 0x7b0800120d20 after lost signal to thread 30083
W20260924 14:30:44.997478 26112 debug-util.cc:398] Leaking SignalData structure 0x7b08003cc100 after lost signal to thread 30092
I20260924 14:30:44.976372 26327 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:45.024874 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 933) took 1394 ms. Trace:
I20260924 14:30:45.025357 26254 rpcz_store.cc:276] 0924 14:30:43.630165 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.529543 (+899378us) service_pool.cc:227] Handling call
0924 14:30:45.024627 (+495084us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54004538d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:45.024802 (+   175us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":736797056}
W20260924 14:30:44.963618 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 256161us
I20260924 14:30:44.961506 30085 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:44.960464 30134 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:45.049515 26256 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
W20260924 14:30:45.087646 23140 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9b80 after lost signal to thread 26298
I20260924 14:30:45.094920 26257 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260924 14:30:45.095327 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:45.096405 26256 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:30:45.100723 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "986260198b71418c831bfc4617da4603"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:45.101670 27188 raft_consensus.cc:595] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
W20260924 14:30:45.007825 29777 consensus_meta.cc:287] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.003s
I20260924 14:30:44.967065 30025 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:45.086313 26249 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:30:45.105967 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:45.117583 26254 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:44.986829 29989 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:45.109874 26079 catalog_manager.cc:5495] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 6 to 7, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 7 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:45.111028 26327 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:45.007370 29998 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:45.118779 26332 raft_consensus.cc:2396] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260924 14:30:44.983479 29994 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:45.121667 27125 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:30:45.091952 29985 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.678s	user 0.010s	sys 0.004s
I20260924 14:30:45.122834 26369 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:45.047518 30069 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:30:45.154009 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:45.097908 29788 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:44.986598 29716 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.141s
I20260924 14:30:45.157876 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:45.165499 26371 leader_election.cc:334] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.166s (SENT)
I20260924 14:30:45.165688 26254 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:45.007804 29966 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260924 14:30:45.163076 27125 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:45.172647 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1364) took 1023 ms. Trace:
I20260924 14:30:45.175595 26256 rpcz_store.cc:276] 0924 14:30:44.149705 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:45.097441 (+947736us) service_pool.cc:227] Handling call
0924 14:30:45.171380 (+ 73939us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:45.172197 (+   817us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:45.172237 (+    40us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:45.172306 (+    69us) raft_consensus.cc:1631] Marking committed up to 2
0924 14:30:45.172352 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:45.172426 (+    74us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:45.172556 (+   130us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:30:45.178592 26297 leader_election.cc:334] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.566s (SENT)
I20260924 14:30:45.179353 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:45.140209 29920 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:45.202576 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:45.204399 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:45.207800 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:45.208861 26254 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:45.209844 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 338883us
I20260924 14:30:45.212587 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:45.169049 29989 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:45.213997 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:45.211081 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:45.171721 30145 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:45.120887 29830 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260924 14:30:45.177035 29983 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:45.227461 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.247764 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:45.248005 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.250216 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:45.254829 26254 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:45.256124 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.190972 30084 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:45.214522 30076 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:45.256954 27192 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:45.266553 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.262236 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:45.188757 29920 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:45.227975 30151 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:45.244484 29981 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:45.185961 29788 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:45.260836 30071 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:45.263968 30146 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260924 14:30:45.326668 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1188) took 1876 ms. Trace:
W20260924 14:30:45.327909 30143 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260924 14:30:45.327280 26255 rpcz_store.cc:276] 0924 14:30:43.449914 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.886791 (+1436877us) service_pool.cc:227] Handling call
0924 14:30:44.887166 (+   375us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:44.895412 (+  8246us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:44.895471 (+    59us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:44.896023 (+   552us) log.cc:842] Serialized 26 byte log entry
0924 14:30:45.222037 (+326014us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:45.222094 (+    57us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:45.222194 (+   100us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:45.326372 (+104178us) raft_consensus.cc:1671] finished
0924 14:30:45.326404 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:45.326592 (+   188us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":7936,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":14411,"thread_start_us":325636,"threads_started":1,"wal-append.queue_time_us":428691}
W20260924 14:30:45.328998 30150 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260924 14:30:45.333057 30156 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:45.333118 26255 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:45.335436 26255 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:45.337512 26255 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:45.341600 29788 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:45.343950 26255 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:45.345816 29185 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:45.348839 29185 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:45.349995 29185 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:45.350401 29682 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260924 14:30:45.346758 26249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 923) took 1927 ms (client timeout 2241 ms). Trace:
W20260924 14:30:45.352246 26249 rpcz_store.cc:269] 0924 14:30:43.419381 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.424520 (+1005139us) service_pool.cc:227] Handling call
0924 14:30:45.346566 (+922046us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b54000c26d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:45.346700 (+   134us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":144614144}
I20260924 14:30:45.352962 30055 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:45.353598 30055 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:45.354266 26249 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260924 14:30:45.182456 30134 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:45.300877 29845 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:45.356160 29845 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:45.356673 29845 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:45.351356 29185 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:45.361830 30161 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:45.362143 29920 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:45.362394 30161 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:45.368336 26249 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:30:45.351758 29682 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260924 14:30:45.371237 29682 raft_consensus.cc:3032] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:45.371857 29682 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:45.373601 29682 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:45.375239 29682 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:45.377688 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1191) took 1773 ms. Trace:
I20260924 14:30:45.377202 30071 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:45.378142 26250 rpcz_store.cc:276] 0924 14:30:43.603822 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.889197 (+1285375us) service_pool.cc:227] Handling call
0924 14:30:44.889497 (+   300us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:44.889703 (+   206us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:44.889726 (+    23us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:44.890121 (+   395us) log.cc:842] Serialized 26 byte log entry
0924 14:30:45.247206 (+357085us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:45.247251 (+    45us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:45.247340 (+    89us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:45.377442 (+130102us) raft_consensus.cc:1671] finished
0924 14:30:45.377464 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:45.377626 (+   162us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":356693,"threads_started":1,"wal-append.queue_time_us":426588}
I20260924 14:30:45.268810 30073 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:45.399928 26251 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260924 14:30:45.401475 26255 pending_rounds.cc:76] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 4
I20260924 14:30:45.401885 26255 pending_rounds.cc:97] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260924 14:30:45.402711 26255 raft_consensus.cc:2866] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:45.358703 29999 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:45.311173 30076 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:30:45.410475 26371 leader_election.cc:334] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.978s (SENT)
I20260924 14:30:45.413755 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:45.416330 26079 catalog_manager.cc:5495] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
W20260924 14:30:45.421343 26332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 869) took 5290 ms (client timeout 2972 ms). Trace:
I20260924 14:30:45.421463 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:45.421965 26332 rpcz_store.cc:269] 0924 14:30:40.124029 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:40.125946 (+  1917us) service_pool.cc:227] Handling call
0924 14:30:45.410067 (+5284121us) spinlock_profiling.cc:243] Waited 4.99 s on lock 0x7b54005bb8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:45.414292 (+  4225us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":11478947200}
I20260924 14:30:45.418418 26079 catalog_manager.cc:5495] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 4 to 5, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 5 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
W20260924 14:30:45.424536 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 212050us
I20260924 14:30:45.381966 30156 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:45.382972 30151 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:45.437186 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "f59f3234edf84aab852fa989fb754d31"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:45.445966 27189 raft_consensus.cc:595] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Received request to transfer leadership to TS f59f3234edf84aab852fa989fb754d31
I20260924 14:30:45.446527 27189 raft_consensus.cc:603] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:45.405737 30101 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:45.498056 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:45.499508 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 71975.9us
I20260924 14:30:45.421399 29716 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:45.507967 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:45.301656 29994 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:45.480624 29814 consensus_meta.cc:287] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.555s	user 0.013s	sys 0.000s
I20260924 14:30:45.408805 30045 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
W20260924 14:30:45.523288 26371 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.581s (SENT)
I20260924 14:30:45.526294 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:45.463215 30156 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:45.528635 30153 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260924 14:30:45.556339 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:45.539932 29925 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.981s	user 0.010s	sys 0.003s
I20260924 14:30:45.558058 29925 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:45.558878 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:45.559940 26403 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:45.560022 30128 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:45.541571 29952 consensus_meta.cc:287] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.717s	user 0.005s	sys 0.005s
W20260924 14:30:45.545955 30054 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.979s	user 0.011s	sys 0.000s
I20260924 14:30:45.546628 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:45.547506 29442 consensus_meta.cc:287] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.540s	user 0.007s	sys 0.002s
I20260924 14:30:45.582538 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:45.584117 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1367) took 1433 ms. Trace:
W20260924 14:30:45.585913 26371 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.011s (SENT)
I20260924 14:30:45.550437 30055 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:45.551652 29788 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:45.556455 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "f59f3234edf84aab852fa989fb754d31"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:45.606643 27189 raft_consensus.cc:595] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Received request to transfer leadership to TS f59f3234edf84aab852fa989fb754d31
I20260924 14:30:45.560113 29925 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:45.512107 29902 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.950s	user 0.012s	sys 0.001s
I20260924 14:30:45.541002 29999 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:45.564962 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 63991.2us
I20260924 14:30:45.560870 26252 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:45.591995 29941 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.003s	sys 0.016s
W20260924 14:30:45.592264 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 381337us
I20260924 14:30:45.447722 29845 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:45.637184 29845 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:45.592442 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:45.560936 29952 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:45.550760 30077 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.833s	user 0.010s	sys 0.000s
W20260924 14:30:45.582278 30149 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260924 14:30:45.363365 30161 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:45.625535 26399 raft_consensus.cc:2396] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260924 14:30:45.622383 30033 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.989s	user 0.012s	sys 0.001s
I20260924 14:30:45.620687 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:45.642674 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 56786.6us
W20260924 14:30:45.633678 29941 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.003s	sys 0.017s
I20260924 14:30:45.586443 29989 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:45.654016 26371 leader_election.cc:334] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.041s (SENT)
I20260924 14:30:45.657088 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:45.645215 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:45.626508 26402 raft_consensus.cc:2396] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:30:45.651984 29830 consensus_queue.cc:1225] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260924 14:30:45.585080 26256 rpcz_store.cc:276] 0924 14:30:44.150542 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:45.226538 (+1075996us) service_pool.cc:227] Handling call
0924 14:30:45.270862 (+ 44324us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:45.583683 (+312821us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:45.583728 (+    45us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:45.583786 (+    58us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:45.583830 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:45.583898 (+    68us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:45.584050 (+   152us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":128694,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":2793,"thread_start_us":309847,"threads_started":1}
I20260924 14:30:45.652734 26403 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:45.653137 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:45.644963 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:45.658823 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:45.649750 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.662667 27189 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:45.672209 26331 raft_consensus.cc:2354] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5.
I20260924 14:30:45.673185 26332 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:30:45.676039 26371 leader_election.cc:334] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.596s (SENT)
W20260924 14:30:45.676987 26371 leader_election.cc:334] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.614s (SENT)
I20260924 14:30:45.678298 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:45.681620 26398 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:45.682124 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:45.588186 29442 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:45.689015 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:45.650729 29788 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:45.689178 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.690876 26332 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:45.651245 30054 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:45.691777 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:45.710565 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.656380 30077 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9.
I20260924 14:30:45.711283 27189 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:45.712430 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:45.717655 30164 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260924 14:30:45.717368 29989 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:45.723702 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.716188 26403 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:45.735335 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.757476 27188 raft_consensus.cc:2379] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 5: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:45.674093 30033 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:45.689139 30161 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:30:45.786163 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.083s	sys 0.103s
I20260924 14:30:45.764474 26406 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:45.677681 30171 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:45.787137 27186 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
W20260924 14:30:45.786837 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.083s	sys 0.103s
W20260924 14:30:45.791103 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 546567us
I20260924 14:30:45.793545 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:45.793919 30157 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260924 14:30:45.699129 30084 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.067s
I20260924 14:30:45.717669 30091 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
W20260924 14:30:45.727283 30069 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.001s
I20260924 14:30:45.795400 29952 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:45.617395 30151 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:45.821165 27191 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260924 14:30:45.823426 30165 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260924 14:30:45.795725 29442 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:45.825752 30084 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
W20260924 14:30:45.825045 30145 consensus_meta.cc:287] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.000s
I20260924 14:30:45.827606 30154 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:45.819420 30091 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:45.713171 29970 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260924 14:30:45.940088 26224 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:45.908663 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:30:45.941133 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:45.945080 29830 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:45.914650 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1189) took 2460 ms. Trace:
W20260924 14:30:45.854291 30048 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.313s	user 0.013s	sys 0.000s
I20260924 14:30:45.952706 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.952966 30048 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:45.954046 27182 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:45.954257 29830 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:45.921554 26327 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.008s	sys 0.013s
I20260924 14:30:45.725677 30054 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:45.955405 26327 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.009s	sys 0.013s
I20260924 14:30:45.829985 30069 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:45.835325 30084 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:45.956403 29830 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:45.840113 30111 consensus_meta.cc:287] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.995s	user 0.008s	sys 0.005s
I20260924 14:30:45.864343 30026 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:45.995062 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:45.996501 26327 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:45.928613 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1103) took 1063 ms. Trace:
I20260924 14:30:45.997659 26401 rpcz_store.cc:276] 0924 14:30:44.864938 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.883371 (+ 18433us) service_pool.cc:227] Handling call
0924 14:30:44.884033 (+   662us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:44.884242 (+   209us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:44.884268 (+    26us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:44.884722 (+   454us) log.cc:842] Serialized 26 byte log entry
0924 14:30:45.248572 (+363850us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:45.248611 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:45.248684 (+    73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:45.927770 (+679086us) raft_consensus.cc:1671] finished
0924 14:30:45.927824 (+    54us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:45.928289 (+   465us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":363483,"threads_started":1,"wal-append.queue_time_us":613455}
I20260924 14:30:45.976265 30091 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.001513 30091 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:30:45.860428 30109 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.252s	user 0.013s	sys 0.000s
I20260924 14:30:46.002933 30109 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.005400 27190 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:45.970515 30084 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.006558 30109 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:45.932331 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:46.008113 29629 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:46.008901 29629 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:46.009652 26400 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
W20260924 14:30:45.799705 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 117806us
I20260924 14:30:46.010696 29955 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 3 FOLLOWER]: Advancing to term 4
I20260924 14:30:46.013058 30084 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:45.856032 30172 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.315s	user 0.003s	sys 0.000s
I20260924 14:30:45.837087 30145 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:45.916919 26255 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.000s	sys 0.008s
W20260924 14:30:46.026154 26255 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.001s	sys 0.008s
I20260924 14:30:46.026996 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1361) took 1878 ms. Trace:
I20260924 14:30:46.028388 30048 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:46.028618 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:46.027624 26258 rpcz_store.cc:276] 0924 14:30:44.148711 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.996500 (+847789us) service_pool.cc:227] Handling call
0924 14:30:45.088503 (+ 92003us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:45.112370 (+ 23867us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:45.112420 (+    50us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:45.125302 (+ 12882us) log.cc:842] Serialized 26 byte log entry
0924 14:30:45.430423 (+305121us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:45.855429 (+425006us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:45.920163 (+ 64734us) log.cc:842] Serialized 14 byte log entry
0924 14:30:45.920786 (+   623us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:45.920978 (+   192us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:46.026384 (+105406us) raft_consensus.cc:1671] finished
0924 14:30:46.026527 (+   143us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:46.026781 (+   254us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":48324,"prepare.queue_time_us":99741,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":47216,"raft.run_cpu_time_us":508,"raft.run_wall_time_us":8420,"spinlock_wait_cycles":7617920,"thread_start_us":639474,"threads_started":2,"wal-append.queue_time_us":380480}
I20260924 14:30:46.030043 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:45.693643 26324 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:45.863734 30134 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:45.841264 30154 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:46.035938 30154 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:46.037842 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:46.037896 27190 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 1084) took 1741 ms (client timeout 1683 ms). Trace:
W20260924 14:30:46.039562 27190 rpcz_store.cc:269] 0924 14:30:44.296098 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.436295 (+140197us) service_pool.cc:227] Handling call
0924 14:30:46.037575 (+1601280us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540031f954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:46.037783 (+   208us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3342042496}
I20260924 14:30:46.039961 30099 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:46.040499 30099 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:45.858171 30025 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.737s	user 0.009s	sys 0.000s
W20260924 14:30:46.041249 26252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 946) took 1897 ms (client timeout 1798 ms). Trace:
W20260924 14:30:45.852943 29998 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.000s
I20260924 14:30:46.041965 30099 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:46.042574 29998 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.044366 26250 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:46.046332 26405 raft_consensus.cc:2396] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260924 14:30:46.047546 27185 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:45.923080 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:46.049038 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:45.965685 26397 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:45.966300 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:46.051483 26332 raft_consensus.cc:2396] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260924 14:30:45.983826 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 64172.1us
I20260924 14:30:45.947705 26253 rpcz_store.cc:276] 0924 14:30:43.453977 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.889013 (+1435036us) service_pool.cc:227] Handling call
0924 14:30:44.889318 (+   305us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:44.890498 (+  1180us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:44.890525 (+    27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:44.890870 (+   345us) log.cc:842] Serialized 26 byte log entry
0924 14:30:45.279408 (+388538us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:45.279481 (+    73us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:45.279581 (+   100us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:45.913683 (+634102us) raft_consensus.cc:1671] finished
0924 14:30:45.913772 (+    89us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:45.914325 (+   553us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":294314,"threads_started":1,"wal-append.queue_time_us":564659}
I20260924 14:30:45.852882 30033 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:46.041778 26252 rpcz_store.cc:269] 0924 14:30:44.143244 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.561737 (+418493us) service_pool.cc:227] Handling call
0924 14:30:46.040773 (+1479036us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b54006d2ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:46.041012 (+   239us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2216541952}
I20260924 14:30:46.057482 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1442) took 1091 ms. Trace:
I20260924 14:30:46.032292 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:46.058028 26404 rpcz_store.cc:276] 0924 14:30:44.965996 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:45.594716 (+628720us) service_pool.cc:227] Handling call
0924 14:30:45.605530 (+ 10814us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:45.619180 (+ 13650us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:45.619223 (+    43us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:45.619280 (+    57us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:45.619345 (+    65us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:46.056847 (+437502us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b5400476154. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:46.056915 (+    68us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:46.057218 (+   303us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":30789504}
I20260924 14:30:46.046228 29998 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:46.065188 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:46.066395 26404 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:46.001623 26401 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260924 14:30:45.929208 26371 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.713s (SENT)
I20260924 14:30:45.932765 30055 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.017197 29442 raft_consensus.cc:979] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:46.021186 27189 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:30:46.034965 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:45.873764 30156 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:45.918488 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:46.089602 29994 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.015s	sys 0.008s
W20260924 14:30:46.092084 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 163207us
I20260924 14:30:46.079777 30099 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:46.094277 30048 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:46.096984 26256 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
W20260924 14:30:46.097256 29994 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.015s	sys 0.008s
I20260924 14:30:46.098397 30025 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:46.098718 26398 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
W20260924 14:30:46.099413 26257 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.007s	sys 0.007s
I20260924 14:30:46.102365 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1085) took 1683 ms. Trace:
I20260924 14:30:46.103708 29442 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:46.104668 30033 raft_consensus.cc:1067] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:46.106101 26250 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:46.107152 27194 raft_consensus.cc:2379] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:30:46.078346 29983 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.010s	sys 0.016s
W20260924 14:30:46.121003 26257 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.008s	sys 0.007s
I20260924 14:30:46.123100 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1354) took 1964 ms. Trace:
I20260924 14:30:46.123656 26251 rpcz_store.cc:276] 0924 14:30:44.145887 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.964387 (+818500us) service_pool.cc:227] Handling call
0924 14:30:45.383933 (+419546us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:45.400546 (+ 16613us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:45.995360 (+594814us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54000c26d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:46.109821 (+114461us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":762312448}
I20260924 14:30:46.125696 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1435) took 1369 ms. Trace:
I20260924 14:30:46.126201 26397 rpcz_store.cc:276] 0924 14:30:44.756772 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.770486 (+ 13714us) service_pool.cc:227] Handling call
0924 14:30:46.125392 (+1354906us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400123ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:46.125618 (+   226us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2564779264}
W20260924 14:30:46.128255 26405 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1430) took 1377 ms (client timeout 1664 ms). Trace:
I20260924 14:30:46.107692 30054 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:46.128726 26405 rpcz_store.cc:269] 0924 14:30:44.750756 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.750945 (+   189us) service_pool.cc:227] Handling call
0924 14:30:46.127988 (+1377043us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400476654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:46.128193 (+   205us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2924117888}
I20260924 14:30:46.136065 29442 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.138047 26405 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260924 14:30:46.138520 26405 pending_rounds.cc:76] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 8
I20260924 14:30:46.138839 29442 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:46.132385 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 1094) took 1610 ms. Trace:
I20260924 14:30:46.140033 26402 rpcz_store.cc:276] 0924 14:30:44.521667 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.521814 (+   147us) service_pool.cc:227] Handling call
0924 14:30:46.131865 (+1610051us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002fb7d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:46.132151 (+   286us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2533418624}
I20260924 14:30:46.140445 26398 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:46.141908 26402 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260924 14:30:46.142045 26398 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
W20260924 14:30:46.143884 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.962s	user 0.010s	sys 0.016s
I20260924 14:30:46.129838 26403 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:46.146838 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1243) took 1032 ms. Trace:
I20260924 14:30:46.144340 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1363) took 1995 ms. Trace:
I20260924 14:30:46.148236 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:46.148047 26257 rpcz_store.cc:276] 0924 14:30:44.149397 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:45.096460 (+947063us) service_pool.cc:227] Handling call
0924 14:30:45.130384 (+ 33924us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:45.130673 (+   289us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:30:45.419665 (+288992us) log.cc:842] Serialized 14 byte log entry
0924 14:30:46.143693 (+724028us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:46.143837 (+   144us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:46.143991 (+   154us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:46.144078 (+    87us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:46.144281 (+   203us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":399036,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":917,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":151,"thread_start_us":956171,"threads_started":2}
I20260924 14:30:46.129734 30048 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.150430 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:46.139585 26399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 1091) took 1614 ms (client timeout 1560 ms). Trace:
I20260924 14:30:46.150677 29942 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260924 14:30:46.144349 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.011s	sys 0.016s
W20260924 14:30:46.151492 26399 rpcz_store.cc:269] 0924 14:30:44.493193 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.493370 (+   177us) service_pool.cc:227] Handling call
0924 14:30:46.036347 (+1542977us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400691ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:46.107880 (+ 71533us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2460770560}
I20260924 14:30:46.127718 29914 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260924 14:30:46.151358 26401 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:46.147177 26403 rpcz_store.cc:276] 0924 14:30:45.114097 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:45.730047 (+615950us) service_pool.cc:227] Handling call
0924 14:30:46.129431 (+399384us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:46.144865 (+ 15434us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:46.146640 (+  1775us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b5400123ed0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:46.146779 (+   139us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":917255040}
W20260924 14:30:46.172657 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 124092us
I20260924 14:30:46.173081 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:46.173842 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:46.171834 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 993195us
I20260924 14:30:46.152393 26397 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:46.152395 30048 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.143232 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:46.181890 26251 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
W20260924 14:30:46.183902 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 65463.5us
I20260924 14:30:46.165836 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:46.188690 29677 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:46.189612 30071 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:46.188540 26402 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:46.115818 27191 rpcz_store.cc:276] 0924 14:30:44.418768 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.570269 (+151501us) service_pool.cc:227] Handling call
0924 14:30:44.570586 (+   317us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:45.828394 (+1257808us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:46.017361 (+188967us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54000028d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:46.102147 (+ 84786us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2872592768}
I20260924 14:30:46.189767 29677 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.138763 26405 pending_rounds.cc:97] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260924 14:30:46.190688 30071 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:46.165613 26406 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1434) took 1340 ms (client timeout 1683 ms). Trace:
I20260924 14:30:46.207792 26399 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:46.210668 26405 raft_consensus.cc:2866] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:46.216712 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:46.217519 26251 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:46.218415 29442 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:46.218873 29629 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.219182 29442 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:46.221606 30048 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:46.221726 29629 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:46.228714 29942 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:46.229306 29942 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:46.229048 30033 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260924 14:30:46.221987 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48459.5us
W20260924 14:30:46.201292 29983 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.011s	sys 0.016s
W20260924 14:30:46.213378 26406 rpcz_store.cc:269] 0924 14:30:44.755413 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:44.755577 (+   164us) service_pool.cc:227] Handling call
0924 14:30:46.034433 (+1278856us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54002fbcd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:46.096227 (+ 61794us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2045385984}
I20260924 14:30:46.164616 30156 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:46.237005 30033 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:46.230576 29942 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.222010 29442 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:46.259794 26258 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:46.260887 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:46.242394 29983 raft_consensus.cc:686] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:46.251390 30033 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:46.278663 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45179us
I20260924 14:30:46.267766 30109 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260924 14:30:46.286190 29814 consensus_queue.cc:1046] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:46.289971 30156 raft_consensus.cc:3032] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:46.290357 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:30:46.290519 30156 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:46.292524 26371 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.798s (SENT)
W20260924 14:30:46.293320 26371 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.810s (SENT)
I20260924 14:30:46.293695 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:46.296604 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:46.297650 26252 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:46.304495 30071 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:46.307842 26401 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:46.292105 30156 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:46.269501 30054 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:46.319125 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:46.320288 27125 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:46.334323 29942 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:30:46.334609 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 79364.2us
I20260924 14:30:46.335865 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:46.333076 26404 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:46.344115 27194 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:46.314294 29994 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:46.384073 30154 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:46.345541 26402 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:46.349066 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:46.355307 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:46.355814 29716 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:46.357059 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:46.387482 30025 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260924 14:30:46.357901 29886 consensus_meta.cc:287] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.332s	user 0.012s	sys 0.012s
I20260924 14:30:46.388141 30154 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:46.332181 26401 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:35444
I20260924 14:30:46.388502 29886 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:46.389889 26401 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:46.395699 26401 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:46.344280 30048 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:46.399418 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.362699 30156 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:46.372031 29677 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:46.364609 29983 consensus_queue.cc:237] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:46.387924 30194 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260924 14:30:46.387988 30025 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:46.405472 30025 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:46.407743 26256 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:46.377942 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:46.331292 30054 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:46.418982 30054 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:46.427578 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 70709.4us
I20260924 14:30:46.347921 30171 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:46.387898 29716 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:46.390282 26255 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:46.391366 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:46.451148 26401 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:46.308351 29914 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260924 14:30:46.392654 29886 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.393236 29998 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260924 14:30:46.393801 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 73649.8us
W20260924 14:30:46.359145 29682 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.983s	user 0.016s	sys 0.005s
I20260924 14:30:46.395843 30134 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:30:46.361171 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:46.407629 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:46.464413 26251 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
W20260924 14:30:46.409132 30197 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260924 14:30:46.411283 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:46.314733 30071 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260924 14:30:46.414261 29185 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:46.376359 30033 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:46.325369 30047 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.270s	user 0.027s	sys 0.033s
W20260924 14:30:46.471839 30047 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.548s	user 0.028s	sys 0.033s
I20260924 14:30:46.384243 29994 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.475033 29994 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.428639 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:46.427722 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:46.429353 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:46.426827 26221 connection.cc:232] RPC call timeout handler was delayed by -3.4043560390000493s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2972s for a response
I20260924 14:30:46.426208 29971 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:46.435877 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.467034 29716 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:46.428599 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.452399 30171 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:46.461972 29998 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:46.515102 30167 raft_consensus.cc:3032] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:46.520213 27190 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:30:46.520628 30167 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:46.521880 30167 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.523308 30091 consensus_queue.cc:1046] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:30:46.524312 30091 raft_consensus.cc:3032] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:46.524637 30091 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:46.525439 30091 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.526619 30091 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:46.527191 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:46.528083 30167 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:46.476343 30033 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:46.469559 26401 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:46.491892 27192 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:46.485936 29834 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:46.486033 29716 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.487332 30171 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:46.487506 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1206) took 1157 ms. Trace:
W20260924 14:30:46.492836 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 81733.3us
W20260924 14:30:46.493806 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 102624us
W20260924 14:30:46.496655 26221 connection.cc:232] RPC call timeout handler was delayed by -2.3079840360005619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
I20260924 14:30:46.486508 29971 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:46.498937 26402 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260924 14:30:46.552224 26371 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.573s (SENT)
I20260924 14:30:46.499891 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.500597 26397 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:46.485854 30048 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:46.555316 26221 connection.cc:232] RPC call timeout handler was delayed by -1.9326840359999551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2054s for a response
I20260924 14:30:46.505057 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:46.504632 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.507283 29914 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:46.509404 30054 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:46.509969 29185 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.513785 30025 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.550424 26249 rpcz_store.cc:276] 0924 14:30:45.330099 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:45.371330 (+ 41231us) service_pool.cc:227] Handling call
0924 14:30:45.371765 (+   435us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:45.423893 (+ 52128us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:30:46.132950 (+709057us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b18001a6a40. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6423155 00007f09bcd23167 00007f09b9358807 00007f09b9367f0f 00007f09b9367e88
0924 14:30:46.207087 (+ 74137us) log.cc:842] Serialized 14 byte log entry
0924 14:30:46.287414 (+ 80327us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:46.288266 (+   852us) log.cc:842] Serialized 26 byte log entry
0924 14:30:46.288518 (+   252us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:46.288569 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:46.403305 (+114736us) spinlock_profiling.cc:243] Waited 47.6 ms on lock 0x7b5400288f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:46.403367 (+    62us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:46.487125 (+ 83758us) raft_consensus.cc:1671] finished
0924 14:30:46.487150 (+    25us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:46.487308 (+   158us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3973,"prepare.queue_time_us":106062,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":4518,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":18891,"spinlock_wait_cycles":115336192,"thread_start_us":102269,"threads_started":2,"wal-append.queue_time_us":232089}
I20260924 14:30:46.562265 29834 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.563624 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:46.564190 30025 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.564404 29834 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.515729 30077 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:46.564882 30134 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:30:46.565230 30077 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:46.565869 30134 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:46.516073 27125 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:46.566403 30134 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:46.566792 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50867.3us
I20260924 14:30:46.520138 29998 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:46.489637 27189 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:46.567476 26400 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:30:46.549223 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:46.499684 29886 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:46.554598 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.557751 26324 raft_consensus.cc:2396] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260924 14:30:46.572947 30134 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:46.575625 30071 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election won for term 5
I20260924 14:30:46.576046 29185 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.576220 30071 raft_consensus.cc:686] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:46.576799 29716 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.559515 26402 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:46.557922 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:46.561270 26221 connection.cc:232] RPC call timeout handler was delayed by -1.8786840359998678s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
I20260924 14:30:46.557597 29914 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:46.578603 26221 connection.cc:232] RPC call timeout handler was delayed by -1.8417840359998081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
I20260924 14:30:46.565483 30054 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.570550 27190 raft_consensus.cc:3032] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
W20260924 14:30:46.579031 26221 connection.cc:232] RPC call timeout handler was delayed by -1.4970840360001603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25379999999999997s for a response
I20260924 14:30:46.578368 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:46.575527 29830 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:46.575407 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.580526 27182 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:46.581826 27190 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:46.585922 26221 connection.cc:232] RPC call timeout handler was delayed by -1.2117840360006085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28550000000000003s for a response
I20260924 14:30:46.588461 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:46.582561 29830 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:46.590271 29983 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:46.590458 29886 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:46.591559 30099 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:46.591142 29830 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:46.591420 29886 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:46.580323 29914 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:46.598920 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:46.599740 27182 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:30:46.600682 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 188118us
I20260924 14:30:46.601544 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.561028 30048 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.607487 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.594796 27190 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:46.594959 30156 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:46.598043 29834 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:46.617869 27186 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:46.622503 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:46.622691 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:46.619087 29830 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:46.618820 30099 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:46.577255 30071 consensus_queue.cc:237] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:46.626859 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:46.605774 30091 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:46.631413 29983 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:46.632043 29983 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:46.632392 29983 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:46.627888 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:46.620872 29834 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:46.650735 29762 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:46.626641 30156 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:46.619220 30154 consensus_queue.cc:1046] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260924 14:30:46.649470 29834 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:46.672165 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:46.678253 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 126051us
I20260924 14:30:46.633610 30099 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:46.636085 30091 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.693661 30091 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.680375 30151 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:30:46.688333 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 90411.3us
I20260924 14:30:46.699340 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:46.701308 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:46.702822 29925 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:46.708243 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 660) took 1051 ms. Trace:
I20260924 14:30:46.708842 27192 rpcz_store.cc:276] 0924 14:30:45.657166 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:45.657344 (+   178us) service_pool.cc:227] Handling call
0924 14:30:46.707988 (+1050644us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54002c51d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:46.708173 (+   185us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1797057792}
I20260924 14:30:46.721498 29971 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:46.723830 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 116575us
I20260924 14:30:46.726900 29989 consensus_queue.cc:1046] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260924 14:30:46.727576 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 132407us
I20260924 14:30:46.728796 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:46.730453 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:46.732599 30048 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.735713 30091 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:46.736914 30091 raft_consensus.cc:2741] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260924 14:30:46.740118 30054 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.740803 27190 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:46.741547 30156 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:46.669052 29814 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:46.749464 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.753185 27182 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:30:46.750394 27194 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:46.762158 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:46.764451 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:46.728809 29989 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:46.769414 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:46.759091 30151 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:46.772722 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:46.799201 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 68904.7us
W20260924 14:30:46.813699 26371 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.212s (SENT)
I20260924 14:30:46.814438 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:46.819154 27192 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:46.784861 30048 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:46.778676 29989 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:46.775648 30054 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:46.837729 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45963.6us
I20260924 14:30:46.844790 29983 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.856160 29983 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.812983 29716 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:30:46.844105 30048 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:46.877090 29682 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260924 14:30:46.889320 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:46.893158 29834 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.894433 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:46.863070 30025 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:46.883738 29981 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:46.898797 30099 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:46.896025 29716 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:46.911132 26297 leader_election.cc:398] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
I20260924 14:30:46.849592 30091 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:46.934337 26297 leader_election.cc:401] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260924 14:30:46.935138 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:46.935650 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:46.913290 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 99649.6us
I20260924 14:30:46.909825 30048 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:46.925633 29442 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:30:46.960676 27190 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:46.972795 26369 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:46.974674 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:46.977099 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:46.978132 26404 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 8: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:46.977793 26400 raft_consensus.cc:2362] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9. Re-sending same reply.
I20260924 14:30:46.985456 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:46.973333 27191 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:46.944276 30077 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:46.973829 30208 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260924 14:30:46.998795 29834 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.984717 29442 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:46.973737 29989 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:47.001003 29442 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:46.937146 29981 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:47.002170 29989 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:46.880206 30054 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:47.005586 30054 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:47.006863 29925 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:46.932884 30171 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:46.941026 30091 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:47.010365 30091 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:46.986613 29998 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:47.013595 29998 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:47.016474 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 81742.8us
I20260924 14:30:47.019827 30026 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:47.021035 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 171197us
I20260924 14:30:47.022043 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:47.026887 29942 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:47.028537 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:47.029032 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:47.029309 26399 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:30:47.029435 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:47.030743 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:47.031324 26404 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:30:47.036304 30212 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260924 14:30:47.037890 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 65218.9us
I20260924 14:30:47.038017 30077 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:47.038741 30077 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:47.040805 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:47.041656 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:47.027298 29942 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:47.042443 27190 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:47.042732 29942 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:46.988804 29716 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:47.043661 26371 leader_election.cc:334] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.853s (SENT)
W20260924 14:30:47.044826 26371 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.870s (SENT)
I20260924 14:30:47.045253 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.047633 29814 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:47.047762 30077 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:47.048417 27192 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:47.052234 26398 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:47.003755 29981 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:47.054225 29914 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:47.055599 26371 leader_election.cc:334] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.241s (SENT)
I20260924 14:30:47.055908 26297 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:47.056099 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:47.064920 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:47.065773 27191 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:47.064559 26404 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:47.077186 29981 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:47.080031 27192 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:47.074726 26405 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:30:47.068315 30054 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260924 14:30:47.081488 30167 consensus_meta.cc:287] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.553s	user 0.012s	sys 0.000s
I20260924 14:30:47.116756 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:47.066682 29830 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
W20260924 14:30:47.137910 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 82368.3us
I20260924 14:30:47.144555 27191 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:47.115079 29998 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:47.167229 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.135617 29677 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:30:47.134222 30054 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:47.180051 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 478877us
W20260924 14:30:47.181365 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50403.1us
W20260924 14:30:47.184526 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 372123us
I20260924 14:30:47.176364 29834 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:47.188915 29834 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:47.194877 29834 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:47.173131 29998 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:47.191942 27123 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.894s (SENT) [suppressed 56 similar messages]
I20260924 14:30:47.174401 30171 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:47.204299 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:47.206296 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 82378.6us
I20260924 14:30:47.189709 30091 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:47.185068 29677 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:47.209532 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:47.193739 29942 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:47.213261 29942 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:47.215167 30134 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:47.216740 26405 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:47.221951 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:47.227056 27192 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:47.222890 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:47.231153 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:47.231861 27191 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:30:47.185729 30054 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:47.211143 29925 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260924 14:30:47.239427 26371 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.862s (SENT)
I20260924 14:30:47.237701 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:47.247247 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:47.251750 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.258984 27192 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
W20260924 14:30:47.262928 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53505.3us
I20260924 14:30:47.270295 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.214736 30048 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:47.281323 30048 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:47.278110 30218 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260924 14:30:47.283322 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:47.230603 29914 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:47.284051 27191 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:47.233594 30091 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:47.244396 30171 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election won for term 8
W20260924 14:30:47.286598 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 149564us
I20260924 14:30:47.264755 29834 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:47.271566 29983 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260924 14:30:47.297919 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 63519.4us
I20260924 14:30:47.323119 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.284390 29185 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:47.287941 30025 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:47.327857 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 139706us
I20260924 14:30:47.295212 29762 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:47.261112 29830 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260924 14:30:47.301285 29983 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:47.335222 29914 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:47.354974 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:47.352927 26371 leader_election.cc:334] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.957s (SENT)
I20260924 14:30:47.351944 27191 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:47.362332 29955 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:47.336551 29834 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:47.362723 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:47.378687 26250 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:47.367277 29886 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:30:47.346128 29983 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:30:47.368860 29971 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:47.371867 30091 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:47.336761 30171 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:47.379842 27192 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
W20260924 14:30:47.391687 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 88180.9us
I20260924 14:30:47.394953 30225 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:47.395759 30225 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:47.360164 29914 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:47.361526 29981 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:47.333119 29185 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:47.402819 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 132618us
I20260924 14:30:47.403918 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.405340 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:47.412340 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 89339.4us
I20260924 14:30:47.414155 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:47.381646 29886 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:47.335549 29677 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:47.419631 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:47.435882 26295 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:47.406191 29971 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:47.443420 29830 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:47.445340 30224 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260924 14:30:47.452191 29955 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:47.452378 30054 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:47.452831 29955 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:47.453914 30171 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:47.458609 29886 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:47.460626 29886 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:47.462386 30231 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election won for term 9
W20260924 14:30:47.466284 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 113406us
I20260924 14:30:47.468627 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:47.469934 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:47.408349 29914 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:47.470834 26398 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:47.417062 29998 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:47.485570 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 81754.4us
W20260924 14:30:47.487648 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 67176.7us
I20260924 14:30:47.424767 30225 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:47.492468 29955 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:47.492815 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:47.486764 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.500124 29914 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:47.500937 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 65178us
I20260924 14:30:47.502934 29981 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:47.503387 29981 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:47.503684 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:47.504793 27194 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:30:47.506369 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:47.507939 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 183873us
I20260924 14:30:47.495174 30026 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
I20260924 14:30:47.510586 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:47.503820 29981 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:47.516564 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.489930 30231 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:47.470759 29834 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:47.518237 26404 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:47.445355 29971 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:47.492897 29185 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:47.521458 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.511825 30167 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260924 14:30:47.525816 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106280us
I20260924 14:30:47.513355 29914 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:30:47.523702 26256 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:47.517113 30230 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:30:47.539629 30230 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:47.540107 30230 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:47.541136 30156 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:47.525243 29942 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:47.545773 29762 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:47.546898 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:47.549036 26371 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.089s (SENT)
I20260924 14:30:47.549582 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.550880 30054 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:30:47.551018 26371 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.121s (SENT)
I20260924 14:30:47.556641 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.554706 26251 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260924 14:30:47.555577 30232 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260924 14:30:47.532163 26405 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260924 14:30:47.553983 30233 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260924 14:30:47.562407 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:47.529390 29914 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:47.577589 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:47.559890 30225 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:47.559109 29983 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:47.579281 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:47.584234 26404 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260924 14:30:47.566928 29185 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:47.564174 30025 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:30:47.599979 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 85439us
I20260924 14:30:47.580178 29914 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:30:47.585994 30054 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:30:47.639073 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:47.636781 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 85809.9us
I20260924 14:30:47.648291 27192 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:47.599426 29955 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:30:47.663592 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:47.581375 29442 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260924 14:30:47.636262 30109 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:47.675871 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 153614us
I20260924 14:30:47.636181 29185 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:47.592337 30225 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:47.652282 30054 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:47.690464 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:47.630825 29925 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260924 14:30:47.670562 29830 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:30:47.692631 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 67649.1us
I20260924 14:30:47.694129 26399 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:47.671494 30167 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:47.674729 29955 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:47.696493 26403 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:30:47.694675 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:47.669976 30077 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260924 14:30:47.716207 30054 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:47.717051 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:47.717805 26400 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:47.650347 30071 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260924 14:30:47.722579 30167 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:47.722892 29983 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:47.723800 30167 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:47.705308 29185 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:47.733886 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:47.632998 30231 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:47.735697 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 105161us
I20260924 14:30:47.752625 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:47.753388 27194 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:30:47.682224 30109 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:47.755108 26221 leader_election.cc:398] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5.
I20260924 14:30:47.757252 26249 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260924 14:30:47.766914 26221 leader_election.cc:401] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260924 14:30:47.744149 30238 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.069s	user 0.003s	sys 0.000s
W20260924 14:30:47.743966 30243 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.062s	user 0.003s	sys 0.000s
I20260924 14:30:47.741295 30091 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:47.760880 30230 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:47.753270 26371 leader_election.cc:334] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.243s (SENT)
I20260924 14:30:47.791081 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:47.793282 26250 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:47.798168 29955 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:47.799079 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.800715 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.800750 30167 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:47.801249 30230 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:47.801312 30167 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:47.802033 30167 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:47.802095 30230 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:30:47.802647 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:47.773885 30151 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 3 FOLLOWER]: Advancing to term 5
I20260924 14:30:47.772209 30091 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:47.803347 30156 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:47.804796 29942 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:47.817422 27125 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:47.808840 29983 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:47.813657 30091 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:47.788259 30084 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
W20260924 14:30:47.841181 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 99024.8us
I20260924 14:30:47.825178 29762 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260924 14:30:47.853366 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50822.6us
I20260924 14:30:47.842615 30071 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:47.868078 30071 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:47.869496 30071 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:47.836571 30156 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:47.886700 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 140726us
I20260924 14:30:47.909955 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:47.912364 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:47.913076 26249 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:47.911695 30109 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:47.916370 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:47.924608 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:47.925768 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:47.926344 29925 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:47.926805 29830 raft_consensus.cc:2741] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260924 14:30:47.928772 26405 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:30:47.931892 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:47.936664 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49102.9us
I20260924 14:30:47.938656 29185 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260924 14:30:47.939136 30026 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:47.946857 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:47.947618 26326 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:30:47.948350 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:47.957681 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:47.964640 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:47.965365 26327 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:47.965602 26332 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:30:47.972635 26371 leader_election.cc:334] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.113s (SENT)
I20260924 14:30:47.973194 26371 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:47.973788 30237 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:47.974277 30237 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:47.974608 30237 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:47.977396 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44459.5us
W20260924 14:30:47.983080 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 57403.3us
I20260924 14:30:47.914116 30054 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:47.990382 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:47.996867 26327 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:47.986073 29886 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.004s
I20260924 14:30:47.989089 26331 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:48.061940 27194 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:30:48.094403 26326 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:48.051949 26406 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:47.986253 29983 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:30:48.100736 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 107624us
I20260924 14:30:47.928927 29677 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260924 14:30:48.111310 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:47.985944 30227 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:48.112157 26255 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:48.052294 29830 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:48.114858 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:48.112263 30025 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:47.983026 29981 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:48.115844 26332 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:48.128121 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.132570 30025 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:48.120805 29830 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:48.161330 29914 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.000s
I20260924 14:30:48.107868 30054 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:48.132608 29942 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:48.202843 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 94879.2us
I20260924 14:30:48.207604 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.208200 26327 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
W20260924 14:30:48.210520 26371 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.029s (SENT)
I20260924 14:30:48.176684 29830 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:48.175675 29886 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:48.213541 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.127202 30227 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:48.232440 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:48.171137 30033 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260924 14:30:48.264573 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.265333 26331 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
W20260924 14:30:48.270099 30269 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260924 14:30:48.271062 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.271517 26332 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:30:48.275511 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.275743 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.276046 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.276214 26329 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:48.276436 26327 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
W20260924 14:30:48.278283 30265 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260924 14:30:48.204044 29942 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:48.276589 26330 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 12
W20260924 14:30:48.262482 30255 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260924 14:30:48.260363 29886 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:48.200452 29914 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:48.319931 30109 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:48.320478 30109 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:48.321583 29914 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:48.270176 29981 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:48.298545 26328 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:30:48.284741 26326 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:48.314088 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:48.317452 30254 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:48.313894 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.318462 30274 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:48.279081 30167 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:48.317771 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:48.326819 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 125414us
I20260924 14:30:48.324889 26406 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:30:48.313931 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.344221 26323 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:48.347605 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.348556 26331 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:48.298846 30151 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 4. Current term is 5.
I20260924 14:30:48.352669 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:48.353178 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:48.356185 26406 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:48.317471 30054 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:48.363969 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.327157 30156 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:48.367231 27194 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:30:48.364718 26323 raft_consensus.cc:2362] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13. Re-sending same reply.
I20260924 14:30:48.374162 26331 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:48.350121 30025 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:48.352345 29925 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:48.389859 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:48.391036 26406 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:48.368667 30026 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:48.395143 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.343745 29886 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:48.403756 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:48.404896 30026 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:48.405507 27194 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:30:48.369407 30109 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:48.375999 30231 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:30:48.407406 26325 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:48.408373 27191 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:48.388382 30226 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:48.429697 30226 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:48.430436 30026 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:48.432298 30226 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:48.420383 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 157038us
I20260924 14:30:48.442869 30167 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:48.426604 30237 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:48.421773 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.352114 30274 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:48.436419 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:48.866282 26323 raft_consensus.cc:2362] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8. Re-sending same reply.
W20260924 14:30:48.867237 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 430904us
I20260924 14:30:48.867594 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.868122 30274 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:48.869287 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.672181 26325 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260924 14:30:48.684508 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 288489us
I20260924 14:30:48.870539 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.871248 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:48.871526 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.872051 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.872822 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:48.873208 30270 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:48.873100 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.727043 26332 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:48.874254 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:48.874396 30025 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:30:48.874969 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.875113 30025 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:48.727324 26329 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
W20260924 14:30:48.537624 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 442568us
I20260924 14:30:48.969956 27181 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:30:48.970336 30281 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:48.428022 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:48.513195 29830 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:49.042902 26324 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:49.044068 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.467216 29942 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:48.425567 27186 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260924 14:30:48.422919 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:49.069218 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:48.508579 30091 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.665s	user 0.006s	sys 0.007s
I20260924 14:30:48.427347 27190 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:49.097697 26078 catalog_manager.cc:5495] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 7 to 10, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 10 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:48.607630 29998 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:49.101377 29998 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:48.487272 29442 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:49.104462 29442 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:49.105756 27186 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:48.413925 29716 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:48.471895 30227 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:49.108577 29716 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:48.447862 30231 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:48.514981 30237 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:48.667935 30225 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:49.110987 30231 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:49.112370 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:48.768564 26405 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.013s	sys 0.027s
W20260924 14:30:49.113125 26405 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.013s	sys 0.028s
I20260924 14:30:49.113741 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1269) took 1017 ms. Trace:
I20260924 14:30:49.114071 26405 rpcz_store.cc:276] 0924 14:30:48.096060 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:48.101778 (+  5718us) service_pool.cc:227] Handling call
0924 14:30:48.120679 (+ 18901us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:48.120886 (+   207us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:30:48.134022 (+ 13136us) log.cc:842] Serialized 14 byte log entry
0924 14:30:49.113416 (+979394us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:49.113468 (+    52us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:49.113511 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:49.113579 (+    68us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:49.113692 (+   113us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":39087,"prepare.run_cpu_time_us":6378,"prepare.run_wall_time_us":20028,"raft.queue_time_us":119620,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":85,"thread_start_us":915790,"threads_started":1}
I20260924 14:30:49.115171 29998 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:48.779544 26297 connection.cc:232] RPC call timeout handler was delayed by -1.1917249669995726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
I20260924 14:30:49.117287 26078 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 9 to 12, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 12 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
W20260924 14:30:49.118456 26297 leader_election.cc:334] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.555s (SENT)
I20260924 14:30:49.118999 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:48.420368 26330 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:49.120340 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.119946 26078 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 9 to 10, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 10 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:49.121615 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:49.122323 26330 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:48.837780 26250 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:49.124087 26250 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:49.123771 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:48.842001 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:48.848152 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.124962 26252 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:48.866020 26371 leader_election.cc:334] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 3.086s (SENT)
I20260924 14:30:48.725096 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.124895 26257 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:49.126412 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:49.126161 26250 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:49.126696 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.127267 26250 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:49.127648 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:49.127969 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 261937us
I20260924 14:30:49.128082 29942 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:49.128953 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.129662 26331 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:48.876113 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.129475 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:48.877157 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.878012 30171 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:48.878330 26256 raft_consensus.cc:1203] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[3.3-3.3]   Dedup: 3.3->[]
I20260924 14:30:48.879642 30254 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:30:48.875639 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:48.872743 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.881570 30025 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:49.137007 26250 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260924 14:30:48.885851 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:49.145155 30275 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:30:48.886649 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:48.887399 26326 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:49.146880 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:49.148046 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:48.890055 30286 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:48.888379 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:48.893457 30054 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:48.894971 30280 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:48.895846 30279 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 14:30:48.907233 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1250) took 1373 ms. Trace:
I20260924 14:30:48.914091 26328 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:49.156517 26255 rpcz_store.cc:276] 0924 14:30:47.533625 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:47.546569 (+ 12944us) service_pool.cc:227] Handling call
0924 14:30:48.907000 (+1360431us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54004fa454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:48.907158 (+   158us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1298135296}
I20260924 14:30:49.157707 26330 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:49.157826 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.158752 26328 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:49.158898 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.151971 30286 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:49.159765 30286 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:49.159628 26330 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:49.160673 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.161234 26330 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:49.162055 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.163094 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:49.163180 30280 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:49.163625 26330 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:49.157824 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.164908 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.165220 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.166303 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.166482 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.937955 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:49.167801 26255 raft_consensus.cc:2396] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260924 14:30:48.939513 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:48.941504 27188 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260924 14:30:48.944577 30264 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.844s	user 0.006s	sys 0.002s
I20260924 14:30:48.970973 30281 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:48.974325 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:48.976032 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:49.171378 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:48.976754 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:48.982209 27181 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:48.984385 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:48.983011 26254 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
W20260924 14:30:48.990089 29834 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.268s	user 0.006s	sys 0.009s
I20260924 14:30:48.991549 26323 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:49.014348 27185 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:30:49.179159 29834 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.457s	user 0.007s	sys 0.010s
I20260924 14:30:49.032040 30270 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:49.034909 30230 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:49.183706 30270 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:49.183815 30230 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:49.186059 30270 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:49.186204 30230 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:49.035271 30271 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:49.187178 26258 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
W20260924 14:30:49.036119 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 75956.7us
I20260924 14:30:49.189958 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.189890 30286 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:49.190737 26258 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
W20260924 14:30:49.035884 26221 connection.cc:232] RPC call timeout handler was delayed by -2.0020251929990991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
I20260924 14:30:49.171494 26330 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:49.044888 26324 raft_consensus.cc:2354] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16.
I20260924 14:30:49.191931 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:48.260248 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:49.081328 27189 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:49.097530 30091 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:49.191226 29762 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.002s
I20260924 14:30:49.099212 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:49.194907 26327 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:49.110900 29716 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:49.195079 30091 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:48.424379 30266 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260924 14:30:49.196741 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:49.197333 26399 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:49.189934 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:49.111598 29442 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:48.641920 30267 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.527s	user 0.003s	sys 0.000s
I20260924 14:30:48.841110 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.197639 27181 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:49.198978 30231 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260924 14:30:49.205269 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.205500 26078 catalog_manager.cc:5495] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 6 to 9, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 9 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:30:49.206462 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.207119 26249 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:49.127976 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.207288 26078 catalog_manager.cc:5495] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 4 to 5, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 5 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:30:49.208750 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.208907 26078 catalog_manager.cc:5495] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 6 to 7, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 7 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:30:49.209893 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.210630 26257 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:49.131882 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:49.136015 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:49.211807 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 80014us
I20260924 14:30:49.212388 26295 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:49.212677 26332 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:30:49.213102 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:49.213948 30084 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:49.214349 30084 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:49.214738 30084 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:49.214639 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:48.884815 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:49.215754 26401 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:49.139537 26256 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 3. Current term is 5. Ops: [3.3-3.3]
I20260924 14:30:49.217159 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.217940 26256 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:49.218835 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.139647 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.214380 27125 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:49.221041 26331 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:49.221145 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:49.140465 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.142186 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.143534 30227 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:30:49.143654 30237 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:49.143927 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.223893 30237 raft_consensus.cc:3032] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:49.224264 26332 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:49.224313 30237 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:49.225103 26329 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:49.225450 26332 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:49.225339 30237 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:49.223649 26325 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260924 14:30:49.226321 26330 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:49.226734 30237 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:49.211360 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.223937 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.229449 26256 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:49.230121 30286 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:49.230373 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.231050 26256 raft_consensus.cc:2362] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13. Re-sending same reply.
I20260924 14:30:49.144527 29998 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:30:49.147944 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:49.243975 30071 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:48.895409 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:49.244423 30071 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:49.243779 26078 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 5 to 6, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 6 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
W20260924 14:30:49.244987 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 349639us
W20260924 14:30:49.150185 27123 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.894s (SENT)
I20260924 14:30:49.245704 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:49.245441 30071 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:49.246577 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 96436.6us
I20260924 14:30:49.149049 26251 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:49.149930 26250 pending_rounds.cc:76] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 7
I20260924 14:30:49.247596 26250 pending_rounds.cc:97] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260924 14:30:49.243994 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.170440 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.247900 26250 raft_consensus.cc:2866] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:49.248057 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.173446 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.248700 26251 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:30:49.248889 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.178544 30281 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:49.249519 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.249787 30281 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:49.250097 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.250257 29716 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:49.250901 29716 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:49.172428 26404 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:49.029371 29886 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:49.251679 29998 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:49.180625 26398 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 8: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:49.179157 27191 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:49.176101 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.181843 27188 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260924 14:30:49.163635 30280 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:49.191648 26221 connection.cc:232] RPC call timeout handler was delayed by -1.9318251929998951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
I20260924 14:30:49.253154 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:49.253454 26221 connection.cc:232] RPC call timeout handler was delayed by -1.8958251929998369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
I20260924 14:30:49.044816 29830 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:49.182744 27194 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:49.058387 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:49.253832 30280 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:49.253775 26221 connection.cc:232] RPC call timeout handler was delayed by -1.7293251929995677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
W20260924 14:30:49.254925 26221 connection.cc:232] RPC call timeout handler was delayed by -1.6474251929994352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
I20260924 14:30:49.255155 30280 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:49.255460 30275 raft_consensus.cc:2741] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260924 14:30:49.255178 26221 connection.cc:232] RPC call timeout handler was delayed by -1.6195251929993901s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260924 14:30:49.256719 26221 connection.cc:232] RPC call timeout handler was delayed by -1.611425192999377s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
W20260924 14:30:49.256989 26221 connection.cc:232] RPC call timeout handler was delayed by -1.5835251929993319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19710000000000003s for a response
W20260924 14:30:49.257223 26221 connection.cc:232] RPC call timeout handler was delayed by -1.4827251929991689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
W20260924 14:30:49.257467 26221 connection.cc:232] RPC call timeout handler was delayed by -1.4701251929991486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2097s for a response
W20260924 14:30:49.257719 26221 connection.cc:232] RPC call timeout handler was delayed by -1.4305251929990845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2141s for a response
W20260924 14:30:49.258046 26221 connection.cc:232] RPC call timeout handler was delayed by -1.3432251929998529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2238s for a response
I20260924 14:30:49.259639 30287 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:49.187134 30271 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:49.261508 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:49.262305 30271 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:49.262602 29834 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:49.183854 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:49.130960 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:49.222604 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.264757 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.265995 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.266633 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:49.253779 26255 raft_consensus.cc:2362] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10. Re-sending same reply.
I20260924 14:30:49.268115 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.268656 30287 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:49.247869 26078 catalog_manager.cc:5495] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 7 to 10, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 10 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:49.269393 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.270607 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.271034 26251 raft_consensus.cc:2396] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260924 14:30:49.268148 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:49.274266 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 61961.4us
I20260924 14:30:49.274484 27125 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:49.278530 26079 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55200 (request call id 128) took 31106 ms (client timeout 15000 ms). Trace:
I20260924 14:30:49.205153 30231 consensus_queue.cc:1225] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260924 14:30:49.144783 27186 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:49.198096 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 674) took 1462 ms. Trace:
I20260924 14:30:49.244181 29942 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:30:49.244880 29762 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:49.246953 30071 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:49.152268 26252 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:49.317299 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:49.155073 26297 leader_election.cc:334] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.099s (SENT)
I20260924 14:30:49.249028 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.318902 26250 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:49.320044 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.250373 30281 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:49.251274 29716 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:49.179744 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.328716 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:49.329473 26323 raft_consensus.cc:2362] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10. Re-sending same reply.
I20260924 14:30:49.180295 27185 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:49.194373 26324 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 12. Ops: []
I20260924 14:30:49.222790 26253 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:49.278825 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.271183 26255 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:30:49.232529 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.267015 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.284200 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:49.332329 26249 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:49.333348 29442 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:49.335825 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 69262.5us
I20260924 14:30:49.337489 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:49.279253 26078 catalog_manager.cc:5495] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:30:49.338513 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:49.339061 26078 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:49.340672 29886 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:49.340780 30025 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:30:49.285002 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:49.342559 27182 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:49.342736 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.340672 26078 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 2 to 3, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 3 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:30:49.290292 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:49.293751 29834 raft_consensus.cc:686] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:49.345261 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.345913 26255 raft_consensus.cc:2362] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10. Re-sending same reply.
I20260924 14:30:49.345597 29834 consensus_queue.cc:237] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:49.291927 29185 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:49.347692 29185 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:49.349584 29185 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:49.292101 30054 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260924 14:30:49.292843 30271 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:49.331351 26251 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:49.297462 26224 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:30:49.352322 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 54931.9us
I20260924 14:30:49.300026 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:49.354455 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.290784 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:49.355798 27186 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:49.292670 26398 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:49.302060 26399 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:49.302543 26257 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:30:49.286340 27192 rpcz_store.cc:276] 0924 14:30:47.737326 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:47.737586 (+   260us) service_pool.cc:227] Handling call
0924 14:30:47.750746 (+ 13160us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:47.751277 (+   531us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:47.751337 (+    60us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:47.751953 (+   616us) log.cc:842] Serialized 26 byte log entry
0924 14:30:47.894682 (+142729us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:47.894747 (+    65us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:47.894868 (+   121us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:49.197751 (+1302883us) raft_consensus.cc:1671] finished
0924 14:30:49.197776 (+    25us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:49.198030 (+   254us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1934,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":142175,"threads_started":1,"wal-append.queue_time_us":302371}
I20260924 14:30:49.356916 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.357609 26250 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:49.322300 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:49.277244 26254 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:49.343978 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:49.238822 30084 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:49.370012 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 70085.5us
I20260924 14:30:49.370581 29185 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:49.359612 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:49.280228 30287 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:49.368889 26258 raft_consensus.cc:2396] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260924 14:30:49.370923 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:49.355197 26253 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:30:49.370762 26255 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:49.378865 26255 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:30:49.381564 30084 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:49.383859 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.389549 27182 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260924 14:30:49.389752 26254 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260924 14:30:49.405823 26254 pending_rounds.cc:76] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 4
I20260924 14:30:49.406061 26254 pending_rounds.cc:97] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260924 14:30:49.406297 26254 raft_consensus.cc:2866] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:49.392128 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:49.389482 27185 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:49.407683 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:49.408243 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 253170us
I20260924 14:30:49.408457 26399 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 13
I20260924 14:30:49.392310 30281 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:49.389448 29998 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:49.368325 29830 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:49.414915 27186 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:49.397436 30231 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:49.425045 26399 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:49.402360 26249 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
W20260924 14:30:49.280511 26079 rpcz_store.cc:269] 0924 14:30:18.208302 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:18.208569 (+   267us) service_pool.cc:227] Handling call
0924 14:30:49.278299 (+31069730us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:30:46.211032 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:30:46.211905 (+   873us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:30:46.211974 (+    69us) write_op.cc:435] Acquired schema lock
0924 14:30:46.234205 (+ 22231us) tablet.cc:660] Decoding operations
0924 14:30:46.301612 (+ 67407us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:30:46.301725 (+   113us) write_op.cc:641] Partition lock acquired for write op
0924 14:30:46.301780 (+    55us) tablet.cc:683] Acquiring locks for 11 operations
0924 14:30:46.303196 (+  1416us) tablet.cc:699] Row locks acquired
0924 14:30:46.303236 (+    40us) write_op.cc:260] PREPARE: finished
0924 14:30:46.324297 (+ 21061us) write_op.cc:270] Start()
0924 14:30:46.434663 (+110366us) write_op.cc:276] Timestamp: P: 1790260246303425 usec, L: 0
0924 14:30:46.434713 (+    50us) op_driver.cc:331] REPLICATION: starting
0924 14:30:46.444295 (+  9582us) log.cc:842] Serialized 4454 byte log entry
0924 14:30:47.356040 (+911745us) op_driver.cc:444] REPLICATION: finished
0924 14:30:47.548706 (+192666us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b64085fe 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09bcd5e410
0924 14:30:47.790494 (+241788us) write_op.cc:301] APPLY: starting
0924 14:30:47.808895 (+ 18401us) tablet.cc:1365] starting BulkCheckPresence
0924 14:30:47.875647 (+ 66752us) tablet.cc:1368] finished BulkCheckPresence
0924 14:30:47.885795 (+ 10148us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:30:47.916593 (+ 30798us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:30:47.917072 (+   479us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0924 14:30:47.917170 (+    98us) write_op.cc:312] APPLY: finished
0924 14:30:47.917667 (+   497us) log.cc:842] Serialized 82 byte log entry
0924 14:30:47.919135 (+  1468us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:47.919363 (+   228us) write_op.cc:454] Released schema lock
0924 14:30:47.919505 (+   142us) write_op.cc:341] FINISH: Updating metrics
0924 14:30:47.919983 (+   478us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:47.920029 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":963088,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":361941,"apply.run_cpu_time_us":6027,"apply.run_wall_time_us":148700,"num_ops":11,"prepare.queue_time_us":662412,"prepare.run_cpu_time_us":46016,"prepare.run_wall_time_us":460485,"raft.queue_time_us":326665,"raft.run_cpu_time_us":40226,"raft.run_wall_time_us":342695,"replication_time_us":911042,"spinlock_wait_cycles":64518912,"thread_start_us":1172885,"threads_started":4,"wal-append.queue_time_us":500395,"wal-append.run_cpu_time_us":22919,"wal-append.run_wall_time_us":1971081}]]}
I20260924 14:30:49.316507 30275 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:49.415097 26250 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
W20260924 14:30:49.416232 26371 leader_election.cc:334] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.863s (SENT)
I20260924 14:30:49.375249 26256 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:49.417645 27185 raft_consensus.cc:595] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:49.417987 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47144.5us
I20260924 14:30:49.420135 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:49.417655 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:49.429544 26258 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260924 14:30:49.429986 26255 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:49.395866 30025 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:30:49.371737 29942 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:49.432476 26257 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260924 14:30:49.371685 29442 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:49.432226 27126 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:49.429544 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:49.456075 26254 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:49.457916 26252 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:30:49.458631 26250 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:49.462283 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 46096.5us
W20260924 14:30:49.462985 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 54116.9us
I20260924 14:30:49.434582 30054 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:49.442113 29886 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:49.475705 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 107739us
I20260924 14:30:49.476886 26252 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:49.478165 26256 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: []
W20260924 14:30:49.478976 26297 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.123s (SENT)
I20260924 14:30:49.480649 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:49.442248 29762 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260924 14:30:49.462330 30171 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:49.498071 30171 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:49.499115 30171 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:49.485771 30091 raft_consensus.cc:3032] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:49.500989 30091 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:49.495254 29830 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:49.422701 29998 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:49.464272 29442 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:49.510627 26255 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:49.469625 30054 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:49.464982 30231 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:49.521054 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 52167.4us
I20260924 14:30:49.479382 29682 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:30:49.534196 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:49.515333 29762 consensus_queue.cc:1225] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260924 14:30:49.518672 29830 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:49.545323 30270 raft_consensus.cc:2741] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
W20260924 14:30:49.548875 30302 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260924 14:30:49.555572 29971 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:30:49.545360 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:49.535179 30054 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:49.569325 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 105532us
W20260924 14:30:49.578603 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 59383.2us
I20260924 14:30:49.586848 27194 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:49.587608 27194 raft_consensus.cc:595] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:30:49.587957 27194 raft_consensus.cc:603] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:49.560185 30025 raft_consensus.cc:465] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Not starting pre-election -- already a leader
I20260924 14:30:49.573685 29682 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:49.604825 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 59579.6us
I20260924 14:30:49.556377 29442 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:49.578100 30309 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:30:49.625134 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:49.583390 30033 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260924 14:30:49.646391 26326 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:30:49.658449 26329 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:30:49.659149 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.660447 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.660812 26332 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:49.661574 26326 raft_consensus.cc:2379] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:49.663142 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.674890 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.675705 26328 raft_consensus.cc:2362] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12. Re-sending same reply.
I20260924 14:30:49.675920 26330 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:49.676724 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:49.677294 26330 raft_consensus.cc:2362] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10. Re-sending same reply.
I20260924 14:30:49.678017 26399 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:30:49.678138 29716 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:49.676656 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.679563 26328 raft_consensus.cc:2362] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9. Re-sending same reply.
I20260924 14:30:49.680333 29716 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:49.624305 30048 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:49.681056 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:49.688041 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 67973.1us
I20260924 14:30:49.688426 26324 raft_consensus.cc:2362] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10. Re-sending same reply.
I20260924 14:30:49.678176 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.684577 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:49.690429 26331 raft_consensus.cc:2362] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8. Re-sending same reply.
I20260924 14:30:49.691100 30281 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:30:49.641922 30309 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:49.704298 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:49.707464 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 103604us
W20260924 14:30:49.689283 30266 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.050s	user 0.003s	sys 0.000s
I20260924 14:30:49.681695 30084 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:30:49.743474 26371 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.742s (SENT)
I20260924 14:30:49.744642 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:49.767479 26427 debug-util.cc:398] Leaking SignalData structure 0x7b0800377520 after lost signal to thread 23138
W20260924 14:30:49.768910 26427 debug-util.cc:398] Leaking SignalData structure 0x7b080037fae0 after lost signal to thread 26115
W20260924 14:30:49.770268 26427 debug-util.cc:398] Leaking SignalData structure 0x7b080032b000 after lost signal to thread 26282
W20260924 14:30:49.772848 26427 debug-util.cc:398] Leaking SignalData structure 0x7b08001a9280 after lost signal to thread 26356
W20260924 14:30:49.773151 26427 debug-util.cc:398] Leaking SignalData structure 0x7b0800155e60 after lost signal to thread 26357
W20260924 14:30:49.774632 26427 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5c80 after lost signal to thread 26430
W20260924 14:30:49.788157 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48575.2us
W20260924 14:30:49.791694 26427 debug-util.cc:398] Leaking SignalData structure 0x7b08002be220 after lost signal to thread 27222
I20260924 14:30:49.631356 30231 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:49.814738 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 72054.2us
I20260924 14:30:49.816977 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:49.800822 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:49.805591 30091 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:49.810546 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:49.823148 26427 debug-util.cc:398] Leaking SignalData structure 0x7b0800122ae0 after lost signal to thread 30250
I20260924 14:30:49.770720 30084 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:49.702869 30048 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:49.762126 30077 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:30:49.742826 30286 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:30:49.886023 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 93751.2us
I20260924 14:30:49.773330 29998 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:49.821698 26332 raft_consensus.cc:2362] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12. Re-sending same reply.
I20260924 14:30:49.876441 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:49.711469 30280 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.108s
I20260924 14:30:49.878671 26328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "64e0b4058f18400dab940d3832586746"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "7e09fbebe29a49d291d23f9d395fe335"
 from {username='slave'} at 127.0.0.1:32930
W20260924 14:30:49.879431 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 58130.8us
I20260924 14:30:49.710970 29834 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
W20260924 14:30:49.904587 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 79944.5us
W20260924 14:30:49.880259 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 88846.8us
I20260924 14:30:49.823556 29716 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:49.884004 30304 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260924 14:30:49.917233 26221 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:49.884262 30091 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:49.913426 30311 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260924 14:30:49.918907 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:49.761183 29682 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:50.034145 30264 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.112s	user 0.005s	sys 0.005s
I20260924 14:30:49.964902 30309 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:30:50.062985 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 144166us
W20260924 14:30:50.123042 30306 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.499s	user 0.001s	sys 0.001s
W20260924 14:30:50.131299 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 66237.1us
W20260924 14:30:50.130229 30305 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.445s	user 0.001s	sys 0.000s
W20260924 14:30:50.150328 30171 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.000s	sys 0.015s
W20260924 14:30:50.157976 26371 leader_election.cc:334] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.940s (SENT)
W20260924 14:30:50.145957 30308 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.491s	user 0.000s	sys 0.001s
I20260924 14:30:50.130561 30225 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260924 14:30:50.160404 30171 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.000s	sys 0.018s
I20260924 14:30:50.144673 30309 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260924 14:30:50.197854 30313 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.487s	user 0.003s	sys 0.000s
I20260924 14:30:50.204303 30048 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:50.270221 30312 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
W20260924 14:30:50.273491 30332 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260924 14:30:50.275924 26327 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260924 14:30:50.276353 26327 pending_rounds.cc:76] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 5
I20260924 14:30:50.276543 26327 pending_rounds.cc:97] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260924 14:30:50.276796 26327 raft_consensus.cc:2866] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:30:50.276536 30328 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260924 14:30:50.279079 26369 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:50.281055 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:50.282500 30322 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.458s	user 0.001s	sys 0.000s
I20260924 14:30:50.282644 26331 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260924 14:30:50.283047 26331 pending_rounds.cc:76] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 8
I20260924 14:30:50.283259 26331 pending_rounds.cc:97] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 14.9
W20260924 14:30:50.286868 30321 log.cc:924] Time spent T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Append to log took a long time: real 0.514s	user 0.001s	sys 0.000s
I20260924 14:30:50.287163 26331 raft_consensus.cc:2866] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:50.287802 29942 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:50.284284 30171 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
W20260924 14:30:50.284837 30330 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260924 14:30:50.284970 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:50.285373 30324 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.398s	user 0.001s	sys 0.000s
W20260924 14:30:50.289669 29830 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.717s	user 0.009s	sys 0.002s
I20260924 14:30:50.285490 30048 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:30:50.290977 30171 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:50.285925 30317 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.385s	user 0.000s	sys 0.001s
I20260924 14:30:50.290606 29830 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:50.286460 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1401) took 1023 ms. Trace:
I20260924 14:30:50.286700 30091 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:50.293551 30091 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:50.293243 26253 rpcz_store.cc:276] 0924 14:30:49.262637 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:49.458060 (+195423us) service_pool.cc:227] Handling call
0924 14:30:49.458458 (+   398us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:49.465838 (+  7380us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:49.465867 (+    29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:30:49.492756 (+ 26889us) log.cc:842] Serialized 52 byte log entry
0924 14:30:49.606388 (+113632us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:49.633453 (+ 27065us) log.cc:842] Serialized 14 byte log entry
0924 14:30:49.639184 (+  5731us) log.cc:842] Serialized 14 byte log entry
0924 14:30:49.641941 (+  2757us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:49.642020 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:50.286244 (+644224us) raft_consensus.cc:1671] finished
0924 14:30:50.286264 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:50.286402 (+   138us) inbound_call.cc:173] Queueing success response
0924 14:30:50.287381 (+   979us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b18002e0960. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f
0924 14:30:50.288829 (+  1448us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b4800000fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
Metrics: {"mutex_wait_us":22997,"prepare.queue_time_us":158966,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":4363,"raft.queue_time_us":14155,"raft.run_cpu_time_us":11904,"raft.run_wall_time_us":660028,"spinlock_wait_cycles":23309440,"thread_start_us":106173,"threads_started":1,"wal-append.queue_time_us":166973}
I20260924 14:30:50.296555 30091 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:50.286222 30254 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:50.303501 29716 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:50.292311 30171 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:50.297365 30331 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260924 14:30:50.288298 29942 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:50.297688 30054 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.716s	user 0.011s	sys 0.000s
I20260924 14:30:50.298629 26329 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:50.300017 29830 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:50.289636 26332 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:50.325042 26401 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:50.323143 26329 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:50.373003 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:50.390540 26325 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:50.369833 26329 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:50.392407 26329 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:50.369680 30054 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:50.383224 30280 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260924 14:30:50.439921 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1299) took 1164 ms. Trace:
W20260924 14:30:50.440810 30326 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
W20260924 14:30:50.446710 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.018s	sys 0.078s
W20260924 14:30:50.441555 30327 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
W20260924 14:30:50.447291 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.019s	sys 0.078s
I20260924 14:30:50.443539 26250 rpcz_store.cc:276] 0924 14:30:49.275993 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:49.459394 (+183401us) service_pool.cc:227] Handling call
0924 14:30:49.459692 (+   298us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:49.622681 (+162989us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:30:49.792612 (+169931us) log.cc:842] Serialized 14 byte log entry
0924 14:30:50.202381 (+409769us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:50.206374 (+  3993us) log.cc:842] Serialized 26 byte log entry
0924 14:30:50.258627 (+ 52253us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:50.261903 (+  3276us) log.cc:842] Serialized 14 byte log entry
0924 14:30:50.262202 (+   299us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:50.262322 (+   120us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:50.277812 (+ 15490us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b180041fe60. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:30:50.439638 (+161826us) raft_consensus.cc:1671] finished
0924 14:30:50.439663 (+    25us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:50.439841 (+   178us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":18683,"prepare.queue_time_us":65182,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":86100,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":13254,"spinlock_wait_cycles":15462144,"thread_start_us":609569,"threads_started":4,"wal-append.queue_time_us":442162}
I20260924 14:30:50.456630 26401 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
W20260924 14:30:50.457676 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 49981.6us
W20260924 14:30:50.458545 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 393463us
W20260924 14:30:50.459550 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 502996us
I20260924 14:30:50.487282 27185 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: []
W20260924 14:30:50.490649 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 309170us
I20260924 14:30:50.495829 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:50.509902 27192 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:50.516394 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:50.516476 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:50.517469 27189 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:50.520624 27188 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260924 14:30:50.523881 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:50.537051 27192 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:50.509146 30302 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260924 14:30:50.563663 26332 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:50.563378 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:30:50.566591 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 50304.3us
I20260924 14:30:50.563289 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:50.574847 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 125059us
I20260924 14:30:50.581926 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:50.582603 26326 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:50.582551 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:50.587625 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:50.522158 29830 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
W20260924 14:30:50.616950 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 173379us
I20260924 14:30:50.629882 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:50.631493 27192 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260924 14:30:50.640501 26326 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:50.552832 30271 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260924 14:30:50.613270 29942 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:50.654307 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 130901us
I20260924 14:30:50.668186 30346 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:30:50.682022 26328 raft_consensus.cc:595] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Received request to transfer leadership to TS 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:50.682621 26328 raft_consensus.cc:603] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:30:50.682934 30366 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:50.687577 30352 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.135s
I20260924 14:30:50.690640 30341 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.166s
W20260924 14:30:50.693321 30350 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260924 14:30:50.695933 30357 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260924 14:30:50.698524 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 95393.7us
I20260924 14:30:50.703703 26399 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
W20260924 14:30:50.733466 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49827.2us
W20260924 14:30:50.757236 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 175053us
I20260924 14:30:50.759155 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:50.668933 30340 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:30:50.759016 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 59223.7us
I20260924 14:30:50.762845 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 768) took 1111 ms. Trace:
W20260924 14:30:50.765883 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 132945us
I20260924 14:30:50.679180 30054 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:50.681234 30225 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:50.780179 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:50.780472 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:50.780965 27192 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:50.780684 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:50.782377 27123 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:50.787990 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:50.783548 26295 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:50.763116 29682 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260924 14:30:50.784020 30346 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:30:50.850747 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 67322.7us
I20260924 14:30:50.821275 29942 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:50.785877 30225 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:50.861618 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 98036.2us
I20260924 14:30:50.817668 30054 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260924 14:30:50.894605 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 112527us
I20260924 14:30:50.822196 30171 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
W20260924 14:30:50.894812 30254 consensus_meta.cc:287] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.593s	user 0.014s	sys 0.014s
I20260924 14:30:50.907398 30254 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:50.851058 30077 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260924 14:30:50.929697 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:50.896055 29942 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:50.900162 30033 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:50.957266 30338 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:51.144255 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.004s	sys 0.015s
W20260924 14:30:51.111790 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 323906us
I20260924 14:30:51.146582 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:51.114117 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 317200us
I20260924 14:30:50.899158 30352 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:50.963336 30356 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260924 14:30:50.901055 30091 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:30:50.898882 30345 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:51.126266 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 320151us
W20260924 14:30:51.141603 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 212019us
W20260924 14:30:51.144709 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.004s	sys 0.015s
I20260924 14:30:50.952682 30376 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:51.157876 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:51.158385 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:51.158718 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:51.118237 29830 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:51.163534 30376 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:51.178519 30033 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:51.164412 30352 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.196089 30376 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:51.180896 29830 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:51.230873 30369 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:51.154752 30338 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.205571 30391 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:51.214516 30077 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:30:51.260219 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 113712us
W20260924 14:30:51.260870 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 113062us
I20260924 14:30:51.268177 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:51.270857 30033 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.272286 30315 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:30:51.270915 30379 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.266s	user 0.005s	sys 0.007s
I20260924 14:30:51.275699 30400 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260924 14:30:51.270920 30368 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.312s	user 0.004s	sys 0.006s
I20260924 14:30:51.276304 29998 raft_consensus.cc:979] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:51.268204 30391 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:51.276954 29998 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:51.277683 29998 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:51.278208 29942 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:51.278632 30391 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:51.278826 29942 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:51.279270 30385 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:51.279520 29942 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:51.279678 30352 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:51.279951 30385 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.280130 29942 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.279692 29834 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260924 14:30:51.169014 30345 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.280608 30274 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:51.269771 30077 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:51.288206 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:51.336791 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 159823us
W20260924 14:30:51.343116 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 54990.1us
I20260924 14:30:51.276191 30400 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:51.269434 30171 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election won for term 9
W20260924 14:30:51.281514 30358 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.501s	user 0.000s	sys 0.007s
I20260924 14:30:51.251716 30369 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.283046 30084 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:51.328554 30364 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.206s	user 0.000s	sys 0.010s
I20260924 14:30:51.396661 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:51.281419 30351 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:51.318429 30371 raft_consensus.cc:1067] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:51.288689 30225 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:51.347136 30385 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:51.332587 30274 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:51.368602 30352 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:51.516572 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1139) took 1351 ms. Trace:
I20260924 14:30:51.362519 30400 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:51.517235 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:51.267935 30279 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
W20260924 14:30:51.521304 26371 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.923s (SENT)
I20260924 14:30:51.570809 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:51.572041 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50774.3us
W20260924 14:30:51.572866 26371 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.015s (SENT)
W20260924 14:30:51.573822 26371 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.042s (SENT)
I20260924 14:30:51.574249 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:51.527069 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 130502us
W20260924 14:30:51.576953 26371 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.045s (SENT)
W20260924 14:30:51.578465 26371 leader_election.cc:334] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.148s (SENT)
W20260924 14:30:51.549963 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 391605us
I20260924 14:30:51.550061 27185 rpcz_store.cc:276] 0924 14:30:50.165036 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:50.165237 (+   201us) service_pool.cc:227] Handling call
0924 14:30:50.188530 (+ 23293us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:50.488781 (+300251us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:51.449023 (+960242us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b540000f354. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:51.516503 (+ 67480us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":686553472}
I20260924 14:30:51.558687 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 766) took 1915 ms. Trace:
W20260924 14:30:51.516572 30331 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.069s	user 0.007s	sys 0.000s
I20260924 14:30:51.581671 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:51.582440 26371 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.143s (SENT)
I20260924 14:30:51.583084 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:51.583920 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:51.585691 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:51.586978 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:51.580739 26327 rpcz_store.cc:276] 0924 14:30:49.643094 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:49.643245 (+   151us) service_pool.cc:227] Handling call
0924 14:30:49.681200 (+ 37955us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:50.277200 (+596000us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:51.558394 (+1281194us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b540012e654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:51.558621 (+   227us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1366645120}
I20260924 14:30:51.539283 29834 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:51.523283 30171 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:51.620500 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:51.525216 30281 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:51.525153 30403 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:51.636801 30389 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:51.643290 30403 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:30:51.644522 30403 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:51.645210 30171 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:51.646723 30281 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:51.647742 30281 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:51.648319 30338 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:51.648859 30338 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:51.649250 30338 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.598537 30388 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:51.643242 30389 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.653373 30389 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:51.644533 30372 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
I20260924 14:30:51.553686 30391 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:51.655782 30391 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.658170 30391 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:51.589514 30225 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:51.531648 30071 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.106s
I20260924 14:30:51.679956 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1348) took 1306 ms. Trace:
I20260924 14:30:51.681397 26330 rpcz_store.cc:276] 0924 14:30:50.373523 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:50.373652 (+   129us) service_pool.cc:227] Handling call
0924 14:30:50.374039 (+   387us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:50.399591 (+ 25552us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:30:50.399639 (+    48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:50.412853 (+ 13214us) log.cc:842] Serialized 26 byte log entry
0924 14:30:50.621581 (+208728us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:50.621636 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:50.621725 (+    89us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:51.679676 (+1057951us) raft_consensus.cc:1671] finished
0924 14:30:51.679706 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:51.679881 (+   175us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10543,"raft.queue_time_us":19044,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"thread_start_us":167959,"threads_started":1,"wal-append.queue_time_us":440619}
I20260924 14:30:51.651190 30338 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:51.716735 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 190988us
I20260924 14:30:51.579162 30352 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:51.730717 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 58168.8us
I20260924 14:30:51.678613 30280 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:51.601970 30315 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260924 14:30:51.718092 30414 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:51.717708 30225 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:51.637907 29834 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:51.551904 30351 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:51.785753 26369 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.008s	sys 0.019s
W20260924 14:30:51.789265 26370 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.017s	sys 0.011s
W20260924 14:30:51.795254 26369 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.008s	sys 0.019s
W20260924 14:30:51.795547 26370 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.018s	sys 0.011s
W20260924 14:30:51.795620 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 636972us
W20260924 14:30:51.792558 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 205632us
I20260924 14:30:51.751817 30279 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:51.804752 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.805354 30406 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:51.805181 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:51.809928 30377 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260924 14:30:51.810225 30406 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:51.810601 26257 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:51.805903 30407 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:51.811146 30369 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:51.806087 30352 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:51.812081 30352 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:51.812546 30352 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.813369 30274 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:51.813529 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:51.813963 30274 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:30:51.814883 30352 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:51.808765 30401 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
I20260924 14:30:51.809166 30225 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:51.811563 30407 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:51.805842 30336 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.018s	sys 0.064s
I20260924 14:30:51.814165 26404 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:30:51.815587 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:51.816326 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:51.806061 30394 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:51.817117 26250 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:51.817169 30351 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:51.751190 29942 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:51.821772 26400 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:30:51.821913 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:51.718216 30371 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260924 14:30:51.823673 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.825541 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.826254 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.820760 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:51.811240 30406 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:51.854400 30352 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:51.867388 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.854429 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:51.886399 26258 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:51.749095 29830 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:51.859684 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:51.789778 30345 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:51.863433 26257 raft_consensus.cc:2362] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13. Re-sending same reply.
I20260924 14:30:51.864574 26254 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:51.896318 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:51.896775 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.854662 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.870100 26253 raft_consensus.cc:2379] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:51.884037 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.898567 26250 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
I20260924 14:30:51.899802 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.889833 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.901044 26249 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:51.865502 26251 raft_consensus.cc:2396] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 12 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260924 14:30:51.897006 26257 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:51.805235 30315 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election won for term 11
W20260924 14:30:51.866981 30336 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.018s	sys 0.069s
I20260924 14:30:51.841262 30077 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:51.934773 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:51.931803 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 43875.3us
I20260924 14:30:51.933178 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:51.933833 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 345017us
I20260924 14:30:51.868355 30279 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:51.863265 30414 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:51.941416 26254 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:30:51.929282 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:51.957338 27181 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:51.812439 30415 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260924 14:30:51.958822 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:51.937634 26252 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:51.959117 26324 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
W20260924 14:30:51.991219 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 236198us
I20260924 14:30:51.955971 30315 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:51.940651 30406 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:51.980268 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:51.993247 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:52.011955 26257 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:51.950415 30336 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:52.022356 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:51.889707 30352 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:52.035962 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:52.040840 26257 raft_consensus.cc:2362] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14. Re-sending same reply.
I20260924 14:30:52.029528 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:52.048946 26324 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:52.048825 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:52.052445 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:52.052615 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 182016us
I20260924 14:30:52.055136 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:51.999315 30396 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:52.082123 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:52.091846 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:52.094700 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:52.095183 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:52.097791 26327 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:51.885273 30394 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:52.107724 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:52.098814 26330 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:30:52.114725 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:51.964224 30279 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:52.120730 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65867.1us
I20260924 14:30:51.996542 30315 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:51.981585 30414 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:52.057202 30071 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260924 14:30:52.145660 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:52.151413 26252 raft_consensus.cc:2396] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260924 14:30:52.048324 30336 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:52.130887 30437 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:52.131292 30423 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:52.140003 30420 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:52.150643 30425 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:52.156044 30281 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:52.190449 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 95845.5us
I20260924 14:30:52.091580 30396 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:52.246261 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:52.288625 30077 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:52.289192 30077 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:52.289639 30077 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:52.312093 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:52.233007 29834 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:52.334323 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:52.295416 30407 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:52.356479 26221 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:52.237135 30281 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:52.189400 30423 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:52.189801 30437 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:52.392161 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 145955us
I20260924 14:30:52.234078 30420 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:52.377480 30407 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:52.238234 30425 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:52.380008 30352 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:52.402877 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 291870us
W20260924 14:30:52.395428 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 242743us
I20260924 14:30:52.399235 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1344) took 2108 ms. Trace:
I20260924 14:30:52.445225 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:52.443704 26325 rpcz_store.cc:276] 0924 14:30:50.282145 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:50.293145 (+ 11000us) service_pool.cc:227] Handling call
0924 14:30:50.293450 (+   305us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:50.391122 (+ 97672us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:52.390737 (+1999615us) spinlock_profiling.cc:243] Waited 96.7 ms on lock 0x7b540001e5d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:52.390934 (+   197us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":222371200}
I20260924 14:30:52.388356 30171 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:52.450477 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1350) took 1676 ms. Trace:
I20260924 14:30:52.401409 30430 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:52.451076 26329 rpcz_store.cc:276] 0924 14:30:50.774100 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:50.774285 (+   185us) service_pool.cc:227] Handling call
0924 14:30:50.774733 (+   448us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:50.775016 (+   283us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:30:50.805862 (+ 30846us) log.cc:842] Serialized 14 byte log entry
0924 14:30:51.891014 (+1085152us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b18004231c0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6423155 00007f09b93463d0 00007f09b9358576 00007f09b93690d7 00007f09b9369058
0924 14:30:52.303533 (+412519us) log.cc:842] Serialized 14 byte log entry
0924 14:30:52.450076 (+146543us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:52.450156 (+    80us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:52.450221 (+    65us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:52.450292 (+    71us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:52.450419 (+   127us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1255,"prepare.queue_time_us":302135,"prepare.run_cpu_time_us":2935,"prepare.run_wall_time_us":22785,"raft.queue_time_us":120159,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":21241,"spinlock_wait_cycles":5230848,"thread_start_us":556843,"threads_started":3}
W20260924 14:30:52.447232 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 90869.5us
W20260924 14:30:52.450120 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 491137us
I20260924 14:30:52.458968 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:52.475498 26258 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.589s	user 0.012s	sys 0.000s
I20260924 14:30:52.397262 30227 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:52.477221 26258 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:52.493685 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:52.495252 26255 pending_rounds.cc:76] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 6
I20260924 14:30:52.495653 26255 pending_rounds.cc:97] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260924 14:30:52.495910 26255 raft_consensus.cc:2866] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:52.448289 30416 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:52.452184 30430 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:52.497395 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:52.513000 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:52.510998 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:52.519662 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:52.520468 26252 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:52.533962 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:52.447621 30407 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:52.557044 30392 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260924 14:30:52.557283 30419 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:52.557924 30419 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:52.558055 30387 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260924 14:30:52.507427 30336 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:52.560393 30419 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:52.560791 30426 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260924 14:30:52.561167 30410 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:52.456094 30171 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:52.561754 30410 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:52.562729 30404 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.232s
W20260924 14:30:52.563656 30395 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.008s	sys 0.030s
W20260924 14:30:52.564092 30395 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.008s	sys 0.030s
I20260924 14:30:52.567545 30423 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:52.568418 30463 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:52.568912 30467 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:30:52.511606 30428 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:52.525426 30279 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:52.569607 30467 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:52.569895 30428 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:52.570945 30467 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:52.568969 30463 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:52.572232 30428 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:52.572916 30428 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:52.573283 30428 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:52.573652 30428 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:52.573868 30463 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:52.562153 30409 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:52.574383 30463 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:52.574508 30409 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:52.575631 30457 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:52.512105 30227 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:52.575721 26257 raft_consensus.cc:2362] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9. Re-sending same reply.
I20260924 14:30:52.512091 30352 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:52.574885 30463 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:52.580289 26254 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.006s
W20260924 14:30:52.584021 26254 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.000s	sys 0.006s
W20260924 14:30:52.584988 26297 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.000s	sys 0.008s
W20260924 14:30:52.585392 26297 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.000s	sys 0.008s
W20260924 14:30:52.585705 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 594969us
I20260924 14:30:52.556381 30394 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:52.576336 30457 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:52.619990 30470 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:52.620805 30227 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:52.621466 30457 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:52.563695 30171 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:52.623900 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1497) took 2252 ms. Trace:
I20260924 14:30:52.624332 26401 rpcz_store.cc:276] 0924 14:30:50.371602 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:50.371741 (+   139us) service_pool.cc:227] Handling call
0924 14:30:50.397956 (+ 26215us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:50.498619 (+100663us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:52.623599 (+2124980us) spinlock_profiling.cc:243] Waited 57 ms on lock 0x7b54002fbcd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:52.623796 (+   197us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":131086080}
I20260924 14:30:52.625821 30463 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:52.627274 30425 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:52.629065 30420 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:52.630375 30437 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:52.631656 30396 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:52.581343 30414 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:52.632735 30443 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:30:52.637588 30376 consensus_meta.cc:287] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.389s	user 0.015s	sys 0.000s
W20260924 14:30:52.640708 27181 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.673s	user 0.009s	sys 0.002s
I20260924 14:30:52.648319 27181 raft_consensus.cc:2396] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260924 14:30:52.607775 26297 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.881s (SENT) [suppressed 42 similar messages]
I20260924 14:30:52.650458 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:52.650715 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:52.651178 26329 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:52.651401 26325 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:52.658360 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:52.659065 26327 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:52.665961 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:52.668732 26330 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:52.583369 30352 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:52.615274 30281 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:52.675343 30436 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260924 14:30:52.673163 30414 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260924 14:30:52.695151 30414 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:52.695538 30414 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:52.696374 30414 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:52.697688 30414 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:52.674507 30400 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.054s	user 0.009s	sys 0.005s
I20260924 14:30:52.725051 30275 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:52.725603 30275 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:52.714956 30428 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:52.726447 30275 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:52.728332 30428 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:52.665112 30376 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:52.733178 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 54353.2us
I20260924 14:30:52.734695 26258 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:30:52.731863 30376 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:52.668413 30443 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:52.738621 30443 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:52.739892 30407 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:52.740724 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:52.692186 30410 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:52.720608 30400 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:52.752750 30400 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:52.571413 30467 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:52.785655 30384 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.336s	user 0.000s	sys 0.004s
I20260924 14:30:52.787520 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1352) took 2011 ms. Trace:
I20260924 14:30:52.787992 26332 rpcz_store.cc:276] 0924 14:30:50.777347 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:50.777854 (+   507us) service_pool.cc:227] Handling call
0924 14:30:50.778199 (+   345us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:50.778635 (+   436us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:50.778659 (+    24us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:50.779113 (+   454us) log.cc:842] Serialized 26 byte log entry
0924 14:30:50.820022 (+ 40909us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:50.820061 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:50.820128 (+    67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:52.747122 (+1926994us) spinlock_profiling.cc:243] Waited 96.5 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:52.787075 (+ 39953us) raft_consensus.cc:1671] finished
0924 14:30:52.787105 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:52.787424 (+   319us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":71806,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":8846,"spinlock_wait_cycles":221839104,"thread_start_us":40503,"threads_started":1,"wal-append.queue_time_us":490610}
W20260924 14:30:52.793378 30393 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.484s	user 0.002s	sys 0.000s
I20260924 14:30:52.787294 30077 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:52.799141 27125 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.031s	sys 0.057s
I20260924 14:30:52.782506 30275 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:52.801157 27192 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.020s	sys 0.025s
W20260924 14:30:52.803701 27192 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.020s	sys 0.025s
W20260924 14:30:52.802248 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.031s	sys 0.057s
W20260924 14:30:52.806878 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 805022us
I20260924 14:30:52.807319 30371 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:52.807706 30025 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260924 14:30:52.802639 27188 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.002s	sys 0.043s
W20260924 14:30:52.818073 27188 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.002s	sys 0.044s
W20260924 14:30:52.817822 26297 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.109s (SENT)
I20260924 14:30:52.820103 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:52.821822 30077 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:52.826401 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:52.827229 26252 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
W20260924 14:30:52.803468 30406 consensus_meta.cc:287] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.788s	user 0.013s	sys 0.000s
W20260924 14:30:52.848431 26251 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.003s	sys 0.017s
W20260924 14:30:52.848932 26251 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.004s	sys 0.017s
I20260924 14:30:52.878469 26252 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
I20260924 14:30:52.879244 26252 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:52.879664 26252 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:52.883431 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:52.884384 26256 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
W20260924 14:30:52.891722 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 75133.9us
I20260924 14:30:52.892899 30339 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:52.893647 30339 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:52.896109 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:52.899191 30352 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:52.901227 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 101045us
I20260924 14:30:52.903170 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:52.903748 26330 pending_rounds.cc:76] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 6
I20260924 14:30:52.904086 26330 pending_rounds.cc:97] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260924 14:30:52.904290 26330 raft_consensus.cc:2866] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:52.907774 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:52.909981 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:52.939015 30339 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:52.850531 30237 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:52.938894 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:52.946110 30333 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:52.940034 30274 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260924 14:30:52.946940 30333 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:52.949815 30333 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:52.954469 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:52.877669 30077 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:53.001726 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62917.2us
I20260924 14:30:53.005757 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:53.008188 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 53808.9us
I20260924 14:30:53.008178 27188 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:30:53.013824 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:53.016665 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:53.017808 26256 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:52.939868 30437 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:53.023862 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:53.019973 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:52.935534 30091 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.079s
I20260924 14:30:53.022532 30445 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:53.015180 30339 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:53.022897 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:52.845046 30446 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:52.851164 30386 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:52.938144 30451 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260924 14:30:53.029618 26296 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.011s	sys 0.049s
I20260924 14:30:53.012504 30333 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:53.042930 27185 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
W20260924 14:30:53.046159 26296 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.012s	sys 0.049s
I20260924 14:30:53.041755 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:53.050621 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:53.033526 26252 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:53.031016 30370 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 1.769s	user 0.005s	sys 0.006s
W20260924 14:30:53.057094 26249 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.925s	user 0.016s	sys 0.015s
W20260924 14:30:53.061512 26249 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.016s	sys 0.015s
I20260924 14:30:53.062419 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:53.062587 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:53.004842 30237 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:53.031872 30352 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:53.068868 30366 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:53.068706 30352 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:53.069471 30366 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:53.071028 30352 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:53.066588 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:53.072976 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:53.074218 30396 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:53.074645 30396 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:53.075031 30396 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:53.075798 30414 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:53.076265 30414 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:53.067283 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:53.067975 30410 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:53.064832 30471 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:53.067283 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32930 (request call id 33) took 3298 ms. Trace:
W20260924 14:30:53.085927 26255 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.022s	sys 0.036s
I20260924 14:30:53.079649 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:53.082015 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.016s	sys 0.028s
I20260924 14:30:53.052428 30333 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:53.040815 30437 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:53.084430 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:53.085032 26328 rpcz_store.cc:276] 0924 14:30:49.789007 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:49.819151 (+ 30144us) service_pool.cc:227] Handling call
0924 14:30:53.067075 (+3247924us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54005582d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b9348bc3 00007f09be29281a 00007f09b7985d04 00007f09b7985c92 00007f09b7985bc1 00007f09b7985b3c 00007f09b7984df2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:53.067189 (+   114us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1793796352}
I20260924 14:30:53.076673 30414 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:53.086123 26252 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:53.088420 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:53.078692 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.008s	sys 0.029s
W20260924 14:30:53.091183 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.016s	sys 0.028s
W20260924 14:30:53.091881 26255 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.022s	sys 0.036s
W20260924 14:30:53.106348 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 608890us
I20260924 14:30:53.109277 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.112964 26402 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:53.098207 26256 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260924 14:30:53.112716 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.009s	sys 0.030s
I20260924 14:30:53.015690 30077 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:53.124552 30403 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:53.131894 27125 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:53.133850 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:53.135433 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:53.040357 30445 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:53.162547 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.097827 30333 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:53.087407 30410 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:53.190594 30410 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:53.163365 29677 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:30:53.112583 30467 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:53.121991 30349 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:53.193790 30349 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:53.194248 30349 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:53.164497 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:53.165997 26403 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 4.
W20260924 14:30:53.119225 30033 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.845s	user 0.024s	sys 0.034s
I20260924 14:30:53.167059 26252 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
W20260924 14:30:53.128334 30315 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.000s	sys 0.032s
I20260924 14:30:53.169467 30354 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:53.170500 30343 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:53.171850 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.172676 30237 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:53.174374 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.175798 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.176596 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:53.177111 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:53.222559 26372 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.040s	sys 0.065s
W20260924 14:30:53.223099 26372 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.078s	user 0.040s	sys 0.065s
I20260924 14:30:53.177367 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.049635 30386 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:53.178478 30469 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.261s	user 0.000s	sys 0.000s
I20260924 14:30:53.178920 27181 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:53.178931 30465 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:53.225574 30343 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:53.229068 30465 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:53.180308 30437 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:53.180887 30476 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260924 14:30:53.113270 30366 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:53.089190 30471 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:53.186715 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1001) took 1357 ms. Trace:
I20260924 14:30:53.317603 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.320477 26251 rpcz_store.cc:276] 0924 14:30:51.829763 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:51.939600 (+109837us) service_pool.cc:227] Handling call
0924 14:30:51.939972 (+   372us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:51.956396 (+ 16424us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:51.956434 (+    38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:51.985368 (+ 28934us) log.cc:842] Serialized 26 byte log entry
0924 14:30:52.849423 (+864055us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:52.912111 (+ 62688us) log.cc:842] Serialized 14 byte log entry
0924 14:30:52.912396 (+   285us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:52.912465 (+    69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:53.186412 (+273947us) raft_consensus.cc:1671] finished
0924 14:30:53.186443 (+    31us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:53.186631 (+   188us) inbound_call.cc:173] Queueing success response
0924 14:30:53.194562 (+  7931us) spinlock_profiling.cc:243] Waited 9.61 ms on lock 0x7b1800286cc0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11
0924 14:30:53.195947 (+  1385us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b1800286cc0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
Metrics: {"mutex_wait_us":15993,"prepare.queue_time_us":62429,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":21733,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":22098,"spinlock_wait_cycles":111155712,"thread_start_us":792781,"threads_started":1,"wal-append.queue_time_us":1168674}
I20260924 14:30:53.097573 29442 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260924 14:30:53.197847 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 62540.3us
I20260924 14:30:53.197203 30445 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:53.201140 30388 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:53.046660 30446 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:53.209250 26252 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:53.214109 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:53.204066 30033 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.930s	user 0.024s	sys 0.034s
I20260924 14:30:53.218250 26401 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
W20260924 14:30:53.216454 29981 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.021s	sys 0.038s
I20260924 14:30:53.230775 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.396996 26403 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:53.233851 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:53.217741 30315 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.000s	sys 0.033s
I20260924 14:30:53.400589 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:53.216852 30354 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:53.402797 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:53.317791 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 68637.7us
I20260924 14:30:53.286139 30323 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:53.415210 26252 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:53.226040 30077 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:53.323225 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:53.333859 26326 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.035s	sys 0.040s
W20260924 14:30:53.334656 26329 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.010s	sys 0.037s
I20260924 14:30:53.389249 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:53.419777 26329 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.010s	sys 0.037s
I20260924 14:30:53.390270 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.420975 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:53.422219 26398 raft_consensus.cc:2396] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260924 14:30:53.317636 30437 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:53.423808 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:53.405964 26250 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
W20260924 14:30:53.423192 26326 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.035s	sys 0.041s
I20260924 14:30:53.330965 30388 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:53.330713 29442 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:30:53.396469 29981 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.163s	user 0.022s	sys 0.038s
W20260924 14:30:53.415650 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.015s	sys 0.098s
I20260924 14:30:53.383113 30025 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:53.428505 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.015s	sys 0.098s
W20260924 14:30:53.429091 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 918116us
I20260924 14:30:53.423033 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:53.405714 30354 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:53.426373 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:53.407492 30397 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:53.429898 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:53.430449 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.433634 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1199) took 1080 ms. Trace:
W20260924 14:30:53.562049 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 132250us
I20260924 14:30:53.563555 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:53.430907 26332 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:53.531108 30025 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:53.427973 29442 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:53.531250 30343 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:53.548722 30465 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:53.550386 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:53.531734 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:53.540756 26397 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:53.332382 30284 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260924 14:30:53.230095 30472 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.183s
W20260924 14:30:53.561139 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 134845us
I20260924 14:30:53.649964 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:53.574800 30467 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:53.422909 30387 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:53.566622 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:53.562168 26406 rpcz_store.cc:276] 0924 14:30:52.346213 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:52.346453 (+   240us) service_pool.cc:227] Handling call
0924 14:30:52.375424 (+ 28971us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:52.375888 (+   464us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:52.375935 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:52.387888 (+ 11953us) log.cc:842] Serialized 26 byte log entry
0924 14:30:52.600740 (+212852us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:52.600801 (+    61us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:52.600908 (+   107us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:53.426022 (+825114us) raft_consensus.cc:1671] finished
0924 14:30:53.426050 (+    28us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:53.426201 (+   151us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":74192,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5435,"thread_start_us":144715,"threads_started":1,"wal-append.queue_time_us":401972}
W20260924 14:30:53.673167 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 106613us
I20260924 14:30:53.428931 30388 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:53.549459 30476 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260924 14:30:53.563572 26331 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.018s	sys 0.036s
W20260924 14:30:53.678896 26331 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.018s	sys 0.037s
I20260924 14:30:50.767534 26323 rpcz_store.cc:276] 0924 14:30:49.643850 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:49.658500 (+ 14650us) service_pool.cc:227] Handling call
0924 14:30:49.699273 (+ 40773us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:49.699724 (+   451us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:49.699760 (+    36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:49.708401 (+  8641us) log.cc:842] Serialized 26 byte log entry
0924 14:30:50.184535 (+476134us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b18001b25c0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:30:50.395818 (+211283us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:50.610144 (+214326us) log.cc:842] Serialized 14 byte log entry
0924 14:30:50.681829 (+ 71685us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:50.681956 (+   127us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:50.697519 (+ 15563us) raft_consensus.cc:1671] finished
0924 14:30:50.755245 (+ 57726us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:50.755492 (+   247us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":18304,"prepare.queue_time_us":285018,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":41132,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":64777,"spinlock_wait_cycles":6997888,"thread_start_us":420215,"threads_started":3,"wal-append.queue_time_us":811355}
I20260924 14:30:53.573876 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:53.554409 30271 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.889s	user 0.026s	sys 0.044s
I20260924 14:30:53.582765 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:53.680747 26332 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:53.422489 30275 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.142s
I20260924 14:30:53.559505 30171 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:53.423097 30323 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:53.646899 26255 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.006s	sys 0.023s
W20260924 14:30:53.703666 26255 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.006s	sys 0.024s
I20260924 14:30:53.433634 30437 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:53.580875 30410 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:53.580591 30397 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:53.682855 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:53.627063 30474 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:53.723023 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:53.684360 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:30:53.724262 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:53.726450 26332 raft_consensus.cc:2396] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260924 14:30:53.721424 26328 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260924 14:30:53.682761 30271 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 3.022s	user 0.026s	sys 0.044s
W20260924 14:30:53.688902 26327 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.015s	sys 0.019s
I20260924 14:30:53.724949 26331 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
W20260924 14:30:53.741129 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 56881.6us
W20260924 14:30:53.741837 26327 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.016s	sys 0.020s
I20260924 14:30:53.722100 30410 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:53.683866 30471 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:53.703120 30374 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.035s	user 0.012s	sys 0.020s
W20260924 14:30:53.722563 30320 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.017s	sys 0.040s
I20260924 14:30:53.750442 26405 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:30:53.751540 26328 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260924 14:30:53.719481 30459 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:53.760999 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1413) took 1929 ms. Trace:
I20260924 14:30:53.578197 30482 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.156s
I20260924 14:30:53.754136 30048 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260924 14:30:53.755916 30374 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.468s	user 0.013s	sys 0.021s
I20260924 14:30:53.770309 30344 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:53.761536 26254 rpcz_store.cc:276] 0924 14:30:51.840287 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:51.980028 (+139741us) service_pool.cc:227] Handling call
0924 14:30:51.981077 (+  1049us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:51.981645 (+   568us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:51.981718 (+    73us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:52.014535 (+ 32817us) log.cc:842] Serialized 26 byte log entry
0924 14:30:52.584444 (+569909us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:52.616247 (+ 31803us) log.cc:842] Serialized 14 byte log entry
0924 14:30:52.616442 (+   195us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:52.616517 (+    75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:53.760775 (+1144258us) raft_consensus.cc:1671] finished
0924 14:30:53.760801 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:53.760939 (+   138us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":29748,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":4294,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":569501,"threads_started":1,"wal-append.queue_time_us":1050748}
I20260924 14:30:53.750576 30391 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260924 14:30:53.781414 30048 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:53.800344 30048 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:53.757596 30320 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.018s	sys 0.040s
W20260924 14:30:53.782392 30483 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260924 14:30:53.783179 30486 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
W20260924 14:30:53.783967 30487 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260924 14:30:53.787679 30478 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:53.745913 30446 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:53.683105 30387 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:53.752738 30410 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:53.797578 26323 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260924 14:30:53.816229 26323 pending_rounds.cc:76] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 6
I20260924 14:30:53.816543 26323 pending_rounds.cc:97] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260924 14:30:53.816847 26323 raft_consensus.cc:2866] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:30:53.818674 30489 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260924 14:30:53.819203 30491 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260924 14:30:53.754801 30271 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:53.820523 30271 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:53.821059 30445 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:53.752907 30386 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:53.734488 30171 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:53.822119 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 777) took 1166 ms. Trace:
I20260924 14:30:53.676162 30467 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:53.812963 30407 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:53.830766 30430 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:53.831111 30437 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:53.754868 30474 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:53.813418 30025 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:53.823390 26325 rpcz_store.cc:276] 0924 14:30:52.656187 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:52.658671 (+  2484us) service_pool.cc:227] Handling call
0924 14:30:52.659127 (+   456us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:52.670167 (+ 11040us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:52.670209 (+    42us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:52.670707 (+   498us) log.cc:842] Serialized 26 byte log entry
0924 14:30:53.090511 (+419804us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:53.624203 (+533692us) log.cc:842] Serialized 14 byte log entry
0924 14:30:53.626653 (+  2450us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:53.626728 (+    75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:53.821911 (+195183us) raft_consensus.cc:1671] finished
0924 14:30:53.821931 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:53.822062 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":21447,"prepare.queue_time_us":19977,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":14900,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":25584,"spinlock_wait_cycles":1030528,"thread_start_us":916013,"threads_started":2,"wal-append.queue_time_us":984853}
W20260924 14:30:53.833515 29716 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.000s	sys 0.048s
I20260924 14:30:53.832363 30275 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:53.855669 27188 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.024s	sys 0.028s
I20260924 14:30:53.803242 30344 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:53.834568 30467 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
W20260924 14:30:53.849278 30396 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.774s	user 0.012s	sys 0.000s
I20260924 14:30:53.827225 30445 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:53.853856 30333 consensus_meta.cc:287] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.665s	user 0.012s	sys 0.000s
I20260924 14:30:53.864066 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 997) took 2035 ms. Trace:
W20260924 14:30:53.914054 27185 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.028s	sys 0.045s
W20260924 14:30:53.914530 27185 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.028s	sys 0.045s
I20260924 14:30:53.914341 26249 rpcz_store.cc:276] 0924 14:30:51.828581 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:51.901839 (+ 73258us) service_pool.cc:227] Handling call
0924 14:30:51.953144 (+ 51305us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:51.972354 (+ 19210us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:52.008605 (+ 36251us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:52.040119 (+ 31514us) log.cc:842] Serialized 26 byte log entry
0924 14:30:53.062320 (+1022201us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:53.071842 (+  9522us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4800000fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:30:53.580934 (+509092us) log.cc:842] Serialized 14 byte log entry
0924 14:30:53.625284 (+ 44350us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:53.625372 (+    88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:53.863717 (+238345us) raft_consensus.cc:1671] finished
0924 14:30:53.863840 (+   123us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:53.863997 (+   157us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8150,"prepare.queue_time_us":133759,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":41,"raft.queue_time_us":162762,"raft.run_cpu_time_us":3992,"raft.run_wall_time_us":1089340,"spinlock_wait_cycles":25339776,"thread_start_us":1422554,"threads_started":2,"wal-append.queue_time_us":1310757}
W20260924 14:30:53.864845 30349 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.670s	user 0.005s	sys 0.005s
I20260924 14:30:53.860837 30275 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:53.865348 30430 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:53.843616 30280 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:53.855144 29716 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.000s	sys 0.049s
W20260924 14:30:53.866786 30414 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.674s	user 0.006s	sys 0.004s
W20260924 14:30:53.868085 30488 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260924 14:30:53.872078 30171 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:53.872959 30323 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:53.915977 30286 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.025s	sys 0.023s
W20260924 14:30:53.986624 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 299693us
I20260924 14:30:53.877974 30271 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:53.884003 30484 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260924 14:30:53.888829 30387 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:53.843982 30407 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:53.895701 27188 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.024s	sys 0.028s
W20260924 14:30:53.991387 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.009s	sys 0.013s
W20260924 14:30:53.991865 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.009s	sys 0.013s
I20260924 14:30:53.892545 30410 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:53.950268 30430 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:30:53.984772 30286 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.025s	sys 0.023s
W20260924 14:30:53.939079 27192 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.018s	sys 0.062s
W20260924 14:30:54.008060 27192 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.018s	sys 0.062s
W20260924 14:30:53.939541 26372 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.015s	sys 0.034s
W20260924 14:30:54.008984 26372 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.015s	sys 0.035s
W20260924 14:30:53.940079 26330 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.022s	user 0.023s	sys 0.048s
W20260924 14:30:54.010207 26330 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.092s	user 0.023s	sys 0.049s
W20260924 14:30:54.001248 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 591284us
I20260924 14:30:53.887750 30388 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:53.913677 30396 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:54.018828 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 584211us
I20260924 14:30:53.992630 30315 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:53.939323 30333 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:53.949671 30349 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:54.025535 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:54.024009 30387 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:54.012204 30432 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:53.982545 30280 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:53.984426 30414 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:54.036479 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 775) took 1381 ms. Trace:
I20260924 14:30:54.037388 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:54.042760 26402 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:54.051375 26326 rpcz_store.cc:276] 0924 14:30:52.655543 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:52.655659 (+   116us) service_pool.cc:227] Handling call
0924 14:30:52.655944 (+   285us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:52.670507 (+ 14563us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:52.670536 (+    29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:30:52.671030 (+   494us) log.cc:842] Serialized 52 byte log entry
0924 14:30:53.424767 (+753737us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:53.537486 (+112719us) log.cc:842] Serialized 14 byte log entry
0924 14:30:54.026569 (+489083us) log.cc:842] Serialized 14 byte log entry
0924 14:30:54.035944 (+  9375us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.036166 (+   222us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:54.036221 (+    55us) raft_consensus.cc:1671] finished
0924 14:30:54.036237 (+    16us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.036408 (+   171us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":126404,"prepare.queue_time_us":125110,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":1400,"raft.queue_time_us":168879,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":22482,"spinlock_wait_cycles":35960704,"thread_start_us":1237270,"threads_started":2,"wal-append.queue_time_us":1151395}
I20260924 14:30:54.029124 30509 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:54.036125 30406 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:54.064939 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.069172 26254 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:30:54.066592 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:54.070233 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 328071us
I20260924 14:30:54.070742 26251 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
W20260924 14:30:54.078929 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 399784us
W20260924 14:30:54.079903 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 54455.8us
I20260924 14:30:54.080893 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:54.082012 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52643.7us
I20260924 14:30:54.029485 30315 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:54.076400 30493 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260924 14:30:54.052731 30387 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:54.086376 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1004) took 1627 ms. Trace:
I20260924 14:30:54.100986 27186 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:54.104921 26255 rpcz_store.cc:276] 0924 14:30:52.458988 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:52.459244 (+   256us) service_pool.cc:227] Handling call
0924 14:30:52.495066 (+ 35822us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:52.513268 (+ 18202us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:52.513322 (+    54us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:52.523250 (+  9928us) log.cc:842] Serialized 26 byte log entry
0924 14:30:53.111251 (+588001us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:53.704150 (+592899us) log.cc:842] Serialized 14 byte log entry
0924 14:30:54.074070 (+369920us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.074242 (+   172us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:54.074326 (+    84us) raft_consensus.cc:1671] finished
0924 14:30:54.074457 (+   131us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.086186 (+ 11729us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":85065,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":54845,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":18734,"thread_start_us":1549510,"threads_started":3,"wal-append.queue_time_us":884463}
I20260924 14:30:54.127562 27186 raft_consensus.cc:595] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:54.127828 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:54.128904 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.130199 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.132405 26255 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:54.134250 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.135842 26252 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:54.138518 26249 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260924 14:30:54.139917 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:54.127575 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:54.140769 26254 raft_consensus.cc:2362] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10. Re-sending same reply.
I20260924 14:30:54.084347 30519 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:54.154624 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 774) took 1499 ms. Trace:
I20260924 14:30:54.049503 30430 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:54.132846 29981 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.003s	sys 0.016s
W20260924 14:30:54.234984 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 198916us
I20260924 14:30:54.223524 30445 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:54.237987 30396 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:54.179603 30519 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:54.145183 30354 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.565s	user 0.013s	sys 0.003s
I20260924 14:30:54.239054 26251 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:54.215884 30381 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:54.153481 30518 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:54.240922 30518 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:54.157451 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 71997.4us
I20260924 14:30:54.244243 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.245982 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.246219 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.246665 26254 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
W20260924 14:30:54.235694 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 179783us
I20260924 14:30:54.239727 30354 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:54.249876 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.250378 30354 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:54.251060 27190 raft_consensus.cc:2354] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10.
I20260924 14:30:54.250932 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:54.251736 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 161703us
I20260924 14:30:54.252781 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 709) took 1188 ms. Trace:
I20260924 14:30:54.252614 26254 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:54.253499 27185 rpcz_store.cc:276] 0924 14:30:53.064116 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.085179 (+ 21063us) service_pool.cc:227] Handling call
0924 14:30:53.085473 (+   294us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:53.085693 (+   220us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:30:53.915057 (+829364us) log.cc:842] Serialized 14 byte log entry
0924 14:30:54.252297 (+337240us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:54.252378 (+    81us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:54.252448 (+    70us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.252550 (+   102us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.252708 (+   158us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":56706,"prepare.queue_time_us":968319,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":150800,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":1108950,"threads_started":3}
I20260924 14:30:54.255986 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.258564 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.259066 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:54.260771 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.251571 30354 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:54.262336 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.096009 30033 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:30:54.074064 30509 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:54.269032 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:54.174947 30344 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:54.192113 29442 consensus_meta.cc:287] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.001s
I20260924 14:30:54.299419 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:54.330910 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61576.3us
I20260924 14:30:54.268169 27186 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:54.298061 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1370) took 1220 ms. Trace:
I20260924 14:30:54.335422 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.163017 26324 rpcz_store.cc:276] 0924 14:30:52.655193 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:52.655343 (+   150us) service_pool.cc:227] Handling call
0924 14:30:52.655740 (+   397us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:52.660146 (+  4406us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:52.660191 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:52.660779 (+   588us) log.cc:842] Serialized 26 byte log entry
0924 14:30:53.093628 (+432849us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:53.123820 (+ 30192us) log.cc:842] Serialized 14 byte log entry
0924 14:30:53.124277 (+   457us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:53.125516 (+  1239us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:54.154280 (+1028764us) raft_consensus.cc:1671] finished
0924 14:30:54.154303 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.154558 (+   255us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":32859,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":16803,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":36865920,"thread_start_us":432430,"threads_started":1,"wal-append.queue_time_us":915628}
I20260924 14:30:54.337852 27185 raft_consensus.cc:2396] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260924 14:30:54.310948 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:54.328447 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:54.298784 29981 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.005s	sys 0.023s
I20260924 14:30:54.336210 26325 rpcz_store.cc:276] 0924 14:30:53.077723 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.847018 (+769295us) service_pool.cc:227] Handling call
0924 14:30:53.847424 (+   406us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:53.847734 (+   310us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:30:53.857362 (+  9628us) log.cc:842] Serialized 14 byte log entry
0924 14:30:54.297227 (+439865us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:54.297305 (+    78us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:54.297381 (+    76us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.297511 (+   130us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.297944 (+   433us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":12794,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":17101,"thread_start_us":439270,"threads_started":2}
I20260924 14:30:54.176661 30510 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:54.351783 30510 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:54.178372 29989 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.167s
I20260924 14:30:54.268684 30407 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:54.270779 30354 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:54.357159 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:54.365203 26325 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:54.162334 30286 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:54.366796 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.300514 30410 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:54.373335 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.374119 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:54.346879 30493 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260924 14:30:54.379664 30499 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
W20260924 14:30:54.380062 30048 consensus_meta.cc:287] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.579s	user 0.010s	sys 0.002s
I20260924 14:30:54.380543 30414 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:54.382666 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.354928 30430 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:54.381508 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 707) took 1338 ms. Trace:
I20260924 14:30:54.390933 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.395723 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:54.397456 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:54.398869 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:54.368676 30354 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:54.363227 30333 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:54.355347 30388 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:54.393491 27188 rpcz_store.cc:276] 0924 14:30:53.043636 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.043772 (+   136us) service_pool.cc:227] Handling call
0924 14:30:53.044169 (+   397us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:53.044620 (+   451us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:53.044651 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:53.045214 (+   563us) log.cc:842] Serialized 26 byte log entry
0924 14:30:53.989434 (+944220us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:53.989485 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:53.989561 (+    76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:54.381236 (+391675us) raft_consensus.cc:1671] finished
0924 14:30:54.381260 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.381443 (+   183us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":308,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":943817,"threads_started":1,"wal-append.queue_time_us":1038974}
I20260924 14:30:54.348064 29442 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:54.435334 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 705) took 1410 ms. Trace:
I20260924 14:30:54.376124 30280 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:54.429464 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 776) took 1773 ms. Trace:
W20260924 14:30:54.414366 30495 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
W20260924 14:30:54.435218 30398 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 3.173s	user 0.006s	sys 0.005s
I20260924 14:30:54.437899 27192 rpcz_store.cc:276] 0924 14:30:53.024902 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.025050 (+   148us) service_pool.cc:227] Handling call
0924 14:30:53.047306 (+ 22256us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:53.053423 (+  6117us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:53.053458 (+    35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:53.053910 (+   452us) log.cc:842] Serialized 26 byte log entry
0924 14:30:54.008554 (+954644us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:30:54.008613 (+    59us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.008707 (+    94us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:54.435132 (+426425us) raft_consensus.cc:1671] finished
0924 14:30:54.435153 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.435276 (+   123us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4181,"raft.queue_time_us":7172,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":4305,"thread_start_us":916150,"threads_started":1,"wal-append.queue_time_us":1282825}
I20260924 14:30:54.444775 26329 rpcz_store.cc:276] 0924 14:30:52.655878 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:52.658652 (+  2774us) service_pool.cc:227] Handling call
0924 14:30:52.659075 (+   423us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:52.672132 (+ 13057us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:52.672171 (+    39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:52.677107 (+  4936us) log.cc:842] Serialized 26 byte log entry
0924 14:30:53.420257 (+743150us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:53.432808 (+ 12551us) log.cc:842] Serialized 14 byte log entry
0924 14:30:54.011300 (+578492us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.011415 (+   115us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:54.429217 (+417802us) raft_consensus.cc:1671] finished
0924 14:30:54.429236 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.429400 (+   164us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2913,"prepare.queue_time_us":115747,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":132152,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":9004,"spinlock_wait_cycles":23808,"thread_start_us":1164017,"threads_started":2,"wal-append.queue_time_us":997813}
I20260924 14:30:54.445354 30510 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:54.376565 30407 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260924 14:30:54.474700 27190 raft_consensus.cc:2354] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:30:54.481400 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:54.481911 30514 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:54.482649 30514 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:54.483078 30514 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:54.380589 30048 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:54.423056 30518 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:54.508709 30515 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:54.509339 30515 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:54.509783 30515 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:54.509869 30048 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:54.510007 30498 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260924 14:30:54.377525 30286 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:54.525859 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:54.515282 30532 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260924 14:30:54.514997 30345 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:54.557226 30345 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:54.557750 30345 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:30:54.515161 30501 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
W20260924 14:30:54.521152 30524 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260924 14:30:54.532503 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:54.536275 30497 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260924 14:30:54.577525 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.578204 27186 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:54.540824 30387 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:54.542017 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.579764 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1359) took 1681 ms. Trace:
I20260924 14:30:54.580235 26330 rpcz_store.cc:276] 0924 14:30:52.898269 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:52.903198 (+  4929us) service_pool.cc:227] Handling call
0924 14:30:52.903564 (+   366us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:52.904763 (+  1199us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:52.904792 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:52.918160 (+ 13368us) log.cc:842] Serialized 26 byte log entry
0924 14:30:54.011263 (+1093103us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:54.011321 (+    58us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.011419 (+    98us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:54.579534 (+568115us) raft_consensus.cc:1671] finished
0924 14:30:54.579554 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.579700 (+   146us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":1092776,"threads_started":1,"wal-append.queue_time_us":1150262}
I20260924 14:30:54.581014 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.585222 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 998) took 2756 ms. Trace:
I20260924 14:30:54.585702 26253 rpcz_store.cc:276] 0924 14:30:51.828868 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:51.906809 (+ 77941us) service_pool.cc:227] Handling call
0924 14:30:51.931508 (+ 24699us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:51.946397 (+ 14889us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:51.946443 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:51.946999 (+   556us) log.cc:842] Serialized 26 byte log entry
0924 14:30:53.074600 (+1127601us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b180004c840. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:30:53.567250 (+492650us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:53.587366 (+ 20116us) log.cc:842] Serialized 14 byte log entry
0924 14:30:53.990916 (+403550us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:53.991018 (+   102us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:54.584901 (+593883us) raft_consensus.cc:1671] finished
0924 14:30:54.585011 (+   110us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.585140 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":52155,"prepare.queue_time_us":19792,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":142852,"raft.run_cpu_time_us":5013,"raft.run_wall_time_us":91996,"spinlock_wait_cycles":9784576,"thread_start_us":854067,"threads_started":2,"wal-append.queue_time_us":1996687}
I20260924 14:30:54.579291 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:54.589838 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.585285 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1305) took 1409 ms. Trace:
I20260924 14:30:54.595633 26402 rpcz_store.cc:276] 0924 14:30:53.198463 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.198601 (+   138us) service_pool.cc:227] Handling call
0924 14:30:53.202831 (+  4230us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:54.053344 (+850513us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.585075 (+531731us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b54001c1e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:54.585230 (+   155us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1931126656}
I20260924 14:30:54.545050 30354 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:54.597304 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.598256 26253 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:30:54.599948 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:54.591434 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.512948 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.552546 29442 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:54.554147 30375 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260924 14:30:54.558971 26258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1008) took 2098 ms (client timeout 1978 ms). Trace:
I20260924 14:30:54.561942 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:30:54.611943 26258 rpcz_store.cc:269] 0924 14:30:52.460498 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:52.494781 (+ 34283us) service_pool.cc:227] Handling call
0924 14:30:54.558766 (+2063985us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400381dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:54.558895 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":458035584}
I20260924 14:30:54.574129 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:54.612336 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50539.2us
I20260924 14:30:54.615624 26325 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:54.618268 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.621135 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.625845 26224 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:54.629019 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:54.628471 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 59993us
I20260924 14:30:54.631300 30521 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260924 14:30:54.632169 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:54.632777 30498 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260924 14:30:54.640362 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:54.642079 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.648556 27192 raft_consensus.cc:2379] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 13: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:54.648603 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.635679 30509 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:54.676990 26253 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:54.685339 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 45080.7us
I20260924 14:30:54.687348 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:54.687888 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:54.689085 26253 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 10
W20260924 14:30:54.646381 30459 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.870s	user 0.013s	sys 0.000s
I20260924 14:30:54.692803 30387 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:54.693358 30387 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:54.645220 30410 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:30:54.693744 30387 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:54.642495 30375 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:54.699460 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1332) took 1693 ms. Trace:
I20260924 14:30:54.651319 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:54.705444 26256 rpcz_store.cc:276] 0924 14:30:53.006057 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.125185 (+119128us) service_pool.cc:227] Handling call
0924 14:30:54.559540 (+1434355us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:54.559725 (+   185us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:54.559750 (+    25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:54.559796 (+    46us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:54.559835 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.559894 (+    59us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.699202 (+139308us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400381dd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:54.699370 (+   168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3291420544}
I20260924 14:30:54.692859 29981 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:54.676704 30521 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:54.720312 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.740125 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.704897 30410 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:54.708250 30375 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:54.765746 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.770972 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.709339 30459 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:54.708408 30509 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:54.772505 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 54384.7us
I20260924 14:30:54.710974 30349 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:54.738327 29981 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:54.710642 30541 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:54.764825 30467 consensus_meta.cc:287] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.849s	user 0.012s	sys 0.000s
I20260924 14:30:54.775439 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.765266 30521 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:54.772347 30410 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:54.783607 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.719370 30054 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260924 14:30:54.793879 26326 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:54.786921 30496 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.003s	user 0.003s	sys 0.001s
I20260924 14:30:54.795778 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:54.808591 27185 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:54.811594 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.743110 30522 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260924 14:30:54.778559 30509 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:54.809386 30374 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.022s	sys 0.024s
I20260924 14:30:54.787209 30467 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:54.843230 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.808709 30286 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:54.839572 30287 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.031s	sys 0.055s
I20260924 14:30:54.852589 26326 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:54.853308 27186 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:54.784910 30541 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:54.855037 30354 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:54.854748 30287 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.032s	sys 0.056s
W20260924 14:30:54.858734 30531 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.085s	user 0.001s	sys 0.001s
W20260924 14:30:54.866899 26371 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.022s	sys 0.031s
W20260924 14:30:54.855960 30534 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260924 14:30:54.856473 27125 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.009s	sys 0.025s
W20260924 14:30:54.868752 26327 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.013s	sys 0.021s
W20260924 14:30:54.837281 30374 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.022s	sys 0.025s
W20260924 14:30:54.869371 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.011s	sys 0.018s
W20260924 14:30:54.873381 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.011s	sys 0.018s
W20260924 14:30:54.871762 26371 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.022s	sys 0.031s
W20260924 14:30:54.872931 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.009s	sys 0.025s
I20260924 14:30:54.822916 29981 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:54.877128 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 568705us
W20260924 14:30:54.876616 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 627368us
W20260924 14:30:54.871510 26254 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.014s	sys 0.021s
I20260924 14:30:54.877907 26253 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
W20260924 14:30:54.879678 26254 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.014s	sys 0.021s
I20260924 14:30:54.879953 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:54.873725 26327 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.013s	sys 0.022s
I20260924 14:30:54.886020 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1368) took 1860 ms. Trace:
I20260924 14:30:54.899358 26253 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:54.909538 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:54.904428 26371 leader_election.cc:334] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.074s (SENT)
I20260924 14:30:54.910918 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:54.913250 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:54.915403 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:54.923328 26325 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:30:54.925859 27194 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.011s	sys 0.022s
W20260924 14:30:54.928359 27194 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.012s	sys 0.023s
I20260924 14:30:54.916319 26329 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:54.935086 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:54.936012 26325 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:54.937390 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:54.952822 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:54.914672 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:54.961891 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:54.962399 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.967705 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:54.969100 26397 raft_consensus.cc:2396] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260924 14:30:54.971175 26295 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.011s	sys 0.056s
I20260924 14:30:54.971558 26402 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:54.886451 26327 rpcz_store.cc:276] 0924 14:30:53.026024 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.778429 (+752405us) service_pool.cc:227] Handling call
0924 14:30:53.801762 (+ 23333us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:53.802043 (+   281us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:30:53.814492 (+ 12449us) log.cc:842] Serialized 14 byte log entry
0924 14:30:54.885643 (+1071151us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:54.885708 (+    65us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:30:54.885754 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.885820 (+    66us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:54.885951 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":47443,"prepare.queue_time_us":11151,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":3297,"raft.queue_time_us":195331,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":1949,"spinlock_wait_cycles":7187328,"thread_start_us":1010840,"threads_started":2,"wal-append.queue_time_us":896116}
W20260924 14:30:54.971617 26295 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.012s	sys 0.056s
W20260924 14:30:54.974835 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.015s	sys 0.027s
W20260924 14:30:54.975484 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.015s	sys 0.027s
I20260924 14:30:54.974821 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:54.978615 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.950965 29442 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:54.985589 27189 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.028s	sys 0.049s
W20260924 14:30:54.987694 27189 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.028s	sys 0.049s
I20260924 14:30:54.988405 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:54.989398 26249 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:54.990557 26258 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:54.997020 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:54.998657 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.000078 26329 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:55.001881 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.006107 26325 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260924 14:30:55.010797 26329 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:55.011994 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.014102 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:55.015142 26257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1007) took 2529 ms (client timeout 2074 ms). Trace:
W20260924 14:30:55.042822 26257 rpcz_store.cc:269] 0924 14:30:52.460224 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:52.512672 (+ 52448us) service_pool.cc:227] Handling call
0924 14:30:54.986974 (+2474302us) spinlock_profiling.cc:243] Waited 60.2 ms on lock 0x7b54007b07d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:54.989989 (+  3015us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":138553216}
I20260924 14:30:55.044368 26257 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:54.964046 30522 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:30:54.968151 29629 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:55.045987 29629 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:54.982127 30531 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260924 14:30:55.048506 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 782) took 2026 ms. Trace:
I20260924 14:30:54.987813 30538 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:30:54.992753 29989 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.016s	sys 0.057s
I20260924 14:30:55.031935 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:54.960023 30389 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:55.049269 26332 rpcz_store.cc:276] 0924 14:30:53.022335 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.761576 (+739241us) service_pool.cc:227] Handling call
0924 14:30:53.762002 (+   426us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:53.762277 (+   275us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:53.762312 (+    35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:53.779635 (+ 17323us) log.cc:842] Serialized 26 byte log entry
0924 14:30:54.083407 (+303772us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:54.100633 (+ 17226us) log.cc:842] Serialized 14 byte log entry
0924 14:30:54.291540 (+190907us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:54.291661 (+   121us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:55.048249 (+756588us) raft_consensus.cc:1671] finished
0924 14:30:55.048270 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:55.048425 (+   155us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5329,"prepare.queue_time_us":6839,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":242464,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":13934,"thread_start_us":404917,"threads_started":2,"wal-append.queue_time_us":902697}
W20260924 14:30:55.036810 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 638469us
I20260924 14:30:55.038008 27185 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:54.958273 30513 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260924 14:30:55.031294 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:54.981091 30405 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260924 14:30:55.055390 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.063617 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:55.059352 29989 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.016s	sys 0.058s
I20260924 14:30:55.058789 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:55.083330 26257 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
W20260924 14:30:55.068210 30515 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.558s	user 0.005s	sys 0.008s
I20260924 14:30:55.085897 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.062088 26252 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:55.076795 26253 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 13
I20260924 14:30:55.058543 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:55.056262 30389 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:55.056833 30033 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:55.016600 29442 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:55.089021 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.066473 30351 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:55.047050 29629 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.057055 30509 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:55.074226 30520 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:30:55.070891 30271 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:55.068775 30467 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:55.081900 30371 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:55.066062 30227 consensus_queue.cc:1046] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260924 14:30:55.091380 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.091779 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:55.112708 27186 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:55.099108 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.098877 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:55.114192 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:55.112355 27185 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:55.108788 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:55.117138 26402 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:55.117121 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:55.090260 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.000s	sys 0.035s
I20260924 14:30:55.051924 30538 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:55.124109 26253 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:55.096854 30054 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:30:55.101028 30520 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:55.091763 30389 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:55.111768 30467 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:55.129726 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.150537 26398 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260924 14:30:55.152158 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:55.154919 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.155167 27126 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:55.162421 30547 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260924 14:30:55.101419 30351 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.165346 29442 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:55.165482 30351 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:55.137730 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:55.168066 26325 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:55.168918 30538 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:55.169895 30514 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.002s
I20260924 14:30:55.170382 30514 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:55.125517 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.000s	sys 0.035s
I20260924 14:30:55.171778 30520 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.173142 30520 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:55.170840 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.109813 30333 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260924 14:30:55.091522 30515 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:55.144176 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 1.640s (SENT)
I20260924 14:30:55.146762 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:55.181128 26295 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:55.172390 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 773565us
I20260924 14:30:55.109112 30271 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.182899 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:55.183149 26295 leader_election.cc:334] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 1.801s (SENT)
I20260924 14:30:55.184988 26295 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.200359 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:55.193114 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 413506us
I20260924 14:30:55.212242 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1429) took 1083 ms. Trace:
I20260924 14:30:55.212952 27186 raft_consensus.cc:2396] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260924 14:30:55.140069 30541 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:55.231683 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:55.230558 26330 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:55.212773 26251 rpcz_store.cc:276] 0924 14:30:54.128490 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.128631 (+   141us) service_pool.cc:227] Handling call
0924 14:30:55.211988 (+1083357us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54004fae54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:55.212167 (+   179us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":251666176}
I20260924 14:30:55.186972 30227 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:30:55.247550 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 46069.9us
I20260924 14:30:55.236681 30558 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:55.190694 29442 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:30:55.254374 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.254845 26297 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:30:55.255661 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 57623.7us
I20260924 14:30:55.256193 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.146446 30467 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:55.200012 30549 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.115s
W20260924 14:30:55.292639 27181 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 708) took 2236 ms (client timeout 2105 ms). Trace:
I20260924 14:30:55.295256 26251 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:55.294554 27188 raft_consensus.cc:3032] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:55.300817 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 999) took 3471 ms. Trace:
I20260924 14:30:55.293362 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 792) took 1027 ms. Trace:
I20260924 14:30:55.189531 30551 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:55.313364 27188 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:55.234326 30518 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:55.257308 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:55.304466 30554 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260924 14:30:55.313827 26326 rpcz_store.cc:276] 0924 14:30:54.266052 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.266200 (+   148us) service_pool.cc:227] Handling call
0924 14:30:55.293180 (+1026980us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b5400505854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:55.293298 (+   118us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1046343040}
W20260924 14:30:55.292371 30556 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260924 14:30:55.336279 26398 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
W20260924 14:30:55.297315 30387 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.588s	user 0.006s	sys 0.005s
I20260924 14:30:55.336886 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.338894 26402 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:55.319375 27188 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.266424 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 689) took 1123 ms. Trace:
I20260924 14:30:55.348177 27188 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:55.348384 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:55.348256 27182 rpcz_store.cc:276] 0924 14:30:54.142688 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.159361 (+ 16673us) service_pool.cc:227] Handling call
0924 14:30:54.215353 (+ 55992us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:54.641167 (+425814us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:54.641197 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:54.671483 (+ 30286us) log.cc:842] Serialized 26 byte log entry
0924 14:30:55.056469 (+384986us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:30:55.073341 (+ 16872us) log.cc:842] Serialized 14 byte log entry
0924 14:30:55.078604 (+  5263us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:55.078684 (+    80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:55.266157 (+187473us) raft_consensus.cc:1671] finished
0924 14:30:55.266186 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:55.266348 (+   162us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7481,"prepare.queue_time_us":29389,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":93884,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":25029,"thread_start_us":789353,"threads_started":2,"wal-append.queue_time_us":566523}
I20260924 14:30:55.349784 29682 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:55.350487 29682 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:55.350627 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:55.255236 30345 consensus_meta.cc:287] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.697s	user 0.013s	sys 0.000s
I20260924 14:30:55.352089 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.352198 30345 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.352919 27182 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:55.354988 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:55.293514 30515 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:55.337035 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 80973.1us
I20260924 14:30:55.312915 30339 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:55.416193 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.418650 26369 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:30:55.268592 30552 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260924 14:30:55.311355 27181 rpcz_store.cc:269] 0924 14:30:53.055870 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.056116 (+   246us) service_pool.cc:227] Handling call
0924 14:30:55.292359 (+2236243us) spinlock_profiling.cc:243] Waited 87.6 ms on lock 0x7b54004533d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:55.292521 (+   162us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":201524608}
W20260924 14:30:55.345291 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 90524.8us
I20260924 14:30:55.421500 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:55.254948 30522 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:55.315610 26295 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.311352 30432 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:55.353785 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.313637 26250 rpcz_store.cc:276] 0924 14:30:51.829164 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:51.941881 (+112717us) service_pool.cc:227] Handling call
0924 14:30:51.942191 (+   310us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:53.424596 (+1482405us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:55.300520 (+1875924us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54002882d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:55.300732 (+   212us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3365907328}
I20260924 14:30:55.203877 30553 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:55.388671 26325 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
W20260924 14:30:55.434087 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 118580us
I20260924 14:30:55.427991 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:55.429227 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:55.432785 27182 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:55.439632 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:55.341444 30054 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:30:55.447510 26295 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.354439 30551 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:55.268242 30230 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.150s
I20260924 14:30:55.445392 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:55.261202 30410 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:55.266523 30542 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.166s
I20260924 14:30:55.470890 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 687) took 1382 ms. Trace:
I20260924 14:30:55.464164 30391 consensus_queue.cc:1046] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260924 14:30:55.472782 30391 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:55.473172 30391 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:55.471968 27189 rpcz_store.cc:276] 0924 14:30:54.088334 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.088458 (+   124us) service_pool.cc:227] Handling call
0924 14:30:54.131716 (+ 43258us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:54.243525 (+111809us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:54.243582 (+    57us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:54.269221 (+ 25639us) log.cc:842] Serialized 26 byte log entry
0924 14:30:55.013892 (+744671us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:55.314130 (+300238us) log.cc:842] Serialized 14 byte log entry
0924 14:30:55.318899 (+  4769us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:55.318978 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:55.470262 (+151284us) raft_consensus.cc:1671] finished
0924 14:30:55.470292 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:55.470810 (+   518us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":29479,"prepare.queue_time_us":39060,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":3239,"raft.queue_time_us":133455,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":8611,"spinlock_wait_cycles":22272,"thread_start_us":1033667,"threads_started":3,"wal-append.queue_time_us":809254}
I20260924 14:30:55.473541 30391 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:55.474251 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.425196 30522 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:55.475790 30522 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:55.476686 30423 raft_consensus.cc:3032] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:55.477159 30423 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:55.477283 30417 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:55.477746 30417 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:55.478062 30423 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:55.433885 30541 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:55.480980 30362 raft_consensus.cc:3032] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:55.481441 30362 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:55.482300 30362 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:55.484885 30375 consensus_meta.cc:287] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.004s
I20260924 14:30:55.443884 30432 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:55.485589 30375 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:55.478089 30417 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:55.489571 29981 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:55.345549 30518 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:55.345304 30033 consensus_queue.cc:1046] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260924 14:30:55.494503 30271 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:55.496572 30417 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.497123 30467 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:55.351666 29682 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:55.497874 30345 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:55.498693 30551 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:55.498828 30417 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:55.442607 30549 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:55.446171 30244 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260924 14:30:55.462707 30521 consensus_meta.cc:287] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.671s	user 0.005s	sys 0.007s
I20260924 14:30:55.501760 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 688) took 1406 ms. Trace:
I20260924 14:30:55.455579 30351 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:55.467087 27182 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:55.479305 30423 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:55.446614 30077 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:55.510636 27194 rpcz_store.cc:276] 0924 14:30:54.095010 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.135565 (+ 40555us) service_pool.cc:227] Handling call
0924 14:30:54.135868 (+   303us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:54.176897 (+ 41029us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:54.176931 (+    34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:54.214863 (+ 37932us) log.cc:842] Serialized 26 byte log entry
0924 14:30:54.931820 (+716957us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:54.996449 (+ 64629us) log.cc:842] Serialized 14 byte log entry
0924 14:30:55.052984 (+ 56535us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:55.053512 (+   528us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:55.501455 (+447943us) raft_consensus.cc:1671] finished
0924 14:30:55.501490 (+    35us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:55.501682 (+   192us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":80148,"prepare.queue_time_us":63245,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":171541,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":7335,"thread_start_us":594927,"threads_started":1,"wal-append.queue_time_us":893704}
I20260924 14:30:55.520762 27188 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:55.520810 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.522190 27188 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:55.445111 30515 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260924 14:30:55.487226 30432 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:55.525224 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 57513.6us
I20260924 14:30:55.532294 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:55.534067 30271 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:55.534751 30271 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:55.534962 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:55.535261 30271 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:55.535720 30467 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:30:55.537377 27189 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:55.414839 30387 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:55.514106 30351 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:55.433243 30339 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:55.524410 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1012) took 1300 ms. Trace:
I20260924 14:30:55.545102 26254 rpcz_store.cc:276] 0924 14:30:54.223643 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.257916 (+ 34273us) service_pool.cc:227] Handling call
0924 14:30:54.258637 (+   721us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:54.304409 (+ 45772us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:54.304447 (+    38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:54.304999 (+   552us) log.cc:842] Serialized 26 byte log entry
0924 14:30:54.880198 (+575199us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:55.356617 (+476419us) log.cc:842] Serialized 14 byte log entry
0924 14:30:55.357613 (+   996us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:55.357720 (+   107us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:55.524204 (+166484us) raft_consensus.cc:1671] finished
0924 14:30:55.524223 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:55.524353 (+   130us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44592,"prepare.queue_time_us":424084,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":48244,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":66580,"thread_start_us":976708,"threads_started":2,"wal-append.queue_time_us":758594}
I20260924 14:30:55.547106 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:55.460155 27186 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:55.547979 26253 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:55.547256 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:55.500339 30362 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:55.549923 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:55.554841 26250 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:30:55.554860 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:55.550557 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:55.556211 26252 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:55.557128 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:55.557525 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 129633us
I20260924 14:30:55.558280 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:55.558629 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.552214 30568 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election won for term 13
W20260924 14:30:55.531136 30284 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260924 14:30:55.569032 26401 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:55.570467 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:55.551812 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:55.574120 26257 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:55.531589 30077 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:55.553699 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:55.514503 30521 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:55.576422 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:55.591575 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.549263 30351 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:55.600359 26332 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:55.593277 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.593372 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.592976 30467 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:55.519532 30562 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260924 14:30:55.549670 30548 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:55.607100 30541 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260924 14:30:55.598858 30567 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260924 14:30:55.614441 26256 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:30:55.619230 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 171850us
I20260924 14:30:55.631031 27186 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:55.640869 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.642786 26253 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:55.643828 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.651759 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.645507 30230 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:55.647334 30549 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:55.653628 30230 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.653828 30549 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:55.569087 30566 consensus_queue.cc:1046] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260924 14:30:55.656198 30230 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:55.659864 26253 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:55.613044 30541 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:55.664240 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:55.664626 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 73588.9us
I20260924 14:30:55.666129 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.666843 26328 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:55.664299 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:55.632695 30338 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:55.612723 30467 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:55.671631 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.671955 30230 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:55.672544 30230 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.674571 30230 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:55.671087 30338 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:55.677291 30338 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:55.683519 30077 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:55.685755 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1016) took 1363 ms. Trace:
I20260924 14:30:55.686160 26258 rpcz_store.cc:276] 0924 14:30:54.322408 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.765227 (+442819us) service_pool.cc:227] Handling call
0924 14:30:54.990122 (+224895us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:54.991115 (+   993us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:55.685561 (+694446us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54007b07d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:55.685697 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":470088576}
I20260924 14:30:55.619714 30375 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:55.705166 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.671036 30339 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:55.690042 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:55.691387 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.692134 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:55.693818 30391 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.694566 26254 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:55.711926 26254 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:30:55.696206 30389 consensus_meta.cc:287] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.520s	user 0.005s	sys 0.007s
I20260924 14:30:55.696615 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.696971 30467 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:55.704780 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.706573 26295 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.707417 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 686) took 1619 ms. Trace:
I20260924 14:30:55.708957 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1433) took 1551 ms. Trace:
I20260924 14:30:55.709595 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1346) took 1358 ms. Trace:
I20260924 14:30:55.712679 26254 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.717162 26254 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:55.716693 26249 rpcz_store.cc:276] 0924 14:30:54.351522 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.017602 (+666080us) service_pool.cc:227] Handling call
0924 14:30:55.018155 (+   553us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:55.041839 (+ 23684us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:55.041867 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:55.056654 (+ 14787us) log.cc:842] Serialized 26 byte log entry
0924 14:30:55.317973 (+261319us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:55.318026 (+    53us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:55.318117 (+    91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:55.709360 (+391243us) raft_consensus.cc:1671] finished
0924 14:30:55.709380 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:55.709527 (+   147us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":23092,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":229700,"threads_started":1,"wal-append.queue_time_us":376431}
W20260924 14:30:55.710356 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 72906.2us
I20260924 14:30:55.708817 26329 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:55.707293 29442 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:55.610790 30548 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:55.720675 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:55.715993 27191 rpcz_store.cc:276] 0924 14:30:54.087990 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.088131 (+   141us) service_pool.cc:227] Handling call
0924 14:30:55.302335 (+1214204us) spinlock_profiling.cc:243] Waited 877 us on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eebce 000000000034aea9 00007f09be21aa30 00007f09be2e07b4 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:30:55.302551 (+   216us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:55.303010 (+   459us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:55.303044 (+    34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:55.312664 (+  9620us) log.cc:842] Serialized 26 byte log entry
0924 14:30:55.620899 (+308235us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:55.661298 (+ 40399us) log.cc:842] Serialized 14 byte log entry
0924 14:30:55.688054 (+ 26756us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:55.688125 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:55.688169 (+    44us) raft_consensus.cc:1671] finished
0924 14:30:55.707235 (+ 19066us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:55.707356 (+   121us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26455,"prepare.queue_time_us":30159,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":44778,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":11029,"spinlock_wait_cycles":2028288,"thread_start_us":275252,"threads_started":1,"wal-append.queue_time_us":334570}
I20260924 14:30:55.728380 26221 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.729770 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.730244 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.724224 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.731539 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.732861 26295 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.716086 26255 rpcz_store.cc:276] 0924 14:30:54.157898 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.158453 (+   555us) service_pool.cc:227] Handling call
0924 14:30:54.158918 (+   465us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:54.159349 (+   431us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:54.159380 (+    31us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:30:54.160066 (+   686us) log.cc:842] Serialized 52 byte log entry
0924 14:30:54.734759 (+574693us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:30:55.087696 (+352937us) log.cc:842] Serialized 14 byte log entry
0924 14:30:55.554078 (+466382us) log.cc:842] Serialized 14 byte log entry
0924 14:30:55.708527 (+154449us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:55.708631 (+   104us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:55.708693 (+    62us) raft_consensus.cc:1671] finished
0924 14:30:55.708710 (+    17us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:55.708890 (+   180us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":42924,"prepare.queue_time_us":531248,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":105,"raft.queue_time_us":98364,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":49039,"thread_start_us":1363523,"threads_started":4,"wal-append.queue_time_us":662128}
I20260924 14:30:55.737613 26250 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:55.738076 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.734372 27186 raft_consensus.cc:2396] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260924 14:30:55.740180 26327 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260924 14:30:55.740489 29442 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.741755 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.727344 30389 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:55.743749 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.745303 30548 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:55.745814 30389 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:55.746050 30572 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:55.746454 27185 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:55.746785 30572 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:55.749084 30572 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:55.748978 30347 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:30:55.745396 30339 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:55.736708 30345 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:55.754772 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:55.746157 30574 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:55.755980 30387 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:55.757122 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.757329 30519 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:55.757586 30391 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:55.709560 30410 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:55.758217 30391 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:55.758622 30410 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:55.758689 30391 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:55.759346 30410 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:55.745343 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.777243 27191 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:55.745748 30375 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:55.777863 26326 raft_consensus.cc:2396] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260924 14:30:55.761608 30351 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.668623 30541 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:55.764328 30521 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:55.764050 30574 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:30:55.765043 30549 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:55.765129 26254 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:30:55.800414 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.768689 26328 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:55.770151 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:55.733677 30467 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:30:55.772881 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:55.812915 26258 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:55.774006 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.779603 26325 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:55.789762 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:55.725693 30520 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.552s	user 0.004s	sys 0.009s
I20260924 14:30:55.795352 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.800438 26249 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:55.802286 26255 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:55.766314 30366 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:30:55.734951 30568 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:55.844274 27188 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:30:55.845659 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.846645 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.847540 27189 raft_consensus.cc:2396] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260924 14:30:55.845547 26251 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:55.849701 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:55.850414 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.854178 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.854837 26328 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260924 14:30:55.855175 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.855860 27186 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:30:55.856171 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.857782 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:55.858654 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:55.863510 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.868674 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:55.870208 26324 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:55.812723 30410 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:30:55.877351 26258 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
W20260924 14:30:55.860741 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 131015us
I20260924 14:30:55.881523 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:55.882324 26328 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:55.883104 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.883587 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:55.840713 30521 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:55.839771 30584 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:55.843884 26070 maintenance_manager.cc:419] P 277acd3330ed4b40b2cea74b4e602a13: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033349
I20260924 14:30:55.885823 26406 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260924 14:30:55.884629 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.909463 26406 pending_rounds.cc:76] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 7
I20260924 14:30:55.911123 26406 pending_rounds.cc:97] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260924 14:30:55.912009 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:55.885237 30574 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:30:55.845062 30366 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:55.812947 30522 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:55.924503 30569 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.373s	user 0.003s	sys 0.000s
I20260924 14:30:55.924168 30366 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:55.925551 30522 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:55.926223 26398 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:55.891969 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.893558 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:55.927861 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:55.895126 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.928936 26400 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:55.929689 27186 raft_consensus.cc:2396] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260924 14:30:55.890159 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.933135 26397 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:55.908257 26295 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.911706 26406 raft_consensus.cc:2866] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:30:55.890780 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:55.916693 30568 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:55.953374 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.890252 30582 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:55.962159 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.954743 26295 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.930823 26325 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:30:55.931713 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.921707 30574 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:55.943867 30362 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:55.968791 30574 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:30:55.944984 30585 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:30:55.885125 30577 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260924 14:30:55.912081 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:55.974864 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:55.955062 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.975700 26406 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:30:55.955107 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 64420.3us
I20260924 14:30:55.965938 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:55.977277 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 79761.3us
I20260924 14:30:55.922534 30584 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:55.964059 27189 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:55.966065 26328 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:55.965119 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:55.967808 30588 consensus_queue.cc:1046] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260924 14:30:55.987740 29682 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:30:55.988422 29682 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:55.988685 30589 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:30:55.989004 29682 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:55.960713 29981 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:55.971321 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.954473 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:55.897680 30521 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:55.979894 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:56.000108 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:55.980085 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:55.981600 30351 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:55.981858 30391 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:55.975214 30549 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:30:55.974907 30440 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:55.983634 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:30:56.404464 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 385900us
I20260924 14:30:55.986712 26224 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:56.164812 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 153451us
I20260924 14:30:56.008558 26324 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:55.984513 26399 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:55.985309 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:56.005990 26397 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:56.130481 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:56.097635 26406 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:55.979001 27186 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:56.468044 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:56.508502 27186 raft_consensus.cc:595] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:56.180583 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:56.513811 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:56.514720 27191 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:56.181192 27194 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:55.995319 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:55.997999 29981 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:55.952346 30271 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:56.570587 30566 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:56.177176 30230 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:56.576941 30430 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:56.576505 30566 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:56.577351 30230 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:56.106564 30270 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260924 14:30:56.013537 30275 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:56.105440 30373 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:56.201685 30375 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:56.015012 30407 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260924 14:30:56.338791 30409 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:56.471144 26328 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:30:56.466444 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 479594us
I20260924 14:30:56.463428 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:56.505564 30351 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:56.506304 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:56.116230 30417 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:56.095635 30440 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:56.606890 30417 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:56.610621 30440 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:56.251055 30474 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:56.519949 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:56.272361 30514 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:56.120023 30521 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:56.616230 30351 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:30:56.337363 30527 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.563s	user 0.000s	sys 0.003s
I20260924 14:30:56.144855 30541 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:55.984294 30553 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260924 14:30:56.621434 26405 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1512) took 3377 ms (client timeout 2073 ms). Trace:
I20260924 14:30:56.621718 30077 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:56.622426 30077 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:56.622125 26405 rpcz_store.cc:269] 0924 14:30:53.242300 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.242766 (+   466us) service_pool.cc:227] Handling call
0924 14:30:56.604440 (+3361674us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54001c1954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:56.619178 (+ 14738us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":985129728}
I20260924 14:30:56.349550 30574 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:56.195072 30584 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:56.621449 30553 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:56.625521 30553 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:55.940085 30586 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260924 14:30:56.019253 30589 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:55.966354 30591 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:56.593580 26327 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:56.598449 30376 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260924 14:30:56.600742 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 712) took 2070 ms. Trace:
I20260924 14:30:56.605464 26329 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:56.633477 27185 rpcz_store.cc:276] 0924 14:30:54.530533 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.178365 (+647832us) service_pool.cc:227] Handling call
0924 14:30:55.178830 (+   465us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:55.746940 (+568110us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:56.600161 (+853221us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b540073df54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:56.600529 (+   368us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1304942336}
I20260924 14:30:56.605675 26406 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260924 14:30:56.629727 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 174347us
I20260924 14:30:56.632688 26328 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:56.634192 30271 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:56.637786 30541 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:56.646752 30586 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:56.647405 30275 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:56.613372 30474 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260924 14:30:56.640286 30541 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:56.650079 30566 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:56.659760 30275 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:56.662030 30275 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:56.664274 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1369) took 3589 ms. Trace:
I20260924 14:30:56.663899 30077 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:56.664847 26323 rpcz_store.cc:276] 0924 14:30:53.074606 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.845385 (+770779us) service_pool.cc:227] Handling call
0924 14:30:53.857895 (+ 12510us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:53.939784 (+ 81889us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:53.939835 (+    51us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:53.939898 (+    63us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:53.939940 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:56.663732 (+2723792us) spinlock_profiling.cc:243] Waited 81.6 ms on lock 0x7b5400498754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:56.663805 (+    73us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:56.664089 (+   284us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":187608448}
W20260924 14:30:56.668177 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 646386us
I20260924 14:30:56.671540 30574 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:56.671770 26258 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:56.647413 30586 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:56.682683 26255 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:56.651958 30230 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:56.685427 30371 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:56.718358 27191 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:56.689936 30274 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:30:56.690732 29682 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.701s	user 0.014s	sys 0.012s
I20260924 14:30:56.691823 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:56.692590 26330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 913) took 1781 ms (client timeout 2141 ms). Trace:
I20260924 14:30:56.697755 30349 raft_consensus.cc:3032] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:56.697875 26332 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:56.738852 27189 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:56.702495 26328 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:56.702432 30553 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:56.703415 30077 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:56.703781 30412 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:56.706461 30518 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:56.688572 27194 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:56.752367 30275 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:56.768472 30230 consensus_queue.cc:1046] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260924 14:30:56.773339 30275 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:56.774139 30574 raft_consensus.cc:3032] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:56.766090 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:56.774659 30574 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:56.746145 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:56.776001 30574 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:56.776988 30566 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:30:56.777633 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 119785us
I20260924 14:30:56.777655 30574 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:56.778800 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:56.780668 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:56.766475 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:56.770777 30274 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:56.757390 30315 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:56.767156 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:56.755964 26249 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:30:56.788626 26406 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:56.759303 30553 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:56.789105 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 927) took 1199 ms. Trace:
I20260924 14:30:56.789824 26326 rpcz_store.cc:276] 0924 14:30:55.589274 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.589411 (+   137us) service_pool.cc:227] Handling call
0924 14:30:56.788545 (+1199134us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400276fd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:56.788869 (+   324us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":405537536}
I20260924 14:30:56.771812 30586 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:56.792632 30586 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:56.796367 30244 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:56.796906 30244 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:56.797001 30525 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:30:56.730886 30349 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:56.800215 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1514) took 3477 ms. Trace:
I20260924 14:30:56.798431 30244 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:56.800601 30349 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:56.802333 30244 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:56.800662 26404 rpcz_store.cc:276] 0924 14:30:53.332331 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.390995 (+ 58664us) service_pool.cc:227] Handling call
0924 14:30:53.392904 (+  1909us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:53.393131 (+   227us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:53.393163 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:53.393214 (+    51us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:53.393257 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:53.393321 (+    64us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:56.799829 (+3406508us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b54001e50d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:56.800022 (+   193us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3608320}
W20260924 14:30:56.804184 27190 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 699) took 2437 ms (client timeout 2175 ms). Trace:
I20260924 14:30:56.804602 26258 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
W20260924 14:30:56.804730 27190 rpcz_store.cc:269] 0924 14:30:54.366286 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.366454 (+   168us) service_pool.cc:227] Handling call
0924 14:30:56.803733 (+2437279us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400155c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:56.803985 (+   252us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":241206656}
W20260924 14:30:56.728190 26330 rpcz_store.cc:269] 0924 14:30:54.910793 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.912923 (+  2130us) service_pool.cc:227] Handling call
0924 14:30:56.692073 (+1779150us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b5400557dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:56.692359 (+   286us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":681841536}
I20260924 14:30:56.802654 30349 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:56.768049 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:56.773383 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1363) took 1109 ms. Trace:
I20260924 14:30:56.813674 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:56.743738 30412 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:56.817543 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1385) took 1004 ms. Trace:
I20260924 14:30:56.818288 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:56.818426 30412 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:56.820457 30275 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:56.820806 30481 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:30:56.818004 26327 rpcz_store.cc:276] 0924 14:30:55.813264 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.813561 (+   297us) service_pool.cc:227] Handling call
0924 14:30:55.854698 (+ 41137us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:56.635123 (+780425us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:56.817246 (+182123us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b540001ead4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:56.817467 (+   221us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1684821376}
I20260924 14:30:56.822700 30412 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:56.813880 26250 rpcz_store.cc:276] 0924 14:30:55.663530 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.664869 (+  1339us) service_pool.cc:227] Handling call
0924 14:30:55.665125 (+   256us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:55.738062 (+ 72937us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:56.772803 (+1034741us) spinlock_profiling.cc:243] Waited 72.3 ms on lock 0x7b5400195ad4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:56.773163 (+   360us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":166274304}
I20260924 14:30:56.765183 30541 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:56.832592 26329 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
W20260924 14:30:56.825618 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46981.6us
I20260924 14:30:56.742120 29682 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:56.838742 29682 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:56.840412 30540 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260924 14:30:56.840600 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 856843us
I20260924 14:30:56.841032 27185 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
I20260924 14:30:56.846037 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:56.846554 30566 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:56.847083 30474 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:56.848266 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:56.853354 30518 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:56.859799 30287 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:56.859972 27126 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:56.860363 30518 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:56.860774 30287 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:56.861624 30481 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:30:56.862798 30402 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:56.863514 30402 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:56.863782 30284 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:56.864566 30284 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:56.859870 27192 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 715) took 2284 ms (client timeout 2032 ms). Trace:
W20260924 14:30:56.866284 27192 rpcz_store.cc:269] 0924 14:30:54.575086 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.580612 (+  5526us) service_pool.cc:227] Handling call
0924 14:30:56.859322 (+2278710us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b54003269d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:56.859637 (+   315us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":149812480}
I20260924 14:30:56.857651 30474 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:56.868170 30474 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:56.869066 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41190 (request call id 24) took 1025 ms. Trace:
I20260924 14:30:56.869495 27186 rpcz_store.cc:276] 0924 14:30:55.843847 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.978864 (+135017us) service_pool.cc:227] Handling call
0924 14:30:56.868585 (+889721us) spinlock_profiling.cc:243] Waited 91.7 ms on lock 0x7b5400201aa0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92e3dc0 00007f09b93489c2 00007f09be29281a 00007f09b7985d04 00007f09b7985c92 00007f09b7985bc1 00007f09b7985b3c 00007f09b7984df2 00007f09b777894c 00007f09b7777d86
0924 14:30:56.868967 (+   382us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":211016576}
I20260924 14:30:56.870260 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:56.871996 26325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 925) took 1317 ms (client timeout 1640 ms). Trace:
I20260924 14:30:56.872514 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:56.872465 26325 rpcz_store.cc:269] 0924 14:30:55.554335 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.554610 (+   275us) service_pool.cc:227] Handling call
0924 14:30:56.871548 (+1316938us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400276ad4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:56.871820 (+   272us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":515115392}
I20260924 14:30:56.873414 26252 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:56.858877 30566 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:56.872004 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:56.876093 30274 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:56.880429 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:56.881390 26257 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:56.883750 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:56.884531 27182 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:56.885010 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1306) took 1090 ms. Trace:
W20260924 14:30:56.885278 27182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 726) took 1381 ms (client timeout 1546 ms). Trace:
W20260924 14:30:56.885802 27182 rpcz_store.cc:269] 0924 14:30:55.503821 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.514827 (+ 11006us) service_pool.cc:227] Handling call
0924 14:30:56.847892 (+1333065us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:30:56.885196 (+ 37304us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9640832}
I20260924 14:30:56.885507 26402 rpcz_store.cc:276] 0924 14:30:55.815245 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.843532 (+ 28287us) service_pool.cc:227] Handling call
0924 14:30:55.858827 (+ 15295us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:56.090887 (+232060us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:56.090921 (+    34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:56.090963 (+    42us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:56.090999 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:56.884453 (+793454us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400217354. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:56.884543 (+    90us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:56.884827 (+   284us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":456818048}
I20260924 14:30:56.876344 30402 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:56.884871 26404 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:56.863796 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:56.896050 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 717) took 1191 ms. Trace:
I20260924 14:30:56.897109 30481 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:30:56.896678 27188 rpcz_store.cc:276] 0924 14:30:55.704746 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.704939 (+   193us) service_pool.cc:227] Handling call
0924 14:30:56.895664 (+1190725us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54002008d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:56.895878 (+   214us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":318333312}
I20260924 14:30:56.898903 27181 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 11. Ops: []
I20260924 14:30:56.856302 30540 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:56.899892 30540 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:56.900142 30519 raft_consensus.cc:3032] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:56.900552 30519 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:56.902149 30519 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:56.904418 26221 leader_election.cc:398] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:30:56.904924 26221 leader_election.cc:401] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:30:56.905827 30315 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
W20260924 14:30:56.906056 26256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1014) took 2615 ms (client timeout 2238 ms). Trace:
I20260924 14:30:56.906454 26254 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 4. Current term is 5. Ops: []
W20260924 14:30:56.906579 26256 rpcz_store.cc:269] 0924 14:30:54.291047 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:54.739848 (+448801us) service_pool.cc:227] Handling call
0924 14:30:56.905526 (+2165678us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b540015e0d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:56.905796 (+   270us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1939176192}
I20260924 14:30:56.909255 30566 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:56.909821 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1371) took 1141 ms. Trace:
I20260924 14:30:56.910252 26251 rpcz_store.cc:276] 0924 14:30:55.768544 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.768979 (+   435us) service_pool.cc:227] Handling call
0924 14:30:55.769357 (+   378us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:55.855728 (+ 86371us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:56.909350 (+1053622us) spinlock_profiling.cc:243] Waited 71.7 ms on lock 0x7b540015ead4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:56.909653 (+   303us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":164934016}
I20260924 14:30:56.912351 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1166) took 1825 ms. Trace:
I20260924 14:30:56.912947 27181 rpcz_store.cc:276] 0924 14:30:55.086829 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.463400 (+376571us) service_pool.cc:227] Handling call
0924 14:30:56.712210 (+1248810us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b58004d0a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:30:56.712637 (+   427us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:56.898504 (+185867us) spinlock_profiling.cc:243] Waited 63.1 ms on lock 0x7b5400345920. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92de730 00007f09b9350300 00007f09b9351371 00007f09b934dedf 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc
0924 14:30:56.899325 (+   821us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:56.912289 (+ 12964us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":170753664}
I20260924 14:30:56.913015 30417 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:56.915076 30417 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:56.900182 30366 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:56.921646 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1373) took 1272 ms. Trace:
I20260924 14:30:56.863122 30287 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:56.922206 26332 rpcz_store.cc:276] 0924 14:30:55.649410 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:55.650374 (+   964us) service_pool.cc:227] Handling call
0924 14:30:56.697311 (+1046937us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:56.736712 (+ 39401us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:56.921348 (+184636us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400557dd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:56.921555 (+   207us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2406308736}
I20260924 14:30:56.900463 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:56.929032 30519 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:30:56.935654 26371 leader_election.cc:334] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.675s (SENT)
I20260924 14:30:56.936406 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:56.937706 30254 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260924 14:30:56.937956 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 80612us
I20260924 14:30:56.940084 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:56.941121 30254 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:30:56.863598 30518 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:56.949892 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:56.867269 30284 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:56.956053 27186 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "986260198b71418c831bfc4617da4603"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:30:56.957103 27186 raft_consensus.cc:595] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
W20260924 14:30:56.961236 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 874779us
I20260924 14:30:56.964737 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:56.965725 26398 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:56.966917 26403 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:30:56.977427 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260924 14:30:56.980666 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:56.984429 29677 consensus_queue.cc:1046] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:30:56.984809 29998 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:30:56.986128 29677 consensus_queue.cc:1046] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:56.986516 30576 raft_consensus.cc:3032] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:56.987017 30576 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:56.988214 27125 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:56.988929 30354 raft_consensus.cc:3032] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:30:56.989812 30354 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:56.990011 30287 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:56.990463 30287 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:56.990810 30287 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:56.990937 30354 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:56.992489 30354 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:56.993788 30576 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:56.995262 30576 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:56.989025 30383 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:56.997332 30383 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:56.999826 30383 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:57.000535 30383 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:57.000636 30284 consensus_queue.cc:1046] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:30:57.002079 30383 raft_consensus.cc:3032] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Stepping down as leader of term 4
I20260924 14:30:57.002593 30383 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:57.004000 30383 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:57.032804 29989 raft_consensus.cc:979] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:57.033460 30386 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:57.033847 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:57.034199 30386 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:57.034706 26253 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:57.035976 30544 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:57.036674 30544 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:57.033483 29989 raft_consensus.cc:1067] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:30:57.019819 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:57.044910 26252 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:30:57.057156 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:30:57.059527 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:57.058452 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:30:57.039306 30544 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:57.068677 26254 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
I20260924 14:30:57.078388 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:57.079836 29677 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:57.066448 30386 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:57.081508 26254 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:57.081993 26254 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:57.071759 30540 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:57.089092 30540 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:57.072832 30574 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:57.091271 30574 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:57.091861 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:57.093465 30574 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:57.095388 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:30:57.092031 30333 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.126s	user 0.000s	sys 0.034s
I20260924 14:30:57.072281 30518 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:57.073246 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:57.100050 27182 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
W20260924 14:30:57.110029 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 46732us
I20260924 14:30:57.111181 26254 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.121260 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:57.128005 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.129104 26332 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:30:57.129176 30354 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:57.129819 30354 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:57.130275 30354 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:57.130771 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 784) took 4058 ms. Trace:
I20260924 14:30:57.131340 26331 rpcz_store.cc:276] 0924 14:30:53.072036 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.780762 (+708726us) service_pool.cc:227] Handling call
0924 14:30:53.804254 (+ 23492us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:56.662977 (+2858723us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b18000f3e20. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b8fdd82c 00007f09b64089dc 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6423155 00007f09b93463d0 00007f09b9346f6b 00007f09b935178b
0924 14:30:56.663308 (+   331us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:56.663341 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:56.689315 (+ 25974us) log.cc:842] Serialized 26 byte log entry
0924 14:30:56.811902 (+122587us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:57.068041 (+256139us) log.cc:842] Serialized 14 byte log entry
0924 14:30:57.109627 (+ 41586us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:57.121960 (+ 12333us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:57.122046 (+    86us) raft_consensus.cc:1671] finished
0924 14:30:57.130360 (+  8314us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:57.130558 (+   198us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26827,"prepare.queue_time_us":178213,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":218162,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":5395,"spinlock_wait_cycles":11620224,"thread_start_us":364195,"threads_started":3,"wal-append.queue_time_us":254617}
W20260924 14:30:57.098486 30333 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.185s	user 0.001s	sys 0.035s
I20260924 14:30:57.138921 26325 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:57.144418 26253 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:57.153530 26254 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:57.154302 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:57.113468 30518 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:30:57.155896 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:57.123072 30574 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:57.162117 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:57.166257 26331 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:57.152501 30528 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:57.168639 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:57.169931 30284 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:30:57.170450 30544 raft_consensus.cc:979] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:57.173542 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:57.177129 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:57.179690 26327 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:57.182927 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:57.153637 30474 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.187873 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:57.172542 30284 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:57.157301 30518 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:57.178478 27182 raft_consensus.cc:2362] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11. Re-sending same reply.
I20260924 14:30:57.170960 30574 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.205654 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.206264 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.213305 26252 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:57.213361 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.213205 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.212905 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.212581 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:57.217358 26332 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:57.209573 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:57.218271 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.212400 30319 raft_consensus.cc:1067] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:30:57.202912 30383 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:30:57.206027 30333 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:57.221561 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:57.206233 30518 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:57.231664 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:57.241374 30389 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260924 14:30:57.242343 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.243201 26331 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:57.249729 26398 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "986260198b71418c831bfc4617da4603"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:35444
I20260924 14:30:57.250775 26398 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:30:57.250631 30319 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:30:57.251235 26398 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:57.254395 30284 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:57.227383 30474 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:57.241683 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:57.237315 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:57.231151 30333 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:30:57.275390 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 151220us
I20260924 14:30:57.278527 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.279484 26331 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:57.270411 30254 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:57.281512 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:57.283205 26330 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:57.287518 26323 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:57.283520 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.236457 30544 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260924 14:30:57.292820 26398 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:57.236699 30287 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.275552 30586 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:57.299975 26326 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:57.300247 26324 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:57.301587 30354 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:57.305680 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 88288.3us
I20260924 14:30:57.309209 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:57.312984 30609 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
I20260924 14:30:57.312566 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:57.314558 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:57.317052 30354 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:57.321758 26397 raft_consensus.cc:3032] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:30:57.322250 26397 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:57.323127 26397 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:57.324014 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 100722us
I20260924 14:30:57.324752 26397 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:57.326081 30354 consensus_queue.cc:1046] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:30:57.327594 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:57.328807 29989 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:30:57.328426 30333 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.329679 26077 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 11, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 11 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
W20260924 14:30:57.333568 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 53194us
I20260924 14:30:57.334909 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:57.327147 26078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 6) took 31526 ms (client timeout 15000 ms). Trace:
I20260924 14:30:57.341988 26398 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:57.338428 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.340310 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:57.346029 26328 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:57.346235 26405 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:57.348590 27182 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:30:57.345172 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.350713 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:57.370183 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.370885 27192 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:57.371088 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.359439 27186 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:57.335934 29989 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:57.358099 30319 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260924 14:30:57.372013 26324 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:57.382396 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:57.385820 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.394366 30558 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:57.394987 30572 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:30:57.397015 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.397565 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.398553 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.399660 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:57.401757 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:57.386235 30574 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:57.382200 30528 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:57.375672 30402 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:57.383249 26327 raft_consensus.cc:2362] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10. Re-sending same reply.
W20260924 14:30:57.406371 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 78855.5us
I20260924 14:30:57.407205 26331 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:57.420524 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:57.425930 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:57.405227 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.454911 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.455233 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:57.458647 26252 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:57.452925 29989 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.463344 29989 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:57.463857 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.413388 30402 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:57.405506 30541 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:57.410398 30544 raft_consensus.cc:3032] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:57.465566 26404 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:30:57.466104 30544 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:57.407222 30572 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:57.468372 30572 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:57.414058 30333 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:57.464000 26401 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:57.467043 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.479817 26397 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260924 14:30:57.481758 26405 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:57.482790 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.486588 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:57.403391 29998 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:57.489296 29998 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.406598 30528 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:57.491779 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:57.492296 29998 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:57.492697 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.493495 26324 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:57.408789 30287 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:57.497953 26224 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:57.498987 30519 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:30:57.499583 30519 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:57.494701 30402 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260924 14:30:57.500831 30519 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:57.503577 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.496441 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:30:57.475986 30366 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.559s	user 0.006s	sys 0.006s
I20260924 14:30:57.514853 26398 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:57.447466 26325 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:57.509095 26077 catalog_manager.cc:5495] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:57.404621 30586 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260924 14:30:57.525767 30576 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:57.460668 30254 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260924 14:30:57.533064 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:57.543685 26399 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:57.517848 30412 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:57.557780 26255 raft_consensus.cc:2396] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260924 14:30:57.545429 26224 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:57.562384 26329 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:30:57.544106 30287 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:57.467073 30544 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.574734 30607 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260924 14:30:57.590696 30374 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:57.558478 26077 catalog_manager.cc:5495] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 9, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 9 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:57.579506 30528 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:57.585366 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 49252.2us
I20260924 14:30:57.571251 30566 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:57.572474 30474 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:57.588057 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:57.586807 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.577811 30576 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:57.593997 30417 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260924 14:30:57.587371 30544 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:57.570608 27188 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
W20260924 14:30:57.608094 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 67984.8us
I20260924 14:30:57.610173 30048 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:57.610555 30576 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:57.610755 30048 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:57.600754 30566 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:57.615988 30417 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:57.583532 30287 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.642493 26077 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 10, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 10 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:30:57.622432 30417 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:30:57.652261 30417 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:57.653570 30417 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:57.656667 30048 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:57.623365 26252 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:57.622856 30558 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:57.606612 30374 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:57.691745 27125 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:57.343417 26078 rpcz_store.cc:269] 0924 14:30:25.800242 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:25.803843 (+  3601us) service_pool.cc:227] Handling call
0924 14:30:57.326858 (+31523015us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:30:49.612133 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:30:49.612856 (+   723us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:30:49.612902 (+    46us) write_op.cc:435] Acquired schema lock
0924 14:30:49.612927 (+    25us) tablet.cc:660] Decoding operations
0924 14:30:49.614285 (+  1358us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:30:49.614379 (+    94us) write_op.cc:641] Partition lock acquired for write op
0924 14:30:49.614442 (+    63us) tablet.cc:683] Acquiring locks for 11 operations
0924 14:30:49.615721 (+  1279us) tablet.cc:699] Row locks acquired
0924 14:30:49.615748 (+    27us) write_op.cc:260] PREPARE: finished
0924 14:30:49.615958 (+   210us) write_op.cc:270] Start()
0924 14:30:49.616102 (+   144us) write_op.cc:276] Timestamp: P: 1790260249615890 usec, L: 0
0924 14:30:49.616130 (+    28us) op_driver.cc:331] REPLICATION: starting
0924 14:30:49.616993 (+   863us) log.cc:842] Serialized 4446 byte log entry
0924 14:30:50.289461 (+672468us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b180015ec20. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426764 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:30:50.652099 (+362638us) op_driver.cc:444] REPLICATION: finished
0924 14:30:51.818128 (+1166029us) write_op.cc:301] APPLY: starting
0924 14:30:51.818286 (+   158us) tablet.cc:1365] starting BulkCheckPresence
0924 14:30:51.818518 (+   232us) tablet.cc:1368] finished BulkCheckPresence
0924 14:30:51.818538 (+    20us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:30:51.820037 (+  1499us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:30:51.820279 (+   242us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0924 14:30:51.820328 (+    49us) write_op.cc:312] APPLY: finished
0924 14:30:51.820612 (+   284us) log.cc:842] Serialized 82 byte log entry
0924 14:30:52.564475 (+743863us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:52.564624 (+   149us) write_op.cc:454] Released schema lock
0924 14:30:52.564673 (+    49us) write_op.cc:341] FINISH: Updating metrics
0924 14:30:52.565051 (+   378us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:30:52.565083 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":85332,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":1156786,"apply.run_cpu_time_us":42369,"apply.run_wall_time_us":747414,"num_ops":11,"prepare.queue_time_us":60089,"prepare.run_cpu_time_us":16507,"prepare.run_wall_time_us":62788,"raft.queue_time_us":326101,"raft.run_cpu_time_us":11733,"raft.run_wall_time_us":504327,"replication_time_us":1027262,"spinlock_wait_cycles":30940032,"thread_start_us":1428877,"threads_started":5,"wal-append.queue_time_us":881914,"wal-append.run_cpu_time_us":24072,"wal-append.run_wall_time_us":2856071}]]}
W20260924 14:30:57.716617 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 483280us
I20260924 14:30:57.724828 26077 catalog_manager.cc:5495] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 6 to 7, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 7 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:57.735644 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:57.729108 30084 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:57.737031 30084 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:57.745077 26077 catalog_manager.cc:5495] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:57.733002 26398 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:57.725471 29682 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:30:57.785529 27126 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:57.771962 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:57.708729 30558 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:30:57.790386 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 54865.3us
I20260924 14:30:57.788544 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:57.790333 26403 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:57.796844 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:57.802067 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:57.817260 30333 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:57.818179 30541 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:30:57.844916 30366 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:57.796655 30558 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:57.816660 26253 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.026s	sys 0.067s
I20260924 14:30:57.845552 30366 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.848304 30333 raft_consensus.cc:686] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:57.820834 30084 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:30:57.862087 26253 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.027s	sys 0.067s
I20260924 14:30:57.835829 30607 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:30:57.829891 30572 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:57.798539 29682 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:57.798902 26077 catalog_manager.cc:5495] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
W20260924 14:30:57.892930 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 104480us
I20260924 14:30:57.895527 27185 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:30:57.852522 30287 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:57.861516 30541 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:57.905435 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:57.884443 30566 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:57.910441 26403 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:30:57.886133 30607 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:57.798417 30319 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.106s
I20260924 14:30:57.913338 26403 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:57.918146 29677 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:30:57.918622 29677 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:57.915352 30607 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:57.924333 30566 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:57.923503 30374 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:57.947083 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 50443.9us
W20260924 14:30:57.958941 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 161653us
I20260924 14:30:57.944674 26403 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:57.934136 30572 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:30:57.970100 30254 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:57.970392 30417 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:57.970623 30254 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:57.971158 30417 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:57.970635 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.973644 30417 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:57.973865 26329 raft_consensus.cc:2362] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12. Re-sending same reply.
I20260924 14:30:57.961203 26077 catalog_manager.cc:5495] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 8, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 8 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:30:57.975386 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:30:57.977120 26251 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.049s	sys 0.033s
W20260924 14:30:57.977866 26251 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.049s	sys 0.033s
I20260924 14:30:57.982353 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.985054 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.986120 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.987085 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.984927 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:57.990867 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.991190 26323 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260924 14:30:57.991813 26330 raft_consensus.cc:2379] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 13: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:57.991931 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:57.971177 30254 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:57.994287 26329 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260924 14:30:57.995486 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:57.995846 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:57.995990 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:57.968945 30572 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:57.934695 30333 consensus_queue.cc:237] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:57.960404 30566 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:58.003005 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44733.5us
I20260924 14:30:57.934090 29677 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:58.004258 26077 catalog_manager.cc:5495] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 6 to 8, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 8 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:58.008071 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.015934 29677 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:30:58.019982 26249 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.008s	sys 0.022s
I20260924 14:30:58.024921 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.037820 26324 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
I20260924 14:30:58.029892 26077 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:58.046496 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:58.042224 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:58.023988 26371 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.236s (SENT) [suppressed 45 similar messages]
I20260924 14:30:57.997875 30374 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260924 14:30:58.047335 26326 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:58.017272 30366 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:58.025115 30558 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:58.024320 30574 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:58.068236 26077 catalog_manager.cc:5495] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:58.072353 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.072400 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.079682 30411 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:58.080942 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:58.081619 26327 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:58.083817 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:58.084960 26329 raft_consensus.cc:2396] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260924 14:30:58.083884 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:30:58.085534 26323 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:58.095281 30574 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:58.099578 29834 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260924 14:30:58.006814 30541 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:58.104555 26332 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:58.027225 30077 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260924 14:30:58.152586 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 84383.2us
I20260924 14:30:58.148401 26327 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:58.155620 26323 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:58.167346 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:30:58.155539 30613 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260924 14:30:58.152248 30412 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.014s	sys 0.011s
W20260924 14:30:58.157714 30544 consensus_meta.cc:287] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.001s
I20260924 14:30:58.198359 27186 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260924 14:30:58.157364 30518 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.888s	user 0.014s	sys 0.000s
I20260924 14:30:58.199997 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.028596 29682 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:58.216365 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:58.216895 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1399) took 1016 ms. Trace:
I20260924 14:30:58.098838 30572 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:58.204574 30412 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.014s	sys 0.012s
I20260924 14:30:58.223914 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:58.247259 26254 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:58.219978 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:58.249678 26252 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:30:58.256533 27123 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:58.264070 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:58.257517 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 261587us
I20260924 14:30:58.217427 26253 rpcz_store.cc:276] 0924 14:30:57.200433 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:57.201214 (+   781us) service_pool.cc:227] Handling call
0924 14:30:57.309786 (+108572us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:30:57.309996 (+   210us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:57.310242 (+   246us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:30:57.894535 (+584293us) log.cc:842] Serialized 14 byte log entry
0924 14:30:58.215641 (+321106us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:30:58.215712 (+    71us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:58.216595 (+   883us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:58.216676 (+    81us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:58.216826 (+   150us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":762042,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"spinlock_wait_cycles":2866560,"thread_start_us":895045,"threads_started":2}
I20260924 14:30:58.269907 26077 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 11, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:58.285715 27185 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:30:58.301004 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 84729.4us
I20260924 14:30:58.218705 30518 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:58.270862 30287 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:58.325317 26077 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:58.315346 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:58.327888 26326 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:58.332881 27181 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:30:58.307278 30274 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:30:58.333820 26077 catalog_manager.cc:5495] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:30:58.334648 26249 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:58.314812 30287 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:58.345129 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.349470 30586 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:58.349570 30519 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:58.349983 30586 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:58.350229 30519 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:58.350749 30519 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:58.350915 30586 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:58.352344 30586 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:58.354244 26403 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:58.356210 30636 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:58.345994 27185 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:30:58.323376 30566 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:58.346527 26077 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 11, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 11 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:58.434567 26295 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:30:58.332125 30411 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:30:58.440992 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 139250us
I20260924 14:30:58.445437 26399 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:58.446342 26402 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:58.454041 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 732) took 1165 ms. Trace:
W20260924 14:30:58.447170 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 243162us
I20260924 14:30:58.356793 30636 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:58.455112 27181 rpcz_store.cc:276] 0924 14:30:57.288939 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:57.289126 (+   187us) service_pool.cc:227] Handling call
0924 14:30:58.453877 (+1164751us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540056cf54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:30:58.453984 (+   107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2340251904}
I20260924 14:30:58.346022 30287 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:58.465195 26077 catalog_manager.cc:5495] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 4 to 7, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 7 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:58.480639 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.343940 30558 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.125s
I20260924 14:30:58.490155 30374 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:30:58.490736 30647 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:30:58.447219 30566 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:58.482772 26077 catalog_manager.cc:5495] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 11, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 11 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
W20260924 14:30:58.516444 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 176525us
W20260924 14:30:58.529906 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47623.3us
I20260924 14:30:58.481429 30574 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:58.499619 30284 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:58.515039 30274 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:58.464154 30411 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:58.561987 26371 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.546s (SENT)
W20260924 14:30:58.560491 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 85957.9us
I20260924 14:30:58.564193 26369 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.572722 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:58.573282 26077 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 9, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 9 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:30:58.536254 30647 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:58.583668 26399 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:58.519397 30566 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:30:58.580698 30607 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.000s
I20260924 14:30:58.590754 26077 catalog_manager.cc:5495] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 6 to 7, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 7 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:30:58.600528 26077 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 9, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 9 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:30:58.558944 30383 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260924 14:30:58.586239 30518 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:58.566780 30284 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:58.566359 30274 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:58.617671 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:58.626758 26399 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260924 14:30:58.628690 26371 leader_election.cc:334] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.617s (SENT)
I20260924 14:30:58.653249 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.653426 30650 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:58.654592 30284 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:58.655505 30632 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260924 14:30:58.655392 30647 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:30:58.656802 30630 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.335s	user 0.001s	sys 0.000s
I20260924 14:30:58.650846 30414 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:58.658478 30414 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:58.660221 29834 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:30:58.660606 30414 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:58.660791 30629 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:58.662146 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.625473 26323 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
W20260924 14:30:58.663691 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 99574.1us
I20260924 14:30:58.652534 26327 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:58.618974 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:58.604948 30631 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
W20260924 14:30:58.610577 30626 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260924 14:30:58.661413 30629 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:58.681433 30653 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:30:58.682220 30417 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:58.682761 30417 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:58.681034 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1279) took 1062 ms. Trace:
I20260924 14:30:58.608711 30349 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:58.611810 30572 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:58.692898 30417 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:58.654624 30650 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:58.660658 29834 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:30:58.692978 30658 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260924 14:30:58.717290 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 55220.2us
I20260924 14:30:58.716200 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:58.685487 26400 rpcz_store.cc:276] 0924 14:30:57.599641 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:57.599789 (+   148us) service_pool.cc:227] Handling call
0924 14:30:57.600249 (+   460us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:57.608438 (+  8189us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:57.608486 (+    48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:57.640358 (+ 31872us) log.cc:842] Serialized 26 byte log entry
0924 14:30:58.030538 (+390180us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:30:58.030599 (+    61us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:58.030690 (+    91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:58.661649 (+630959us) raft_consensus.cc:1671] finished
0924 14:30:58.661675 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:58.661853 (+   178us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1654,"raft.queue_time_us":5084,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":8685,"thread_start_us":378845,"threads_started":1,"wal-append.queue_time_us":549032}
W20260924 14:30:58.693907 30641 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260924 14:30:58.701318 30411 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:58.683400 30629 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:58.704010 30638 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260924 14:30:58.704305 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.608434 30607 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:58.665511 30412 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:58.726552 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:58.610821 30634 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:30:58.742470 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:58.748402 26399 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260924 14:30:58.762359 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 78716.5us
I20260924 14:30:58.772367 30412 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:58.773176 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:58.774681 30412 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:58.673174 30414 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.056s
I20260924 14:30:58.766201 30663 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:58.776438 30663 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:58.776889 30566 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:58.767062 26405 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:30:58.772737 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:58.775884 30414 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Stepping down as leader of term 9
W20260924 14:30:58.763211 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 144351us
I20260924 14:30:58.783090 26402 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260924 14:30:58.786541 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 114598us
I20260924 14:30:58.787774 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:30:58.765187 30648 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260924 14:30:58.789264 30647 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:30:58.789655 30663 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:58.789786 30274 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:58.790232 30649 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:58.790889 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.790760 30649 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:58.792266 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:58.793200 30649 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:58.793855 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:58.794319 30652 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:58.794538 26403 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:30:58.780594 30634 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:58.728968 30636 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:58.800786 27192 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:58.729611 30574 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:58.800555 30640 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:30:58.804569 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.784315 30558 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:58.803437 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.795336 29834 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:58.807148 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.803758 30412 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:58.808099 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1324) took 1411 ms. Trace:
I20260924 14:30:58.729985 30349 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:58.706329 30572 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:58.826279 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:58.826663 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:58.827320 30636 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:58.827342 26325 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:58.808632 29442 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:58.828547 29442 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:58.830588 29442 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:58.810230 30574 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:58.831717 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:58.832557 26328 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 17: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:58.795136 30634 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:58.815820 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:58.817772 30528 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:30:58.836634 30586 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:30:58.838089 30663 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:58.840101 30663 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:58.840351 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:58.841145 30541 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:30:58.842489 30663 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:58.784993 30414 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:58.823956 26332 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:30:58.824466 27181 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:58.807183 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.827605 30572 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:58.808605 30655 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:58.845048 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.845840 27191 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:58.846732 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:30:58.847044 30254 consensus_meta.cc:287] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.837s	user 0.007s	sys 0.003s
I20260924 14:30:58.847347 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:58.847640 30655 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:30:58.848060 30655 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:58.848399 30655 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:58.853693 30659 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:30:58.836364 30528 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:58.857199 30349 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:58.857776 30528 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:58.817905 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:58.836792 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:58.838017 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.859424 26326 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:58.841408 30586 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:30:58.860216 30586 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:58.832562 30574 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:58.843772 30414 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:58.859660 27182 raft_consensus.cc:2379] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:58.861186 30586 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:58.861984 30414 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:58.808190 30628 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:58.818919 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:58.822515 26406 rpcz_store.cc:276] 0924 14:30:57.396849 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:57.398207 (+  1358us) service_pool.cc:227] Handling call
0924 14:30:57.398594 (+   387us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:57.399152 (+   558us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:57.399200 (+    48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:57.452372 (+ 53172us) log.cc:842] Serialized 26 byte log entry
0924 14:30:57.899547 (+447175us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:57.899599 (+    52us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:57.899696 (+    97us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:58.807889 (+908193us) raft_consensus.cc:1671] finished
0924 14:30:58.807912 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:58.808046 (+   134us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12,"raft.queue_time_us":5285,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":108,"thread_start_us":426751,"threads_started":1,"wal-append.queue_time_us":705135}
I20260924 14:30:58.785336 30411 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260924 14:30:58.794551 30519 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:58.888011 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:58.885185 26369 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:30:58.885449 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 80939.3us
I20260924 14:30:58.889353 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.865427 30652 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:58.899609 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:58.900869 30607 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:58.903855 30576 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:30:58.904418 30519 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:58.908016 26326 raft_consensus.cc:1261] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:30:58.863732 30628 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:58.898020 29834 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:30:58.865111 30640 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:58.864931 30412 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:58.898845 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:58.899798 30652 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:30:58.909857 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:58.909765 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:58.896464 26332 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:30:58.924922 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:58.931517 26405 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:58.930538 26331 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:30:58.926333 26252 raft_consensus.cc:2354] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11.
I20260924 14:30:58.933072 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:58.927573 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:58.934016 26254 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:58.934638 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.932317 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:58.903069 30572 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260924 14:30:58.937888 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.944857 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:30:58.922832 30647 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260924 14:30:59.017108 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 76454.6us
I20260924 14:30:59.017191 26328 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:59.022722 30637 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:30:59.022871 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 85086.2us
I20260924 14:30:58.895416 30649 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260924 14:30:59.023864 30643 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:59.072213 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:58.872895 30254 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:58.930194 30349 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:59.046850 26405 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:30:58.935082 30652 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:58.884189 30287 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.065465 30647 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:59.096391 30647 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:59.080420 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:59.087857 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:30:59.086827 30636 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:59.074678 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:58.939349 30284 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:30:59.109823 30573 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:59.110378 30573 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:59.110749 30573 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:59.093961 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:59.111810 26399 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:30:59.119472 30671 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260924 14:30:59.121174 30237 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:59.120924 30573 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:59.121685 30237 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:59.122153 30237 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:59.123127 30573 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:59.109623 30519 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:30:59.124477 30519 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:59.125459 30519 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:59.131589 26399 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:30:59.130623 30284 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:30:59.132984 27192 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: []
I20260924 14:30:59.134924 30254 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:59.137172 27186 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:59.137362 30541 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:59.097285 30637 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:30:59.137873 30541 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:59.140108 30541 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:59.137233 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:59.141454 27188 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:30:59.142012 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 69922.9us
I20260924 14:30:59.143268 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:59.101016 26404 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:59.144133 30659 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:59.088294 30544 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:59.081032 30607 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260924 14:30:59.166630 27188 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:30:59.168004 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:59.169337 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:59.170392 27181 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:59.092710 30349 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:59.159369 30636 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:59.168783 30541 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election won for term 11
W20260924 14:30:59.194397 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 86625.5us
W20260924 14:30:59.196941 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 122551us
I20260924 14:30:59.207139 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.208312 26251 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:59.212498 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:59.217531 27181 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:30:59.221664 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:59.198283 30639 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:59.238525 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:59.211678 30284 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:59.223399 30573 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:30:59.254136 26405 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:30:59.225867 30541 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:59.196147 30636 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:59.240185 30412 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:59.244102 30576 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:30:59.102615 30640 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:59.337253 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 114555us
I20260924 14:30:59.234447 30237 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:59.246512 30284 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:59.347332 30284 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:59.290294 30628 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:30:59.352625 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 208174us
I20260924 14:30:59.296178 30653 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260924 14:30:59.384996 30636 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:59.384853 30541 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:59.243719 30639 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:30:59.380101 30678 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260924 14:30:59.388928 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:30:59.363873 30643 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:59.400286 30642 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:30:59.422646 30642 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:59.407387 29682 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:59.423167 30642 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:30:59.423696 30237 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:59.423774 29682 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.425971 29682 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:30:59.408099 30680 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260924 14:30:59.404793 30672 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260924 14:30:59.406543 30544 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:59.428412 30544 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:59.406826 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.427793 30349 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:59.433290 26251 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:59.368222 30683 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
W20260924 14:30:59.437533 29834 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.000s
I20260924 14:30:59.387215 30576 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:59.444778 30576 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:30:59.407294 30687 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260924 14:30:59.421840 30639 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:30:59.422004 30634 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.000s
I20260924 14:30:59.448822 30634 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:59.450264 26330 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:30:59.406160 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:30:59.405092 30414 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.000s
I20260924 14:30:59.442744 26258 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
I20260924 14:30:59.442554 30647 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:59.443684 30637 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:30:59.452121 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 44284.1us
I20260924 14:30:59.467338 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.453737 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:59.471510 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.472270 26249 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:30:59.476639 27191 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260924 14:30:59.451107 30655 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.602s	user 0.004s	sys 0.008s
I20260924 14:30:59.483085 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:59.450524 30544 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.450987 30634 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:59.505646 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.509001 26253 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
W20260924 14:30:59.513405 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 168889us
I20260924 14:30:59.525302 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:59.526615 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:59.511258 30682 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260924 14:30:59.530131 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:30:59.527109 26371 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.787s (SENT)
I20260924 14:30:59.533754 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:59.531419 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.505400 30374 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:30:59.535017 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.535218 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.536844 26251 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:59.538239 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.539073 26254 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:59.538314 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:59.540913 26403 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:59.489745 30655 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:59.545277 30655 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:59.548539 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.549319 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:59.549343 26253 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:30:59.550297 26402 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:59.490947 30642 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:59.558372 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.560348 26255 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:59.562978 30642 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:59.563410 30445 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:59.564028 30445 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:59.566298 30445 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:59.567840 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:59.569036 27191 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:59.569895 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:59.576464 26249 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:59.577515 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:59.550915 30374 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:59.586584 26406 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:30:59.589418 27125 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:59.590456 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.594388 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.595135 26254 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:59.595202 30694 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260924 14:30:59.596170 30694 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:59.596284 30688 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:59.596547 30694 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:59.596779 30688 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:59.597162 30688 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:59.597254 30647 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:59.597474 30694 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.598888 30694 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:59.591804 26249 raft_consensus.cc:2362] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11. Re-sending same reply.
I20260924 14:30:59.594792 30445 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:30:59.607947 30445 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:59.608196 30287 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:59.608361 30445 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:59.609721 26254 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260924 14:30:59.614446 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.615516 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.615918 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.616993 26249 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 5: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:59.617794 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.612353 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.619051 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.619683 30637 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:30:59.620735 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.621438 30697 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:30:59.623795 26249 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:59.626159 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:30:59.628772 30572 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:30:59.630960 30697 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:30:59.573915 30652 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:59.632644 30697 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:30:59.633682 30652 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:59.633766 30697 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:59.635437 30697 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.636786 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.638132 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.622006 30693 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:59.639822 26257 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:30:59.639741 30693 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:59.641470 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.619971 26254 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:30:59.643515 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.644673 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.636978 30697 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:30:59.622470 30077 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:59.645161 30544 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:59.645601 30077 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:59.646113 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.646762 26258 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:59.646610 30077 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:59.647799 30077 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:59.621783 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.659578 26249 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:30:59.662679 26253 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9. Ops: []
I20260924 14:30:59.639297 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:30:59.677423 30674 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.431s	user 0.002s	sys 0.001s
I20260924 14:30:59.678790 26253 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 7. Current term is 8. Ops: []
I20260924 14:30:59.678925 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:59.679051 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1300) took 1011 ms. Trace:
I20260924 14:30:59.680080 26253 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
I20260924 14:30:59.681694 26253 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 13. Ops: [11.9-11.9]
I20260924 14:30:59.682657 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:30:59.683581 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 156521us
W20260924 14:30:59.684788 26371 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.969s (SENT)
I20260924 14:30:59.680305 26397 rpcz_store.cc:276] 0924 14:30:58.667264 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:58.678303 (+ 11039us) service_pool.cc:227] Handling call
0924 14:30:58.709561 (+ 31258us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:58.729497 (+ 19936us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:58.729529 (+    32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:58.765594 (+ 36065us) log.cc:842] Serialized 26 byte log entry
0924 14:30:59.009000 (+243406us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:30:59.010236 (+  1236us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:59.010319 (+    83us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:59.610986 (+600667us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:30:59.678713 (+ 67727us) raft_consensus.cc:1671] finished
0924 14:30:59.678736 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:59.678983 (+   247us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2512,"raft.queue_time_us":10545,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":10944,"spinlock_wait_cycles":6755200,"thread_start_us":189189,"threads_started":1,"wal-append.queue_time_us":389222}
I20260924 14:30:59.690651 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:59.692746 26249 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
I20260924 14:30:59.697150 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.697659 26253 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260924 14:30:59.693931 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:30:59.697924 26254 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:30:59.698891 26398 raft_consensus.cc:2354] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:30:59.676712 30693 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:30:59.704360 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.695971 30607 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:30:59.703974 30647 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:59.738174 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 742) took 1487 ms. Trace:
I20260924 14:30:59.738641 27186 rpcz_store.cc:276] 0924 14:30:58.250987 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:58.257676 (+  6689us) service_pool.cc:227] Handling call
0924 14:30:58.302392 (+ 44716us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:30:59.137633 (+835241us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:59.737910 (+600277us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54006c5054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:30:59.738106 (+   196us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1919333632}
W20260924 14:30:59.738799 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 54050.7us
W20260924 14:30:59.743359 26371 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.549s (SENT)
I20260924 14:30:59.743808 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:59.744568 30682 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:30:59.723325 26224 leader_election.cc:398] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:30:59.749272 26224 leader_election.cc:401] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260924 14:30:59.720454 30607 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:59.726189 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:59.696182 30652 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260924 14:30:59.737005 30374 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:59.748575 26258 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:30:59.754374 26257 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:30:59.759132 26249 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: [9.6-10.7]
I20260924 14:30:59.792016 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.794539 27192 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:59.802230 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:59.803668 26251 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:59.759862 30607 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:30:59.814481 26257 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:59.794672 30287 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:30:59.794348 30637 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:30:59.824558 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 79662.9us
W20260924 14:30:59.825649 26371 leader_election.cc:334] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.019s (SENT)
I20260924 14:30:59.826102 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:30:59.836385 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:30:59.839815 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 833) took 1019 ms. Trace:
I20260924 14:30:59.842289 26298 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:30:59.843241 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 832) took 1025 ms. Trace:
I20260924 14:30:59.845997 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:59.846366 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:30:59.846997 27191 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:30:59.852145 27125 leader_election.cc:398] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11.
I20260924 14:30:59.852589 27125 leader_election.cc:401] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260924 14:30:59.856980 27191 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:30:59.801249 30697 consensus_queue.cc:1046] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260924 14:30:59.843669 26324 rpcz_store.cc:276] 0924 14:30:58.817587 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:58.818233 (+   646us) service_pool.cc:227] Handling call
0924 14:30:58.818579 (+   346us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:58.818811 (+   232us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:58.818836 (+    25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:58.819322 (+   486us) log.cc:842] Serialized 26 byte log entry
0924 14:30:58.940212 (+120890us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:30:59.032750 (+ 92538us) log.cc:842] Serialized 14 byte log entry
0924 14:30:59.842936 (+810186us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:59.843014 (+    78us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:59.843054 (+    40us) raft_consensus.cc:1671] finished
0924 14:30:59.843066 (+    12us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:59.843188 (+   122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":809783,"prepare.queue_time_us":81657,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"thread_start_us":120452,"threads_started":1,"wal-append.queue_time_us":259670}
I20260924 14:30:59.840381 26323 rpcz_store.cc:276] 0924 14:30:58.820116 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:58.820374 (+   258us) service_pool.cc:227] Handling call
0924 14:30:58.835246 (+ 14872us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:30:58.835605 (+   359us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:30:58.835633 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:30:58.836112 (+   479us) log.cc:842] Serialized 26 byte log entry
0924 14:30:58.925270 (+ 89158us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:30:58.932926 (+  7656us) log.cc:842] Serialized 14 byte log entry
0924 14:30:59.410104 (+477178us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b1800220840. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:30:59.839239 (+429135us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b1800220840. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09b6261990 00007f09b6423864 00007f09b6423155 00007f09b93463d0 00007f09b9358576 00007f09b93690d7 00007f09b9369058
0924 14:30:59.839460 (+   221us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:30:59.839546 (+    86us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:30:59.839593 (+    47us) raft_consensus.cc:1671] finished
0924 14:30:59.839613 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:30:59.839753 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3689,"prepare.queue_time_us":10833,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":18952,"raft.queue_time_us":2305,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":16908416,"thread_start_us":88733,"threads_started":1,"wal-append.queue_time_us":232060}
I20260924 14:30:59.832450 30287 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:30:59.862433 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:30:59.862460 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:59.859683 30629 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:30:59.840058 30333 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:30:59.820618 30402 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.043s
I20260924 14:30:59.811426 30688 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:59.873590 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.877310 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260924 14:30:59.883906 30685 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.519s	user 0.000s	sys 0.002s
I20260924 14:30:59.863199 30412 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:59.868762 30683 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:59.872514 30287 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:59.862866 30663 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:59.867761 30576 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:30:59.860240 30682 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260924 14:30:59.912357 30703 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260924 14:30:59.912768 30664 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:30:59.913352 30704 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260924 14:30:59.918654 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.919473 26256 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:30:59.920110 30048 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:59.920585 30048 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:30:59.921499 30048 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.922796 30048 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:59.923990 30663 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.924782 30628 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:59.925300 30628 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:30:59.926059 30663 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:59.926590 30663 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:59.927587 30576 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.929838 30576 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:59.932966 30690 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:30:59.926992 30663 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.933471 30690 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:30:59.935702 30690 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:59.936952 30690 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:30:59.941826 30628 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:30:59.913363 30664 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.943581 30683 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11.
I20260924 14:30:59.944864 30664 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:59.949679 30663 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:30:59.956182 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.957027 26256 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:30:59.958298 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.958345 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.959406 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.960253 26256 raft_consensus.cc:2354] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:30:59.960690 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.961355 26253 raft_consensus.cc:2362] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12. Re-sending same reply.
I20260924 14:30:59.961529 30683 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:30:59.961784 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.962049 30683 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:30:59.964069 30683 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:30:59.965605 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.968721 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.969307 26254 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:30:59.969596 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:30:59.972239 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.972839 26251 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:30:59.973884 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.987643 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.989543 27192 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:30:59.989802 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.990784 26250 raft_consensus.cc:2354] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12.
I20260924 14:30:59.992527 26406 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:30:59.994854 26254 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:30:59.887437 30333 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:30:59.996799 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:30:59.997514 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:30:59.998172 26258 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:30:59.990522 26252 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:00.001367 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:00.002413 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:00.002880 30518 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:31:00.003278 30518 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:00.003609 30518 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:31:00.003682 30576 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:00.004350 30576 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:00.005249 30576 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:30:59.910495 30412 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:30:59.997561 30333 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:00.009294 30412 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:00.012562 26258 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:00.015533 30693 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:31:00.022351 30637 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:00.024608 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:00.028724 26256 raft_consensus.cc:2379] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:31:00.030673 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 185393us
I20260924 14:31:00.038432 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:00.051431 27192 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:00.036580 26255 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:00.074631 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.076959 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:00.068034 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 217603us
I20260924 14:31:00.079293 26327 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:00.075708 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:00.081606 26327 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:00.081979 26295 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:00.084954 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.085964 26399 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:00.080375 26397 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 4.
I20260924 14:31:00.090170 26332 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:31:00.068676 30445 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:00.091087 30682 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:00.091629 30682 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:00.092481 30445 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:00.093612 30682 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:00.093876 27181 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:00.094420 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.094907 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:00.095031 26329 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:31:00.095178 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:00.072527 30688 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:00.095862 26252 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:00.064409 30048 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260924 14:31:00.078812 30694 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:00.097643 26324 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:00.098011 30694 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:00.099889 30694 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:00.110663 30688 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:31:00.114624 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:00.115331 26324 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:00.120172 26252 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:00.121390 27126 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:00.122539 30402 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:00.123044 30402 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:00.122846 30287 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:00.132061 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50214.6us
I20260924 14:31:00.125075 27125 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:00.132010 26324 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:00.137866 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.145175 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:00.148973 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:00.149863 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 64988.3us
I20260924 14:31:00.150178 26369 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:00.125742 30412 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:00.151679 26332 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:31:00.154549 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:00.152419 27182 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260924 14:31:00.144311 26295 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:00.155094 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.159981 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:00.134955 26297 connection.cc:232] RPC call timeout handler was delayed by -1.1809828800005562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
W20260924 14:31:00.161918 26297 connection.cc:232] RPC call timeout handler was delayed by -1.0405828800003292s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1771s for a response
I20260924 14:31:00.130861 30048 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:00.146963 30685 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260924 14:31:00.132598 30664 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:00.068282 30607 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:00.148335 30662 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:00.170662 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:00.152330 30682 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:00.154392 30628 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260924 14:31:00.189028 26221 connection.cc:232] RPC call timeout handler was delayed by -1.7414023909996104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
I20260924 14:31:00.191026 30274 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:00.192456 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:00.168401 30284 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:00.179812 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:00.195145 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:00.202294 26324 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 12
I20260924 14:31:00.163321 30694 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:00.196695 27188 raft_consensus.cc:2354] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12.
W20260924 14:31:00.191944 26221 connection.cc:232] RPC call timeout handler was delayed by -1.7396023909996075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260924 14:31:00.180820 27191 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:00.167145 30629 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:00.205163 26327 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:00.152087 30647 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260924 14:31:00.214161 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:00.197676 30714 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.108s	user 0.003s	sys 0.000s
I20260924 14:31:00.211199 27123 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.205183 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:00.155968 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:00.211119 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.197036 30274 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:00.216804 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.233619 26331 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:31:00.218454 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:00.203771 30649 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:00.223042 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:00.207162 26221 connection.cc:232] RPC call timeout handler was delayed by -1.6658023910003976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260924 14:31:00.241840 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:00.183967 30662 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:00.168563 30445 consensus_queue.cc:1035] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260924 14:31:00.231669 27182 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:00.226353 26325 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:00.237546 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.166765 30412 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:00.183602 30402 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:00.186273 30664 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:00.246869 30084 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:00.242249 26326 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:00.260526 30412 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:00.188980 30682 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:00.261173 30682 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:00.248667 30681 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:00.246011 30629 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:00.265254 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:00.265201 30629 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:00.265959 26256 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:00.252513 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:00.266800 26325 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:00.242723 26331 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 11
I20260924 14:31:00.223769 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.257026 30287 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:00.267364 30518 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:00.268573 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.269713 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.270304 26327 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:00.245254 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:00.272897 26332 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:31:00.273571 26325 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:00.274152 26332 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260924 14:31:00.248227 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.253023 30649 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:00.276901 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.277314 30649 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:00.277876 30649 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 4 FOLLOWER]: Advancing to term 5
I20260924 14:31:00.280922 30414 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:00.281634 30414 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:31:00.282620 30414 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:31:00.282979 30414 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:00.284948 30664 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:00.285362 30662 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:00.271152 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:00.289891 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 58458us
W20260924 14:31:00.291674 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 135773us
I20260924 14:31:00.292366 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.292730 30385 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:00.293135 30385 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:00.293473 30385 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:00.247797 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.253346 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.298274 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:00.298864 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:00.170758 30048 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:00.253684 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:00.206784 30694 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:00.236176 30274 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:00.295436 26329 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:00.300419 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:00.301403 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:00.223816 30718 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:00.302373 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:00.242494 26221 connection.cc:232] RPC call timeout handler was delayed by -1.6469023910003671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
I20260924 14:31:00.303814 26252 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:00.297464 26326 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:00.228864 30284 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:00.307394 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:00.298194 30634 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:00.323598 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:00.252782 30607 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:00.315245 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:00.316890 26331 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:00.317909 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:00.318094 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:00.318226 26221 connection.cc:232] RPC call timeout handler was delayed by -1.4930023910001182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
W20260924 14:31:00.338835 26221 connection.cc:232] RPC call timeout handler was delayed by -1.3715023909999218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260924 14:31:00.339344 26221 connection.cc:232] RPC call timeout handler was delayed by -1.2752023909997661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
I20260924 14:31:00.307690 30694 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:00.321889 27189 raft_consensus.cc:2354] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:00.246938 26324 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:00.332854 30682 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:00.333940 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:00.339730 26221 connection.cc:232] RPC call timeout handler was delayed by -1.1303023910004413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
I20260924 14:31:00.341058 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.341679 26332 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:31:00.341830 26326 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:00.312925 26256 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:00.342393 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.347679 26252 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:00.325045 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:00.358232 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.358554 26295 leader_election.cc:398] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:00.356323 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:00.359061 26295 leader_election.cc:401] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:00.360189 29682 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:00.359305 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:00.361061 29682 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:00.361724 26400 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:00.362862 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:00.371758 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 59610us
I20260924 14:31:00.372267 30048 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:00.372728 30642 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:00.375206 26399 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:00.376312 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:00.378437 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.381373 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:00.382018 26256 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:31:00.362373 29682 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:00.402674 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:00.405629 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:00.407913 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:00.412633 30385 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:00.397336 30540 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 13
W20260924 14:31:00.419409 30721 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260924 14:31:00.397679 30607 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:00.424187 30607 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:00.425379 30607 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:00.414088 30662 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:00.365237 30518 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:00.431178 30662 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:00.394296 30634 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:00.410728 26256 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:31:00.431965 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:00.432768 27189 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:00.437199 26331 raft_consensus.cc:2354] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:00.437799 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.325261 30718 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:00.440212 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:00.340590 30414 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:00.447207 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:00.399170 30690 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:00.449352 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.453795 30414 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
W20260924 14:31:00.464718 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 90703.2us
I20260924 14:31:00.465621 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.467301 26330 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:00.470069 26327 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:00.471809 26329 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:31:00.475069 26297 leader_election.cc:398] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:00.446519 30664 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:00.476564 26297 leader_election.cc:401] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260924 14:31:00.449033 30333 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election won for term 6
W20260924 14:31:00.482323 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 248734us
W20260924 14:31:00.522351 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 266766us
I20260924 14:31:00.468127 30720 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:00.468149 30048 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:00.530540 26369 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:00.469970 30642 raft_consensus.cc:2741] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260924 14:31:00.532282 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.473572 30719 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:31:00.534433 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 68919.2us
I20260924 14:31:00.535216 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.536149 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.401721 30284 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:00.537257 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62384.6us
I20260924 14:31:00.537977 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.543536 26298 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.521955 30629 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:00.544302 27123 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.547890 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:00.530395 30668 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:00.482661 30518 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260924 14:31:00.537444 30374 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:00.539178 30664 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:00.560531 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.540566 30642 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:00.537154 30333 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:00.574137 30725 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260924 14:31:00.563584 30683 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:00.567420 30417 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:00.568325 30697 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260924 14:31:00.558207 30693 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:00.585237 30333 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:00.534475 30048 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:00.639617 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 79198.9us
I20260924 14:31:00.640533 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.641882 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:00.642992 30640 raft_consensus.cc:2741] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260924 14:31:00.643759 30640 raft_consensus.cc:2741] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260924 14:31:00.556562 30719 raft_consensus.cc:2741] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260924 14:31:00.557780 30412 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:00.544278 30077 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260924 14:31:00.572072 30637 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260924 14:31:00.596531 30682 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:00.589609 30659 raft_consensus.cc:2741] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
W20260924 14:31:00.654670 30724 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260924 14:31:00.598237 30683 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:00.596227 30374 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:00.776278 30374 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:00.721369 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.773526 30683 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:00.780427 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 59130.8us
I20260924 14:31:00.781293 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:00.701907 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 163977us
W20260924 14:31:00.722404 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 66971.1us
I20260924 14:31:00.784011 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:00.784933 30417 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:31:00.785495 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:00.786430 30417 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:31:00.786893 26297 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:00.639142 30274 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:00.788048 27186 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:00.789091 30274 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:00.580003 30411 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.123s
I20260924 14:31:00.790561 26221 leader_election.cc:398] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12.
I20260924 14:31:00.791386 26221 leader_election.cc:401] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260924 14:31:00.791899 30690 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260924 14:31:00.657922 30717 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260924 14:31:00.754030 30718 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:00.628876 30385 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:00.645983 30681 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:00.794004 30681 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:00.795571 26297 leader_election.cc:398] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12.
I20260924 14:31:00.800536 26297 leader_election.cc:401] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:00.803581 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:00.800263 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 256753us
I20260924 14:31:00.807539 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:00.808058 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.812103 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:00.813983 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:00.814427 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:00.814756 26253 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:00.816998 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:00.818717 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:00.819533 26251 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:00.820444 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:00.821553 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:00.821554 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:00.826704 26250 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260924 14:31:00.603900 27189 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:00.838347 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:00.840018 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:00.783540 30659 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:00.824246 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.848754 27188 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260924 14:31:00.807394 30733 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.056s	user 0.001s	sys 0.001s
I20260924 14:31:00.856941 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:00.859122 26249 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:00.863901 26251 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:00.866261 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:00.867058 26400 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:00.811704 30077 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:00.812299 30637 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12.
I20260924 14:31:00.812574 30718 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12.
I20260924 14:31:00.875845 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:00.837101 30417 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:00.849005 30697 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:00.848731 30719 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:00.892290 26295 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:00.827965 30642 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
W20260924 14:31:00.904994 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 80866.8us
I20260924 14:31:00.906057 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.903107 26256 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:00.907661 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:00.863042 30048 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:00.880378 30540 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:00.898012 30697 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:00.892683 30417 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:00.893628 30681 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:00.961817 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:00.984705 26403 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
W20260924 14:31:00.986337 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 70075.1us
W20260924 14:31:00.986701 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 157129us
W20260924 14:31:00.997884 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 72053.4us
I20260924 14:31:00.917488 30659 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:00.911433 30637 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:00.915127 30718 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election won for term 11
W20260924 14:31:00.993332 30649 consensus_meta.cc:287] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.002s
I20260924 14:31:01.020102 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:01.022801 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:01.023948 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:01.024284 26329 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 12
I20260924 14:31:01.024780 26324 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:00.988866 30697 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:00.985532 30417 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:00.997725 30385 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:01.102416 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 66139.1us
W20260924 14:31:01.075878 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 44215.2us
I20260924 14:31:01.037395 26332 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:01.107862 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:01.108927 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:01.113147 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:01.109480 26332 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:31:01.128693 26324 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:01.131847 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:01.136034 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:01.019099 30566 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:01.015882 30662 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:01.031672 30649 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260924 14:31:01.220997 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 312601us
I20260924 14:31:01.225668 26369 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:01.226221 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:01.230741 26329 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:01.239116 26404 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
W20260924 14:31:01.239331 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 101191us
I20260924 14:31:01.246174 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:01.251634 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:01.255172 30642 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:01.207487 30374 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:01.255714 30659 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:31:01.256243 30374 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:01.257251 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:01.258656 26221 leader_election.cc:398] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:01.259953 30659 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:01.260277 30731 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260924 14:31:01.260838 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:01.261906 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1348) took 1140 ms. Trace:
I20260924 14:31:01.262607 30659 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:31:01.262367 26406 rpcz_store.cc:276] 0924 14:31:00.121250 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:00.121846 (+   596us) service_pool.cc:227] Handling call
0924 14:31:00.122255 (+   409us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:00.122675 (+   420us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:00.122714 (+    39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:00.123285 (+   571us) log.cc:842] Serialized 26 byte log entry
0924 14:31:00.450462 (+327177us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:00.450516 (+    54us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:00.479598 (+ 29082us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:01.261624 (+782026us) raft_consensus.cc:1671] finished
0924 14:31:01.261652 (+    28us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:01.261832 (+   180us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":3965,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":8089,"thread_start_us":326682,"threads_started":1,"wal-append.queue_time_us":750672}
I20260924 14:31:01.218425 30681 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:01.277720 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:01.270102 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:01.270453 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:01.278712 26251 raft_consensus.cc:2362] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10. Re-sending same reply.
I20260924 14:31:01.279879 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:01.278815 26400 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:01.280704 26251 raft_consensus.cc:2354] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12.
I20260924 14:31:01.270865 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:01.281644 26254 raft_consensus.cc:2362] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11. Re-sending same reply.
I20260924 14:31:01.282598 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:01.271093 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:01.279239 30374 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:01.283814 26253 raft_consensus.cc:2354] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:01.276890 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:01.268371 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:01.294000 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:01.294224 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:01.294936 26251 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:01.295694 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:01.297072 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:01.297860 26252 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:01.303984 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 184533us
I20260924 14:31:01.275787 30649 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:01.278398 30681 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:01.282330 30647 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election won for term 11
W20260924 14:31:01.288838 30722 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.923s	user 0.003s	sys 0.000s
I20260924 14:31:01.224083 30662 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:01.326287 26251 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:01.334901 26255 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:01.245316 30690 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260924 14:31:01.292483 30718 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:31:01.330919 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 339219us
I20260924 14:31:01.330849 26258 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:31:01.330108 26252 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:01.303471 30374 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:01.277832 30544 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260924 14:31:01.345196 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 141473us
I20260924 14:31:01.276120 30693 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260924 14:31:01.347054 26329 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:31:01.288141 30642 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260924 14:31:01.280776 30637 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
W20260924 14:31:01.365193 30727 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260924 14:31:01.366981 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1410) took 1293 ms. Trace:
I20260924 14:31:01.367511 26323 rpcz_store.cc:276] 0924 14:31:00.073583 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:00.073700 (+   117us) service_pool.cc:227] Handling call
0924 14:31:00.096078 (+ 22378us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:00.138866 (+ 42788us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:00.138896 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:00.139324 (+   428us) log.cc:842] Serialized 26 byte log entry
0924 14:31:00.617828 (+478504us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:00.617877 (+    49us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:00.617953 (+    76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:01.366676 (+748723us) raft_consensus.cc:1671] finished
0924 14:31:01.366707 (+    31us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:01.366905 (+   198us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":42408,"raft.queue_time_us":42541,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":11211,"thread_start_us":424989,"threads_started":1,"wal-append.queue_time_us":830115}
W20260924 14:31:01.370628 30745 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260924 14:31:01.322189 30649 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:01.388438 30649 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:01.355500 26329 raft_consensus.cc:595] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
W20260924 14:31:01.392621 30734 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.390s	user 0.001s	sys 0.000s
I20260924 14:31:01.394665 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 849) took 1200 ms. Trace:
I20260924 14:31:01.395026 26328 rpcz_store.cc:276] 0924 14:31:00.194543 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:00.205951 (+ 11408us) service_pool.cc:227] Handling call
0924 14:31:00.226972 (+ 21021us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:00.312171 (+ 85199us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:00.312202 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:00.319063 (+  6861us) log.cc:842] Serialized 26 byte log entry
0924 14:31:00.568326 (+249263us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:00.568391 (+    65us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:00.568491 (+   100us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:01.394460 (+825969us) raft_consensus.cc:1671] finished
0924 14:31:01.394479 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:01.394607 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":77087,"raft.queue_time_us":80307,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":7744,"thread_start_us":183752,"threads_started":1,"wal-append.queue_time_us":562163}
W20260924 14:31:01.403743 30728 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.505s	user 0.000s	sys 0.001s
I20260924 14:31:01.405351 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 762) took 1280 ms. Trace:
I20260924 14:31:01.405732 27181 rpcz_store.cc:276] 0924 14:31:00.124852 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:00.125389 (+   537us) service_pool.cc:227] Handling call
0924 14:31:00.142116 (+ 16727us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:00.160597 (+ 18481us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:00.160657 (+    60us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:00.191790 (+ 31133us) log.cc:842] Serialized 26 byte log entry
0924 14:31:00.529589 (+337799us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:00.529625 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:00.529693 (+    68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:01.405103 (+875410us) raft_consensus.cc:1671] finished
0924 14:31:01.405125 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:01.405287 (+   162us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":30094,"raft.queue_time_us":18747,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":5758,"thread_start_us":306842,"threads_started":1,"wal-append.queue_time_us":629290}
I20260924 14:31:01.409976 30649 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:01.328245 30681 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:01.412905 30649 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:01.417167 30402 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:01.417829 30402 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:01.420601 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 741) took 1231 ms. Trace:
I20260924 14:31:01.421288 27186 rpcz_store.cc:276] 0924 14:31:00.189515 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:00.189685 (+   170us) service_pool.cc:227] Handling call
0924 14:31:00.211788 (+ 22103us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:00.788517 (+576729us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:01.420176 (+631659us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b540000f854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:01.420491 (+   315us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1324661120}
I20260924 14:31:01.344422 30732 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:01.344985 30647 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:31:01.453452 30735 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.436s	user 0.002s	sys 0.000s
I20260924 14:31:01.455367 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:01.462097 26324 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260924 14:31:01.469825 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 184682us
I20260924 14:31:01.471026 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:01.483197 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:01.484308 26258 raft_consensus.cc:2354] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
W20260924 14:31:01.437312 30722 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260924 14:31:01.489605 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 212798us
I20260924 14:31:01.492605 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:01.441573 30748 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260924 14:31:01.497614 26297 leader_election.cc:398] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12.
I20260924 14:31:01.498791 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:01.469184 30659 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:01.498945 30647 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:01.500890 26329 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260924 14:31:01.502224 30414 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.034s	user 0.011s	sys 0.000s
I20260924 14:31:01.504199 26324 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:01.498956 30628 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:31:01.498808 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:01.498869 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:01.507665 26252 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:01.548754 26249 raft_consensus.cc:2362] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11. Re-sending same reply.
I20260924 14:31:01.457082 30662 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:01.511638 30649 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:01.517534 30741 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:01.556782 30414 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:01.522395 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:01.569602 30414 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:01.569885 30576 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:31:01.571403 30576 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:01.571805 30414 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:01.418941 30402 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:01.546446 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:01.509970 30683 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.729s	user 0.005s	sys 0.005s
I20260924 14:31:01.563503 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:01.555022 26404 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:31:01.569003 30628 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:01.524947 26325 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:31:01.576900 26324 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260924 14:31:01.578835 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:01.533602 26297 leader_election.cc:401] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:01.545356 30750 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election won for term 12
W20260924 14:31:01.589071 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 91986.4us
W20260924 14:31:01.554512 30720 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.015s	user 0.008s	sys 0.005s
I20260924 14:31:01.582731 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:01.547749 30690 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:01.590884 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 295470us
I20260924 14:31:01.597433 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:01.599386 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:01.596134 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:01.549883 30048 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260924 14:31:01.545715 30659 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:01.616562 26297 leader_election.cc:398] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:01.617911 30659 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:01.618613 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:01.620368 30681 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:01.622745 30681 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:01.602092 30750 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:01.600548 30690 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:01.624585 30750 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:01.590332 30683 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:01.628782 30683 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:01.600948 30720 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:01.587487 30414 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:01.581667 30741 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:01.600713 30374 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:01.632176 27181 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:01.632827 30374 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:01.615288 30697 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:01.576737 30718 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:01.640509 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 1353) took 1318 ms. Trace:
W20260924 14:31:01.641703 26330 consensus_meta.cc:287] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.001s
I20260924 14:31:01.641295 26404 rpcz_store.cc:276] 0924 14:31:00.294586 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:00.303087 (+  8501us) service_pool.cc:227] Handling call
0924 14:31:01.612330 (+1309243us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54001e41d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:01.612526 (+   196us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2038657792}
I20260924 14:31:01.576829 30642 consensus_queue.cc:1225] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260924 14:31:01.592159 30662 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12.
I20260924 14:31:01.594406 30566 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:31:01.642313 26330 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
W20260924 14:31:01.645568 30417 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.529s	user 0.006s	sys 0.005s
I20260924 14:31:01.648469 30751 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
W20260924 14:31:01.650046 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 47510.1us
I20260924 14:31:01.604228 30730 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:01.577603 30649 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:01.658160 26258 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
W20260924 14:31:01.663000 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 58903.3us
I20260924 14:31:01.632174 30414 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:01.668385 26252 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:01.651690 30739 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election won for term 11
W20260924 14:31:01.661571 30722 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260924 14:31:01.675377 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 56828.7us
I20260924 14:31:01.651937 30754 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:01.665441 30751 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:01.683691 30759 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260924 14:31:01.675818 30414 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:01.687621 27124 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.005s	sys 0.021s
W20260924 14:31:01.703325 30694 consensus_meta.cc:287] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.327s	user 0.015s	sys 0.000s
I20260924 14:31:01.689383 30628 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:01.690008 26404 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260924 14:31:01.692095 30739 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:01.692723 26397 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:01.682128 30084 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:01.761214 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:01.692869 30741 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:01.693819 30649 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:01.694257 30697 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:01.694751 30752 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:31:01.695845 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:01.659070 30417 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:01.696374 26369 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:01.699404 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:01.716548 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:01.659014 30690 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:01.721606 26258 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
W20260924 14:31:01.727877 27124 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.005s	sys 0.022s
I20260924 14:31:01.727618 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:01.648346 30640 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260924 14:31:01.728288 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:01.762140 26252 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:01.762300 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1461) took 1832 ms. Trace:
I20260924 14:31:01.682983 30681 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
W20260924 14:31:01.825860 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 109454us
I20260924 14:31:01.827589 26249 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:01.832499 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 136198us
I20260924 14:31:01.829988 26257 rpcz_store.cc:276] 0924 14:30:59.929398 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:59.929587 (+   189us) service_pool.cc:227] Handling call
0924 14:30:59.929984 (+   397us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:30:59.930413 (+   429us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:30:59.930694 (+   281us) log.cc:842] Serialized 14 byte log entry
0924 14:30:59.940595 (+  9901us) spinlock_profiling.cc:243] Waited 8.37 ms on lock 0x7b180000c1c0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:31:00.212364 (+271769us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:00.234058 (+ 21694us) log.cc:842] Serialized 52 byte log entry
0924 14:31:00.234192 (+   134us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:00.234431 (+   239us) log.cc:842] Serialized 14 byte log entry
0924 14:31:00.237015 (+  2584us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:00.237124 (+   109us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:00.894486 (+657362us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:01.762048 (+867562us) raft_consensus.cc:1671] finished
0924 14:31:01.762070 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:01.762234 (+   164us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2234,"prepare.queue_time_us":1172,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":14901,"raft.queue_time_us":34979,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":179,"spinlock_wait_cycles":24652928,"thread_start_us":281294,"threads_started":1,"wal-append.queue_time_us":386764}
I20260924 14:31:01.873661 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:01.824184 30762 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:01.824826 30697 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:01.766805 30349 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260924 14:31:01.885155 26297 leader_election.cc:398] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
I20260924 14:31:01.899883 26297 leader_election.cc:401] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:01.886628 26249 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
W20260924 14:31:01.875630 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48003us
I20260924 14:31:01.834257 30649 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:01.767241 30730 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.109s
I20260924 14:31:01.904807 26330 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260924 14:31:01.898124 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 73710us
I20260924 14:31:01.762121 30694 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:01.800585 30693 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260924 14:31:01.767144 30084 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:01.902613 30697 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:01.911193 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:01.891208 26252 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:01.882187 30385 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:01.920496 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:01.921442 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:01.929219 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:01.935124 30720 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:01.935721 30720 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:01.936221 30720 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:01.938822 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:01.908502 30662 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
I20260924 14:31:01.942428 30694 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:01.943037 26399 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:01.943620 30760 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:01.945412 30417 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:01.946657 29989 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:01.947156 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:01.933060 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:01.926321 30640 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260924 14:31:01.961790 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:01.967036 30414 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:01.971339 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:01.978087 26252 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:01.983973 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:31:01.973706 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48927.4us
I20260924 14:31:01.974908 26404 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:31:01.998284 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:01.898085 30642 consensus_queue.cc:1046] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260924 14:31:01.999155 26400 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:02.000110 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:02.001765 26253 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260924 14:31:02.001188 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:01.950697 30755 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:01.998414 30760 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:02.013285 30694 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:02.013799 30383 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:31:02.003448 30640 consensus_queue.cc:1225] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260924 14:31:02.015173 29989 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:31:01.888180 30739 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:01.902076 30762 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:02.034551 26249 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:01.999681 30754 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:02.003159 30649 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:02.028417 30417 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:02.030651 30694 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:02.055976 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 49549.6us
I20260924 14:31:02.038697 30769 raft_consensus.cc:3032] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:01.907649 30751 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:02.039144 26400 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
W20260924 14:31:02.069852 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51981.3us
I20260924 14:31:02.075377 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:02.030786 30048 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260924 14:31:02.085235 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 84179us
I20260924 14:31:02.086576 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:02.052315 30754 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:02.095937 30319 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:02.098681 30319 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:31:02.068509 30694 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:02.089392 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:02.089494 30628 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:02.063488 30663 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:02.066715 30769 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:02.110954 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5581950b69104df485f842c6b78b1047"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:02.116115 27189 raft_consensus.cc:595] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:02.116528 27189 raft_consensus.cc:603] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:02.109037 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:02.117255 27188 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:02.087106 30753 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:02.134673 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:02.135528 26249 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:02.138716 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:02.144984 27192 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:02.066170 30417 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:02.153016 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:02.151408 27188 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:31:02.158471 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:02.134802 30773 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.254s	user 0.001s	sys 0.001s
I20260924 14:31:02.101179 30741 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
W20260924 14:31:02.145213 30757 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.276s	user 0.002s	sys 0.000s
I20260924 14:31:02.168865 26323 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:02.165413 26221 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:02.174957 26325 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:02.176489 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:02.139147 30319 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:02.184087 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:02.145826 30628 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:02.117892 30663 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:02.205243 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:02.158449 30084 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:02.202746 27191 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "64e0b4058f18400dab940d3832586746"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
W20260924 14:31:02.173216 30764 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.292s	user 0.000s	sys 0.002s
I20260924 14:31:02.275204 26325 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:02.165174 30414 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:02.209014 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:02.273344 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:02.191432 30649 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:02.275146 27191 raft_consensus.cc:595] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:02.169858 30697 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:02.188668 30659 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260924 14:31:02.293773 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1354) took 1752 ms. Trace:
I20260924 14:31:02.310341 26399 rpcz_store.cc:276] 0924 14:31:00.541214 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:00.541514 (+   300us) service_pool.cc:227] Handling call
0924 14:31:00.562709 (+ 21195us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:01.957659 (+1394950us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:02.293406 (+335747us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001e50d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:02.293681 (+   275us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3173864960}
I20260924 14:31:02.134135 30782 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.072s
W20260924 14:31:02.314293 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 173533us
W20260924 14:31:02.321259 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 164112us
I20260924 14:31:02.141538 30753 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:02.323935 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:02.112753 30760 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:02.327706 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:02.272416 30640 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:02.348505 30783 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260924 14:31:02.276374 30414 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:02.360934 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:02.394166 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:02.395190 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:02.273658 30754 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:02.399209 27181 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260924 14:31:02.400225 26325 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:02.416363 26257 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260924 14:31:02.395530 30779 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260924 14:31:02.499042 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:02.394829 30414 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:31:02.506134 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 231428us
I20260924 14:31:02.500000 26324 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
W20260924 14:31:02.418519 30734 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260924 14:31:02.498451 30764 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260924 14:31:02.498835 30757 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260924 14:31:02.418282 30762 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:02.457983 30750 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:02.337415 30786 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.173s
I20260924 14:31:02.542321 26323 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260924 14:31:02.543929 30790 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260924 14:31:02.544504 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1430) took 1517 ms. Trace:
I20260924 14:31:02.544827 30753 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:02.545240 30781 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.404s	user 0.001s	sys 0.000s
W20260924 14:31:02.546404 30780 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260924 14:31:02.399286 30540 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.128s
I20260924 14:31:02.189378 30769 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:02.528167 30741 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:02.618538 26332 rpcz_store.cc:276] 0924 14:31:01.027626 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:01.114767 (+ 87141us) service_pool.cc:227] Handling call
0924 14:31:01.115132 (+   365us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:01.133982 (+ 18850us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:01.134015 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:01.250114 (+116099us) log.cc:842] Serialized 26 byte log entry
0924 14:31:01.656447 (+406333us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:01.676242 (+ 19795us) log.cc:842] Serialized 14 byte log entry
0924 14:31:01.676442 (+   200us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:01.676516 (+    74us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:02.544270 (+867754us) raft_consensus.cc:1671] finished
0924 14:31:02.544292 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:02.544441 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":18392,"prepare.queue_time_us":14860,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":114358,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":24613,"thread_start_us":335193,"threads_started":1,"wal-append.queue_time_us":494084}
I20260924 14:31:02.644093 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:02.640427 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 865) took 1620 ms. Trace:
I20260924 14:31:02.645519 26327 rpcz_store.cc:276] 0924 14:31:01.019973 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:01.020503 (+   530us) service_pool.cc:227] Handling call
0924 14:31:01.020864 (+   361us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:01.046933 (+ 26069us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:01.046989 (+    56us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:01.154693 (+107704us) log.cc:842] Serialized 26 byte log entry
0924 14:31:01.588335 (+433642us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:01.645205 (+ 56870us) log.cc:842] Serialized 14 byte log entry
0924 14:31:01.656211 (+ 11006us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:01.656291 (+    80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:02.640178 (+983887us) raft_consensus.cc:1671] finished
0924 14:31:02.640203 (+    25us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:02.640362 (+   159us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11731,"prepare.queue_time_us":56646,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":14083,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":77111,"thread_start_us":324307,"threads_started":1,"wal-append.queue_time_us":545378}
I20260924 14:31:02.644999 26323 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:02.618167 30762 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:02.596668 30681 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:02.652570 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 374483us
I20260924 14:31:02.643432 30662 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:02.669665 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:02.651707 30741 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:02.515633 30628 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:02.617767 30750 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:02.679051 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:02.681229 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:02.526176 30649 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:02.655725 30417 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:02.704097 26250 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:02.665695 30762 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:02.668068 30681 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:02.708556 30663 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:02.674655 30662 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:02.748884 26255 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.004s	sys 0.012s
W20260924 14:31:02.776129 26255 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.004s	sys 0.012s
I20260924 14:31:02.777068 30751 raft_consensus.cc:979] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:31:02.777683 30751 raft_consensus.cc:1067] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
W20260924 14:31:02.743945 30732 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.000s	sys 0.006s
I20260924 14:31:02.787603 26253 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "64e0b4058f18400dab940d3832586746"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
I20260924 14:31:02.788566 26253 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:31:02.789037 26253 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:02.731488 30787 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.066s
W20260924 14:31:02.784852 30732 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.001s	sys 0.006s
W20260924 14:31:02.812369 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.000s	sys 0.003s
W20260924 14:31:02.813138 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.000s	sys 0.004s
W20260924 14:31:02.819027 26296 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.004s	sys 0.035s
W20260924 14:31:02.819569 26296 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.005s	sys 0.035s
W20260924 14:31:02.828032 30777 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260924 14:31:02.831194 30756 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260924 14:31:02.838650 30662 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:02.796192 30798 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:02.840549 30813 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:02.846714 30720 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.910s	user 0.014s	sys 0.001s
W20260924 14:31:02.900238 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 207044us
W20260924 14:31:02.868646 30694 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.748s	user 0.006s	sys 0.014s
W20260924 14:31:02.905128 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 237524us
I20260924 14:31:02.926121 26369 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:02.928174 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:02.932116 30813 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:02.933992 26369 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:02.929910 30732 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:31:02.947834 26253 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:02.849514 30798 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:02.962239 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:02.922766 30739 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.049s
W20260924 14:31:02.971710 30820 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260924 14:31:02.978066 30824 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:02.984194 30803 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:02.922360 30694 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:02.922597 30741 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:02.996443 30750 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:02.998948 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 64876.6us
I20260924 14:31:02.997043 26404 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:03.009428 26253 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:02.901620 30720 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:03.012353 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 109669us
I20260924 14:31:03.014669 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:03.020035 26398 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:03.033854 30827 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.034276 26257 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:03.034521 30827 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:03.037243 30827 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.038573 30829 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.039198 30829 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:03.041524 30829 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:03.042141 30829 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:03.042558 26328 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260924 14:31:03.010721 30793 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:31:03.010413 30762 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:03.055291 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:03.024009 30792 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.705s	user 0.003s	sys 0.007s
I20260924 14:31:03.060384 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:03.006094 30414 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:03.044236 30798 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.042639 30829 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:03.022176 30803 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.060564 30793 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:03.058357 30681 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:03.163213 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 105965us
I20260924 14:31:03.154206 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1511) took 1132 ms. Trace:
I20260924 14:31:03.184158 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:03.051409 30732 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260924 14:31:03.187235 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:03.188151 26328 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:03.187661 26255 rpcz_store.cc:276] 0924 14:31:02.021975 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:02.022111 (+   136us) service_pool.cc:227] Handling call
0924 14:31:02.022509 (+   398us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:02.048060 (+ 25551us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:02.048111 (+    51us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:02.074052 (+ 25941us) log.cc:842] Serialized 26 byte log entry
0924 14:31:02.776639 (+702587us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:02.776688 (+    49us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:02.776771 (+    83us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:03.153917 (+377146us) raft_consensus.cc:1671] finished
0924 14:31:03.153946 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:03.154133 (+   187us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":33617,"raft.queue_time_us":25746,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":10763,"thread_start_us":670877,"threads_started":1,"wal-append.queue_time_us":712774}
I20260924 14:31:03.060120 30741 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.123s
I20260924 14:31:03.020401 30824 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:03.218067 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:03.183353 30739 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.184952 30800 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.189491 30804 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:03.185324 30843 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.214684 30374 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.288142 30374 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:03.290479 30838 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.145s
I20260924 14:31:03.226574 30681 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.292040 30681 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:03.292546 30838 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:03.226838 30762 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.293659 30762 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:03.294673 30815 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.295269 30815 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.304420 30842 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.142s
I20260924 14:31:03.298116 30649 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.223250 30798 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:03.376231 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 189103us
I20260924 14:31:03.397979 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:03.409917 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:03.302259 30739 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:03.416889 26326 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:31:03.383546 30732 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:03.426223 30836 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260924 14:31:03.366029 30793 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:03.431635 30855 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.431564 30844 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.208s
I20260924 14:31:03.394055 30374 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.388298 30835 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:03.438711 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:03.446856 26399 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:03.412369 30694 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:03.300402 30804 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:03.302959 30843 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.257606 30800 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:03.433081 30832 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.434829 30741 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:03.386480 30815 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:03.399567 30798 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.464134 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:03.473809 30833 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:03.526476 26331 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:03.526541 30833 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.529165 30833 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:03.474467 30694 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.529908 30694 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.532014 30694 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:03.474582 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 886) took 1309 ms. Trace:
I20260924 14:31:03.533646 26330 rpcz_store.cc:276] 0924 14:31:02.164730 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:02.455643 (+290913us) service_pool.cc:227] Handling call
0924 14:31:02.456155 (+   512us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:02.722085 (+265930us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:02.722116 (+    31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:02.722156 (+    40us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:02.722187 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:03.474265 (+752078us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54005bb8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:03.474328 (+    63us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:03.474515 (+   187us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":611047808}
I20260924 14:31:03.432948 30855 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.536420 30855 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:03.436157 30827 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:03.536867 30694 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.537161 30827 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:03.537731 30694 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.450984 30720 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.402285 30649 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.539621 26331 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:03.540987 30694 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:03.541575 30649 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:03.543555 30827 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:03.477074 30845 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260924 14:31:03.452235 30835 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.478030 30851 raft_consensus.cc:3032] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:03.463858 30414 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:03.475365 30840 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:03.557461 26371 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.866s (SENT) [suppressed 14 similar messages]
I20260924 14:31:03.479151 30829 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:03.436303 30732 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.568233 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:03.569351 26404 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:03.550870 30842 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.554428 30815 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.628484 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:03.557540 30851 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:03.546640 30741 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:03.555258 30804 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:03.550392 30832 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.649623 30832 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.651085 26253 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:03.572593 30840 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.651883 30863 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:03.652595 30863 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:03.653508 30840 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.627916 30815 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:03.633173 30842 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:03.655993 30823 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260924 14:31:03.657209 30839 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:31:03.657537 30842 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.657707 30839 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:03.658145 30842 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:03.620011 30414 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:03.646708 30841 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260924 14:31:03.565572 30803 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.655232 30863 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:03.658654 30839 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:03.662320 30827 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.665043 30832 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.667903 30720 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.658601 30842 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:03.726584 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 169141us
I20260924 14:31:03.728796 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1080) took 1774 ms. Trace:
I20260924 14:31:03.729269 26251 rpcz_store.cc:276] 0924 14:31:01.954106 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:01.954226 (+   120us) service_pool.cc:227] Handling call
0924 14:31:01.954597 (+   371us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:01.961408 (+  6811us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:01.961444 (+    36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:02.808001 (+846557us) spinlock_profiling.cc:243] Waited 9.57 ms on lock 0x7b2c001c3658. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:02.842697 (+ 34696us) log.cc:842] Serialized 26 byte log entry
0924 14:31:03.065825 (+223128us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:03.068037 (+  2212us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:03.068126 (+    89us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:03.723905 (+655779us) raft_consensus.cc:1671] finished
0924 14:31:03.723938 (+    33us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:03.728724 (+  4786us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2122,"raft.queue_time_us":5907,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":22016384,"thread_start_us":160818,"threads_started":1,"wal-append.queue_time_us":569000}
I20260924 14:31:03.731796 26326 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:03.732290 26326 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:03.669100 30414 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:03.739902 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:03.741259 30856 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.715055 30750 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.738057 26326 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:03.756464 26258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1075) took 2234 ms (client timeout 1866 ms). Trace:
I20260924 14:31:03.733247 30839 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:03.761037 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:03.765971 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:03.735127 30333 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:03.716360 30827 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:03.742669 30794 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:03.724877 30832 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:03.731339 30720 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:03.685485 30851 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:03.843952 30864 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260924 14:31:03.786741 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:03.825879 30628 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:04.608937 30628 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:04.609463 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.610113 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1510) took 2594 ms. Trace:
I20260924 14:31:04.418525 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1608) took 1203 ms. Trace:
I20260924 14:31:04.610688 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:04.491153 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.010s	sys 0.009s
W20260924 14:31:04.417456 26283 heartbeater.cc:644] Failed to heartbeat to 127.22.151.254:43789 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.22.151.254:43789 timed out after 15.000s (SENT)
I20260924 14:31:04.610532 26252 rpcz_store.cc:276] 0924 14:31:02.015534 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:02.025784 (+ 10250us) service_pool.cc:227] Handling call
0924 14:31:02.026199 (+   415us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:02.037409 (+ 11210us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:31:02.048511 (+ 11102us) log.cc:842] Serialized 14 byte log entry
0924 14:31:03.433038 (+1384527us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b18000b4280. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd557b 00007f09b935888a
0924 14:31:03.561393 (+128355us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:03.562228 (+   835us) log.cc:842] Serialized 26 byte log entry
0924 14:31:03.562463 (+   235us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:03.562502 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:03.562587 (+    85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:04.609750 (+1047163us) raft_consensus.cc:1671] finished
0924 14:31:04.609876 (+   126us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:04.610027 (+   151us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":19771,"prepare.queue_time_us":11100,"raft.queue_time_us":100343,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":5875,"spinlock_wait_cycles":29668864,"thread_start_us":127925,"threads_started":1,"wal-append.queue_time_us":1674246}
I20260924 14:31:04.614571 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.615321 27194 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:03.944643 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 183780us
I20260924 14:31:03.817485 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:04.621170 26249 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:04.622295 26403 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:04.036049 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:04.627202 26255 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:31:04.033768 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 268147us
W20260924 14:31:04.612787 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.012s	sys 0.010s
I20260924 14:31:03.822543 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.611305 26399 rpcz_store.cc:276] 0924 14:31:03.215211 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.219164 (+  3953us) service_pool.cc:227] Handling call
0924 14:31:03.227426 (+  8262us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:03.449267 (+221841us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:04.418197 (+968930us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400475c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:04.418451 (+   254us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":503817984}
I20260924 14:31:04.032467 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.640429 26404 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 18: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:03.786895 26258 rpcz_store.cc:269] 0924 14:31:01.521953 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:01.522070 (+   117us) service_pool.cc:227] Handling call
0924 14:31:03.749031 (+2226961us) spinlock_profiling.cc:243] Waited 88.4 ms on lock 0x7b54004f9f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:03.756336 (+  7305us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":203396480}
I20260924 14:31:04.629568 26253 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:04.642010 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1227) took 2533 ms. Trace:
I20260924 14:31:04.642581 27181 rpcz_store.cc:276] 0924 14:31:02.078458 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:02.078649 (+   191us) service_pool.cc:227] Handling call
0924 14:31:02.102207 (+ 23558us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:02.399702 (+297495us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:04.611272 (+2211570us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54002008d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:04.611553 (+   281us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":675164544}
I20260924 14:31:03.745077 30739 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.763616 30718 raft_consensus.cc:979] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:31:04.657945 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.658372 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.658965 26258 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:04.660323 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.649919 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:04.663205 26406 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:31:04.664455 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:04.665279 26406 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
W20260924 14:31:03.843132 30859 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260924 14:31:04.641472 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:04.653991 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:04.675952 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.676340 26252 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:04.676545 26403 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:04.676890 27192 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:04.661123 26258 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:31:03.879927 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.667920 26331 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:31:04.668655 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.669405 26249 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:04.680866 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.682494 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:04.623291 30754 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.021s	sys 0.015s
I20260924 14:31:03.863384 30843 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.904618 30835 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.824126 30863 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.826121 30860 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.901708 30803 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:03.870610 30798 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:04.034775 30741 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:03.910686 30690 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:04.497081 30867 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:03.920893 30815 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:04.659287 26253 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:04.756916 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.757848 26253 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:31:04.690286 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.202543 30842 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:03.862422 30824 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.688233 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:04.648878 30833 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:04.691062 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:04.770054 26258 raft_consensus.cc:2362] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9. Re-sending same reply.
I20260924 14:31:04.771003 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.693068 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:04.702380 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 72868.3us
W20260924 14:31:04.693780 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 77704us
W20260924 14:31:04.773053 26222 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.009s	sys 0.020s
W20260924 14:31:04.773470 26222 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.009s	sys 0.020s
I20260924 14:31:04.777032 30815 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:04.777693 30815 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:04.778512 30769 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:04.760049 30867 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:04.781680 30867 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:04.727843 30860 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:04.759588 30690 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:04.784530 30860 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.783259 30833 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:03.890170 30732 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:04.706125 26255 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:04.740871 30414 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:04.693059 30754 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.021s	sys 0.015s
I20260924 14:31:04.710448 30869 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:04.717005 30877 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:04.711274 30863 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:04.718878 30827 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:03.825655 30856 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:04.720908 30832 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:04.721714 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.722536 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.653304 30333 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:04.787392 30824 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:04.648224 30794 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:03.824879 30750 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:04.711071 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 745) took 2596 ms. Trace:
I20260924 14:31:04.839614 30876 raft_consensus.cc:1067] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
W20260924 14:31:04.675598 30865 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.813s	user 0.001s	sys 0.000s
I20260924 14:31:04.741207 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:04.705201 30875 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:04.702337 30739 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:04.690001 30861 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:04.728814 30798 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:04.761240 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.764952 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:04.764971 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 893) took 1349 ms. Trace:
I20260924 14:31:04.765843 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.745944 30793 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:04.768709 26399 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:04.895895 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1613) took 1101 ms. Trace:
I20260924 14:31:04.896344 26399 rpcz_store.cc:276] 0924 14:31:03.794589 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:04.651371 (+856782us) service_pool.cc:227] Handling call
0924 14:31:04.895803 (+244432us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:04.793432 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.714579 30803 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:04.740082 30720 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:04.804263 26296 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.013s	sys 0.020s
W20260924 14:31:04.901605 26296 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.014s	sys 0.020s
I20260924 14:31:04.757452 30628 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:04.803223 26403 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:04.903513 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1612) took 1135 ms. Trace:
I20260924 14:31:04.786918 30842 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:04.788506 30843 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:04.724532 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.818259 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.825546 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:04.786140 30815 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.803694 30829 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:04.912472 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 87531.7us
I20260924 14:31:04.810896 30867 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:04.846611 27191 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:04.847632 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.815557 30860 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:04.823957 30414 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:04.825040 26249 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:31:04.851102 27182 rpcz_store.cc:276] 0924 14:31:02.114404 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:02.114589 (+   185us) service_pool.cc:227] Handling call
0924 14:31:02.137083 (+ 22494us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:02.202767 (+ 65684us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:02.202799 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:02.202838 (+    39us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:02.202875 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:04.706705 (+2503830us) spinlock_profiling.cc:243] Waited 65.4 ms on lock 0x7b54002c51d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:04.706773 (+    68us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:04.710984 (+  4211us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":150500736}
I20260924 14:31:04.711822 30835 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:04.924355 30835 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:04.839437 30718 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:04.809166 30877 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:04.926584 30718 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:04.928591 30877 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:04.847494 30732 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:04.929562 30732 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:04.932596 30732 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.856496 26331 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:04.806190 30869 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:04.838614 30824 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:04.934527 26257 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 5.
I20260924 14:31:04.850894 30881 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:04.863646 26256 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:04.935930 26256 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:04.935844 30881 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:04.852394 30833 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.937134 26256 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:04.938423 26256 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:04.938642 30881 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.852631 30827 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260924 14:31:04.907766 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:04.903954 26403 rpcz_store.cc:276] 0924 14:31:03.768206 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:04.653661 (+885455us) service_pool.cc:227] Handling call
0924 14:31:04.903434 (+249773us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:04.858053 30884 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:04.941005 30860 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:04.946245 30884 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:04.947921 30860 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:04.913379 30856 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.913250 30829 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:04.948766 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1384) took 1192 ms. Trace:
I20260924 14:31:04.949237 26398 rpcz_store.cc:276] 0924 14:31:03.756127 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.756335 (+   208us) service_pool.cc:227] Handling call
0924 14:31:03.787949 (+ 31614us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:03.788344 (+   395us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:03.788372 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:03.839462 (+ 51090us) log.cc:842] Serialized 26 byte log entry
0924 14:31:04.845892 (+1006430us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:04.845932 (+    40us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:04.845999 (+    67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:04.948497 (+102498us) raft_consensus.cc:1671] finished
0924 14:31:04.948520 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:04.948706 (+   186us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":52468,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":43200,"thread_start_us":223875,"threads_started":1,"wal-append.queue_time_us":1104643}
I20260924 14:31:04.950434 30829 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.815551 30831 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:04.940562 30718 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:04.951321 30831 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:04.952118 30835 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:04.952369 30794 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:04.863646 30875 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:04.953715 30831 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.914153 30815 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:04.954445 30815 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:04.954867 30875 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:04.954952 30815 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:04.956669 30803 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.911761 30842 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:04.958180 30800 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:04.905699 30828 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:04.940603 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 299200us
I20260924 14:31:04.959143 30828 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:04.959731 26253 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:04.908387 30793 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:04.911070 30720 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:04.961002 30720 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:04.961439 26256 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260924 14:31:04.961850 26256 pending_rounds.cc:76] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 7
I20260924 14:31:04.962134 26256 pending_rounds.cc:97] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260924 14:31:04.962425 26256 raft_consensus.cc:2866] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:04.963176 30762 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:04.963608 30637 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260924 14:31:04.898870 30798 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:04.964295 30333 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:04.964937 30750 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:04.920907 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1451) took 1545 ms. Trace:
I20260924 14:31:04.936285 30869 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:04.901902 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.877825 26325 rpcz_store.cc:276] 0924 14:31:03.415777 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.418048 (+  2271us) service_pool.cc:227] Handling call
0924 14:31:03.418526 (+   478us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:03.418987 (+   461us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:03.419026 (+    39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:03.419646 (+   620us) log.cc:842] Serialized 26 byte log entry
0924 14:31:03.424405 (+  4759us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:03.424461 (+    56us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:03.424556 (+    95us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:04.764744 (+1340188us) raft_consensus.cc:1671] finished
0924 14:31:04.764765 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:04.764914 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1914,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"thread_start_us":4245,"threads_started":1,"wal-append.queue_time_us":255376}
I20260924 14:31:04.940128 30824 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:04.942351 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:04.940752 30867 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:04.958287 30842 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:04.912365 30843 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:04.961939 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:04.895413 30861 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:04.878602 30739 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:05.075006 26253 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:05.030238 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:05.061882 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1524) took 1274 ms. Trace:
I20260924 14:31:05.075213 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.044047 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.077040 27181 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:05.044520 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:04.965857 26328 rpcz_store.cc:276] 0924 14:31:03.375414 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.375632 (+   218us) service_pool.cc:227] Handling call
0924 14:31:03.394493 (+ 18861us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:03.394924 (+   431us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:03.394957 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:03.395513 (+   556us) log.cc:842] Serialized 26 byte log entry
0924 14:31:03.637266 (+241753us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:03.637331 (+    65us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:03.637433 (+   102us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:04.920646 (+1283213us) raft_consensus.cc:1671] finished
0924 14:31:04.920672 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:04.920828 (+   156us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":3483,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":7961,"thread_start_us":241278,"threads_started":1,"wal-append.queue_time_us":331140}
W20260924 14:31:05.079067 26069 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.295s	user 0.016s	sys 0.001s
I20260924 14:31:05.064016 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 778) took 1320 ms. Trace:
I20260924 14:31:05.076701 26251 rpcz_store.cc:276] 0924 14:31:03.794046 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.794225 (+   179us) service_pool.cc:227] Handling call
0924 14:31:03.794604 (+   379us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:04.840011 (+1045407us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:04.840058 (+    47us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:04.840121 (+    63us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:04.840170 (+    49us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.061535 (+221365us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54007828d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:05.061604 (+    69us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.061800 (+   196us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2400696576}
I20260924 14:31:04.969414 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:04.969019 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.074462 26329 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Stepping down as leader of term 10
W20260924 14:31:04.965804 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 193968us
W20260924 14:31:05.031376 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 244362us
I20260924 14:31:04.970916 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.086284 27188 rpcz_store.cc:276] 0924 14:31:03.743868 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.785633 (+ 41765us) service_pool.cc:227] Handling call
0924 14:31:03.785983 (+   350us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:04.847993 (+1062010us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:04.848025 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:04.848070 (+    45us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:04.848101 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.063708 (+215607us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400260a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:05.063769 (+    61us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.063941 (+   172us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2441897856}
W20260924 14:31:05.032660 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 217768us
I20260924 14:31:05.088212 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.090042 26329 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:31:05.056604 30880 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260924 14:31:05.153987 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.150787 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:05.203668 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 348810us
W20260924 14:31:05.277046 26369 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 1.655s (SENT)
I20260924 14:31:05.221060 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 783) took 1340 ms. Trace:
I20260924 14:31:04.963094 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1482) took 3688 ms. Trace:
I20260924 14:31:04.965579 30804 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:05.169930 30402 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:04.961875 30793 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:05.049793 30798 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:04.967785 30877 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:05.056505 30720 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:05.125854 30815 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:05.044881 30077 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:05.278419 26256 rpcz_store.cc:276] 0924 14:31:01.274213 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:01.287683 (+ 13470us) service_pool.cc:227] Handling call
0924 14:31:01.302614 (+ 14931us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:04.863540 (+3560926us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b5400191f20. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92de730 00007f09b9350300 00007f09b9351371 00007f09b934dedf 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc
0924 14:31:04.962851 (+ 99311us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:04.963046 (+   195us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1173662080}
I20260924 14:31:05.160741 30739 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:05.101686 26253 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:05.307044 30819 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:05.307767 30819 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:05.309321 30847 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.851s	user 0.005s	sys 0.006s
I20260924 14:31:05.311079 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1095) took 1341 ms. Trace:
I20260924 14:31:05.311435 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.312459 26251 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:05.291157 30804 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:05.177911 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1391) took 1415 ms. Trace:
I20260924 14:31:05.313902 26258 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:31:05.313849 26402 rpcz_store.cc:276] 0924 14:31:03.762064 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.762636 (+   572us) service_pool.cc:227] Handling call
0924 14:31:05.176859 (+1414223us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:31:05.177120 (+   261us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:05.177463 (+   343us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:05.177501 (+    38us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:05.177559 (+    58us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:31:05.177602 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.177678 (+    76us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.177841 (+   163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":10958592}
I20260924 14:31:05.172173 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:05.086985 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 144722us
I20260924 14:31:04.965973 30414 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:05.317360 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.317435 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:05.292388 30877 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.318852 26256 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:05.320188 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:05.311861 26253 rpcz_store.cc:276] 0924 14:31:03.969967 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:05.085057 (+1115090us) service_pool.cc:227] Handling call
0924 14:31:05.101215 (+ 16158us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:05.306825 (+205610us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.310945 (+  4120us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:05.046407 30690 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:05.280786 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.277683 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:05.284552 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:05.326457 30048 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:05.327512 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:31:05.328187 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50583.2us
I20260924 14:31:05.329258 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:05.282415 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 130826us
I20260924 14:31:05.068640 30876 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.048172 30863 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:05.278311 27189 rpcz_store.cc:276] 0924 14:31:03.880292 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.908284 (+ 27992us) service_pool.cc:227] Handling call
0924 14:31:03.908812 (+   528us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:03.909105 (+   293us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:03.909543 (+   438us) log.cc:842] Serialized 14 byte log entry
0924 14:31:05.070119 (+1160576us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b1800324460. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd557b 00007f09b935888a
0924 14:31:05.220577 (+150458us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:05.220649 (+    72us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:05.220711 (+    62us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.220810 (+    99us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.220980 (+   170us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":216,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":208,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":981,"spinlock_wait_cycles":100918528,"thread_start_us":149974,"threads_started":1,"wal-append.queue_time_us":1298861}
I20260924 14:31:05.260453 30861 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:05.318974 30877 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:05.211252 30681 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:05.344990 30850 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:05.345561 30850 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:05.346834 27123 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:05.347939 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:05.348312 27191 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 776) took 1608 ms (client timeout 1858 ms). Trace:
I20260924 14:31:05.258985 26329 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:05.046380 30884 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:05.087090 30732 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:04.965427 30566 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:05.291953 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:05.358822 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 66942.7us
I20260924 14:31:05.336346 30739 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:05.293601 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.087566 30683 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:05.293560 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.186776 30851 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:05.188400 30867 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:05.295908 30903 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.294647 30793 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260924 14:31:05.123721 30607 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:05.123682 30765 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:05.373164 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.373584 30607 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:05.373699 30765 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:05.374436 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.375638 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.375767 30607 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:05.318097 27182 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:05.324218 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1519) took 2579 ms. Trace:
I20260924 14:31:05.376693 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.331140 26253 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: [7.5-10.7]
I20260924 14:31:05.377552 30819 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:05.377774 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.378471 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:05.378849 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:05.332307 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 160214us
I20260924 14:31:05.380535 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:05.332314 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1094) took 1362 ms. Trace:
I20260924 14:31:05.333371 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.332824 30870 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:05.326489 30374 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:31:05.326993 30048 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:05.335671 26252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1093) took 1366 ms (client timeout 1608 ms). Trace:
W20260924 14:31:05.398263 26252 rpcz_store.cc:269] 0924 14:31:03.969364 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:04.614318 (+644954us) service_pool.cc:227] Handling call
0924 14:31:05.335499 (+721181us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b540002bf54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:05.335609 (+   110us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":140048128}
I20260924 14:31:05.357827 30807 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:05.334127 30572 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:05.316859 30815 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:05.349898 26329 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:05.350458 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:05.348742 27191 rpcz_store.cc:269] 0924 14:31:03.739485 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.739639 (+   154us) service_pool.cc:227] Handling call
0924 14:31:05.348137 (+1608498us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54006c5054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:05.348250 (+   113us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2543416064}
I20260924 14:31:05.336112 30901 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:05.304682 30077 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:05.344718 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:05.417488 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 72814.1us
I20260924 14:31:05.418897 26253 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:31:05.293334 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.360314 30884 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:05.363936 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.364049 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:05.421578 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 57619.4us
I20260924 14:31:05.422370 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:05.422904 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.368033 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:05.362011 30739 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.367323 30628 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:05.372426 30903 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:05.448249 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.381598 26257 rpcz_store.cc:276] 0924 14:31:03.969697 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:04.662429 (+692732us) service_pool.cc:227] Handling call
0924 14:31:05.332147 (+669718us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540070fb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:05.332254 (+   107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":373994880}
I20260924 14:31:05.382302 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:05.351693 30787 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:05.371223 30683 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:05.360862 30688 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260924 14:31:05.376981 26250 rpcz_store.cc:276] 0924 14:31:02.744539 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:02.745701 (+  1162us) service_pool.cc:227] Handling call
0924 14:31:02.746795 (+  1094us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:02.897776 (+150981us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:02.897826 (+    50us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:02.952303 (+ 54477us) log.cc:842] Serialized 26 byte log entry
0924 14:31:03.068315 (+116012us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:03.068381 (+    66us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:03.068441 (+    60us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:05.308695 (+2240254us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:05.317970 (+  9275us) raft_consensus.cc:1671] finished
0924 14:31:05.317991 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.324140 (+  6149us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6215,"raft.queue_time_us":198019,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"spinlock_wait_cycles":5291776,"thread_start_us":161856,"threads_started":2,"wal-append.queue_time_us":428753}
I20260924 14:31:05.375867 30765 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:05.358467 30732 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.388273 27186 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:05.388450 30647 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:31:05.389343 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.463524 29682 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:05.370674 30850 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:05.383184 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.390164 30741 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:05.392273 30750 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.474272 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.393396 30828 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:05.396605 30870 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:05.336619 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:05.363072 30566 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:05.409381 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:05.411588 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 61213.6us
I20260924 14:31:05.481050 27189 raft_consensus.cc:2379] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:05.413672 30826 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:05.413777 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:05.482280 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 145743us
I20260924 14:31:05.419335 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:05.420244 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.486213 26330 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:05.403805 30048 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:05.469493 30732 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:05.440267 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 889) took 2930 ms. Trace:
I20260924 14:31:05.437436 26369 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:05.438362 27194 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:05.438006 26324 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260924 14:31:05.339586 30897 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
I20260924 14:31:05.438813 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1610) took 1680 ms. Trace:
I20260924 14:31:05.443199 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.444509 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.320290 30842 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:05.451889 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.416152 30807 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:05.457043 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.459169 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.343787 30876 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:05.417276 30901 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:05.421078 30385 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:05.407162 26069 maintenance_manager.cc:643] P 277acd3330ed4b40b2cea74b4e602a13: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 9.462s	user 0.218s	sys 0.063s Metrics: {"bytes_written":30626,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":202693,"dirs.run_cpu_time_us":696,"dirs.run_wall_time_us":44840,"drs_written":1,"lbm_read_time_us":441,"lbm_reads_lt_1ms":4,"lbm_write_time_us":436936,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":12,"lbm_writes_lt_1ms":20,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":147130,"threads_started":2,"update_count":20,"wal-append.queue_time_us":261113}
I20260924 14:31:05.438452 30697 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:05.464283 29682 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:05.420958 30884 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:05.463598 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 95621.7us
I20260924 14:31:05.470113 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.443171 30793 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:05.472940 26400 raft_consensus.cc:2396] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260924 14:31:05.447641 30739 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:05.475668 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.481717 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.397750 30374 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.473096 30741 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:05.454466 30628 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:05.488014 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:05.634716 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:05.537154 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:05.550700 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 113324us
I20260924 14:31:05.637401 27189 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:05.507827 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.638090 26331 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:05.523299 30870 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:05.551139 26327 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:05.472657 30891 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:05.617209 26326 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:05.618043 26330 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:05.616302 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.528927 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:05.555169 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.488713 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.618685 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.618732 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.551594 26401 rpcz_store.cc:276] 0924 14:31:03.762274 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.762719 (+   445us) service_pool.cc:227] Handling call
0924 14:31:03.763005 (+   286us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:03.792561 (+ 29556us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:03.792608 (+    47us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:03.792683 (+    75us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:03.792738 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.438538 (+1645800us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5400476154. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:05.438598 (+    60us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.438740 (+   142us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":54978432}
W20260924 14:31:05.524194 30837 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 2.064s	user 0.022s	sys 0.013s
I20260924 14:31:05.552469 26332 rpcz_store.cc:276] 0924 14:31:02.509736 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:02.652452 (+142716us) service_pool.cc:227] Handling call
0924 14:31:02.652885 (+   433us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:02.807557 (+154672us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:03.066666 (+259109us) log.cc:842] Serialized 14 byte log entry
0924 14:31:03.189458 (+122792us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:03.200454 (+ 10996us) log.cc:842] Serialized 26 byte log entry
0924 14:31:03.200598 (+   144us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:03.200625 (+    27us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.046872 (+1846247us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b1800168340. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:31:05.424070 (+377198us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54002774d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:05.440025 (+ 15955us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:05.440077 (+    52us) raft_consensus.cc:1671] finished
0924 14:31:05.440088 (+    11us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.440213 (+   125us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":405271,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":30302,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":28930,"spinlock_wait_cycles":336501888,"thread_start_us":344532,"threads_started":2,"wal-append.queue_time_us":408387,"wal-append.run_cpu_time_us":2444,"wal-append.run_wall_time_us":1007315}
W20260924 14:31:05.625634 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 74554.5us
I20260924 14:31:05.480865 30750 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:05.554558 30813 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:31:05.754447 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.756031 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.487058 30828 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:05.488161 30826 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:05.767496 26400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1609) took 2009 ms (client timeout 1915 ms). Trace:
W20260924 14:31:05.767810 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 377030us
W20260924 14:31:05.767957 26400 rpcz_store.cc:269] 0924 14:31:03.757687 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.758121 (+   434us) service_pool.cc:227] Handling call
0924 14:31:05.311255 (+1553134us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:31:05.764149 (+452894us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54006e78d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:05.767395 (+  3246us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":366329472}
I20260924 14:31:05.525681 30566 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:05.631007 26255 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.010s	sys 0.022s
I20260924 14:31:05.632088 26258 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
W20260924 14:31:05.772991 26255 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.010s	sys 0.023s
I20260924 14:31:05.774220 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.633069 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.523530 30795 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:05.476956 30077 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:05.639986 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.789111 26331 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:05.726003 30891 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:05.651175 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.798013 30077 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:05.799712 30048 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:05.650075 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1392) took 1864 ms. Trace:
W20260924 14:31:05.645363 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 158985us
I20260924 14:31:05.674156 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:05.647279 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 262918us
I20260924 14:31:05.802703 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:05.803499 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.804406 26400 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:31:05.806331 26224 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:05.723810 26328 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:05.647852 27185 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260924 14:31:05.720769 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:05.646456 27186 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 769) took 4192 ms (client timeout 1655 ms). Trace:
I20260924 14:31:05.808676 26327 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:05.626524 30884 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:05.519887 30683 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:05.552059 30901 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:05.811672 30870 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:31:05.812121 30870 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.812491 30870 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:05.727470 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.815539 30697 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:05.815816 26332 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:31:05.816084 30697 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.639685 30690 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:05.816510 30697 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:05.816658 30690 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:05.817124 30690 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:05.632516 30374 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:31:05.734040 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.824287 26399 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:31:05.728969 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.635205 30741 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:05.736308 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.830359 26406 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:31:05.830803 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.734948 30807 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:05.730369 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.740494 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.741720 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.732906 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1388) took 1971 ms. Trace:
I20260924 14:31:05.836149 26397 rpcz_store.cc:276] 0924 14:31:03.761209 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.761339 (+   130us) service_pool.cc:227] Handling call
0924 14:31:05.732166 (+1970827us) spinlock_profiling.cc:243] Waited 7.5 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:31:05.732397 (+   231us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:05.732595 (+   198us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:05.732621 (+    26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:05.732665 (+    44us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:05.732698 (+    33us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.732752 (+    54us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.732850 (+    98us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":17250048}
I20260924 14:31:05.486948 30274 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.182s
I20260924 14:31:05.729292 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.749366 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1497) took 1062 ms. Trace:
I20260924 14:31:05.796904 30891 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:05.648257 26070 maintenance_manager.cc:419] P 277acd3330ed4b40b2cea74b4e602a13: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7698 bytes on disk
I20260924 14:31:05.838367 26251 rpcz_store.cc:276] 0924 14:31:04.687272 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:05.314981 (+627709us) service_pool.cc:227] Handling call
0924 14:31:05.315322 (+   341us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:05.507766 (+192444us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:05.507797 (+    31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:05.507839 (+    42us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:05.507870 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.749089 (+241219us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54007823d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:05.749137 (+    48us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.749298 (+   161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":442075904}
I20260924 14:31:05.839478 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.750768 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.753101 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.840754 27191 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:05.841471 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:05.841671 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:05.756649 27188 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:05.766542 30826 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:05.771425 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44114.6us
I20260924 14:31:05.569227 30385 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.775851 26324 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:05.847863 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.849470 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.649955 30843 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:05.636831 30414 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:05.655776 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.641109 30628 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:05.786993 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.890838 27181 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:05.722957 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.802310 26405 rpcz_store.cc:276] 0924 14:31:03.785756 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.826853 (+ 41097us) service_pool.cc:227] Handling call
0924 14:31:05.283781 (+1456928us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:31:05.283974 (+   193us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:05.291235 (+  7261us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:05.291270 (+    35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:05.294598 (+  3328us) log.cc:842] Serialized 26 byte log entry
0924 14:31:05.411129 (+116531us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:31:05.411173 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.411246 (+    73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:05.649605 (+238359us) raft_consensus.cc:1671] finished
0924 14:31:05.649637 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.649992 (+   355us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6865,"raft.queue_time_us":7856,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":1579,"spinlock_wait_cycles":7987072,"thread_start_us":104294,"threads_started":1,"wal-append.queue_time_us":180840}
W20260924 14:31:05.639492 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 151557us
W20260924 14:31:05.808620 27186 rpcz_store.cc:269] 0924 14:31:01.454326 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:01.486072 (+ 31746us) service_pool.cc:227] Handling call
0924 14:31:05.646257 (+4160185us) spinlock_profiling.cc:243] Waited 3.79 s on lock 0x7b5400130e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:05.646382 (+   125us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8727010688}
I20260924 14:31:05.810961 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.892618 27189 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:05.650714 30793 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.135s
I20260924 14:31:05.782366 30813 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:05.785112 30828 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.838385 30850 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:05.913645 30828 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:05.913937 30850 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:05.916371 26221 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:05.916796 30850 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:05.908154 30890 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260924 14:31:05.821854 26326 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260924 14:31:05.919510 26326 pending_rounds.cc:76] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 7
I20260924 14:31:05.919771 26326 pending_rounds.cc:97] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260924 14:31:05.920048 26326 raft_consensus.cc:2866] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:05.919935 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.920806 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1456) took 2221 ms. Trace:
I20260924 14:31:05.921144 26326 rpcz_store.cc:276] 0924 14:31:03.699466 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.699699 (+   233us) service_pool.cc:227] Handling call
0924 14:31:03.731638 (+ 31939us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:05.616866 (+1885228us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b5800a148b0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92c9f63 00007f09b932daba 00007f09b93453af 00007f09b9346b4e 00007f09b9350c89 00007f09b9351556 00007f09b934dedf 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684
0924 14:31:05.920599 (+303733us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.920763 (+   164us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1646556416}
I20260924 14:31:05.821748 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.922351 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.822683 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.827808 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:05.828819 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.831794 26400 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:05.832684 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.794256 30804 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:05.925155 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.922991 26326 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:05.834671 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1516) took 3155 ms. Trace:
I20260924 14:31:05.729717 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.735319 30876 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:05.842789 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.842872 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:05.935160 27188 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
W20260924 14:31:05.935281 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 92492.1us
I20260924 14:31:05.937172 27188 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260924 14:31:05.938941 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:05.921191 30628 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:05.940145 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.941356 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.941967 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.838802 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.850770 27194 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:05.940169 30628 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:05.944710 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.851130 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.947106 30628 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:05.863345 30048 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:05.970237 30851 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.000s
I20260924 14:31:05.880084 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.882135 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.973542 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.974433 26324 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:05.867534 30798 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:05.887995 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:05.888407 29682 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:05.897506 27181 raft_consensus.cc:595] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:05.976538 27181 raft_consensus.cc:603] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:05.898154 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.892563 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.978255 26405 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:31:05.899731 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.903952 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:05.908416 26298 connection.cc:232] RPC call timeout handler was delayed by -1.27517224300027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
I20260924 14:31:05.980378 26397 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:05.836550 30897 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:05.834038 30888 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:05.836289 30883 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:05.843437 30827 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:05.926892 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.923228 26331 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:05.921902 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1096) took 1951 ms. Trace:
I20260924 14:31:05.923497 30540 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:05.926290 26254 rpcz_store.cc:276] 0924 14:31:02.679415 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:02.699983 (+ 20568us) service_pool.cc:227] Handling call
0924 14:31:02.700329 (+   346us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:02.882727 (+182398us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:02.882773 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:02.883502 (+   729us) log.cc:842] Serialized 26 byte log entry
0924 14:31:03.187674 (+304172us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:31:03.187759 (+    85us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:03.187864 (+   105us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:05.443472 (+2255608us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:05.819003 (+375531us) raft_consensus.cc:1671] finished
0924 14:31:05.819023 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.834600 (+ 15577us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":207221,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":15612,"spinlock_wait_cycles":26410368,"thread_start_us":359020,"threads_started":2,"wal-append.queue_time_us":502117}
I20260924 14:31:05.928723 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.929224 26069 maintenance_manager.cc:643] P 277acd3330ed4b40b2cea74b4e602a13: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.055s	user 0.002s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":607,"lbm_reads_lt_1ms":4}
I20260924 14:31:05.947773 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.863183 30385 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:05.949150 30647 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:05.949947 30414 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:05.943773 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.950568 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:06.115831 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 165400us
I20260924 14:31:05.952065 30813 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:05.953006 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 125314us
I20260924 14:31:06.129568 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:05.953709 30843 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:05.954413 30637 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:05.955812 30374 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:05.958360 30842 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:05.959417 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:05.959995 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:05.962033 30739 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:05.960453 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:05.964502 30750 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:05.897284 30903 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:05.872684 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.976292 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.958408 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:06.168327 26252 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
W20260924 14:31:05.980536 26298 connection.cc:232] RPC call timeout handler was delayed by -1.2067722430001595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
I20260924 14:31:05.888832 30383 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:05.897603 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 775) took 2162 ms. Trace:
I20260924 14:31:05.971174 26399 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:31:05.985109 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.990928 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:06.173643 26330 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:06.112505 30647 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:05.997406 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:05.998445 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.999181 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.889248 30720 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:06.003724 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:05.981719 30798 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:06.021381 26255 rpcz_store.cc:276] 0924 14:31:03.970282 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:05.086357 (+1116075us) service_pool.cc:227] Handling call
0924 14:31:05.086697 (+   340us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:05.099329 (+ 12632us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:05.099358 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:05.099948 (+   590us) log.cc:842] Serialized 26 byte log entry
0924 14:31:05.773402 (+673454us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:05.773669 (+   267us) log.cc:842] Serialized 14 byte log entry
0924 14:31:05.776426 (+  2757us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.784620 (+  8194us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:05.921688 (+137068us) raft_consensus.cc:1671] finished
0924 14:31:05.921706 (+    18us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:05.921845 (+   139us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":10988,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":169,"raft.queue_time_us":3699,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":47407,"spinlock_wait_cycles":1301376,"thread_start_us":673053,"threads_started":1,"wal-append.queue_time_us":813049}
W20260924 14:31:06.029981 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 107716us
I20260924 14:31:06.039539 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:06.016155 30888 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.190137 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:06.099802 30826 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:06.104254 30901 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.109467 30884 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:06.010892 30897 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.123955 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:06.115078 26328 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:31:06.135345 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:06.136011 26224 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:06.032701 30827 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:06.148645 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 48419.7us
I20260924 14:31:06.155426 26250 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:06.157418 27181 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "986260198b71418c831bfc4617da4603"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:06.221973 26400 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
W20260924 14:31:06.115361 30877 consensus_meta.cc:287] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.758s	user 0.012s	sys 0.002s
I20260924 14:31:06.164640 26256 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:31:06.170075 26298 connection.cc:232] RPC call timeout handler was delayed by -1.0645722429999296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
W20260924 14:31:06.223049 26298 connection.cc:232] RPC call timeout handler was delayed by -1.0438722429998961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
I20260924 14:31:06.223801 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:06.224972 26298 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:06.174901 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:05.997000 27186 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260924 14:31:06.172118 27185 rpcz_store.cc:276] 0924 14:31:03.735567 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:03.735740 (+   173us) service_pool.cc:227] Handling call
0924 14:31:05.647482 (+1911742us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:05.808028 (+160546us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:05.897402 (+ 89374us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b5400130e50. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:05.897550 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4395472128}
I20260924 14:31:06.141633 30048 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:06.231369 26403 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:06.112013 30540 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:06.127826 30414 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:06.192625 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:06.193466 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:06.235412 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44849.6us
I20260924 14:31:06.148321 30766 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:06.196487 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:06.199349 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:06.201220 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:06.153972 30637 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:06.207381 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 46756.2us
I20260924 14:31:06.210083 26325 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:06.217514 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 81539.2us
I20260924 14:31:06.025616 30883 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:06.222618 27181 raft_consensus.cc:595] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:06.246151 27181 raft_consensus.cc:603] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:06.155891 30876 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260924 14:31:06.192818 30647 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:06.149511 30843 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:06.276404 30843 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:06.277798 30540 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:06.278486 30540 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:31:06.283629 26328 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
W20260924 14:31:06.236837 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44300.9us
I20260924 14:31:06.195057 30798 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:06.240062 26330 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:06.240603 26332 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:06.145347 30870 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.244652 26401 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
W20260924 14:31:06.244683 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 109415us
I20260924 14:31:06.249066 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:06.245891 30637 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:06.259404 26324 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:06.260383 26403 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:46098
I20260924 14:31:06.261713 30897 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.241207 30750 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:31:06.265415 30826 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:06.258483 30576 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:06.266585 30628 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:06.354126 30628 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:06.354609 30628 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:06.358556 30732 consensus_meta.cc:287] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.742s	user 0.008s	sys 0.002s
I20260924 14:31:06.267409 30827 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.230723 30842 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:06.359980 30842 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:06.360440 30842 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:06.268383 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 759) took 1216 ms. Trace:
I20260924 14:31:06.363651 27186 rpcz_store.cc:276] 0924 14:31:05.052130 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:05.908472 (+856342us) service_pool.cc:227] Handling call
0924 14:31:05.908855 (+   383us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:06.228118 (+319263us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:06.267951 (+ 39833us) spinlock_profiling.cc:243] Waited 70.3 ms on lock 0x7b54002008d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:06.268307 (+   356us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":161609984}
I20260924 14:31:06.256091 30828 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:06.366063 30901 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.366226 30828 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:06.239846 30048 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:06.269127 30888 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.270570 26398 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:06.359076 30732 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:06.369036 30697 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.552s	user 0.006s	sys 0.005s
I20260924 14:31:06.369376 26404 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:06.369532 30697 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:06.194885 30720 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:06.371744 26402 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:06.372352 30720 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:06.191421 30383 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:06.374732 30383 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:06.377230 30690 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.560s	user 0.005s	sys 0.006s
I20260924 14:31:06.377720 30690 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:06.380300 30739 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:06.381031 27191 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:06.381381 30891 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.383157 27192 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:06.393178 26295 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.044s	sys 0.051s
I20260924 14:31:06.289984 26327 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 5.
I20260924 14:31:06.241833 30414 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:06.345800 30750 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:06.334188 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 85305.1us
I20260924 14:31:06.335865 30900 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:06.336361 30903 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:06.337078 30766 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:06.243323 30861 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260924 14:31:06.250392 26329 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.020s	sys 0.027s
I20260924 14:31:06.339797 26403 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:31:06.417469 26403 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:06.418329 26329 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.020s	sys 0.028s
I20260924 14:31:06.338408 27190 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
W20260924 14:31:06.343473 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 54309.6us
I20260924 14:31:06.266004 30374 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:06.348289 26249 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:31:06.350066 26401 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:06.367172 30048 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:06.370702 26400 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:06.332113 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:06.438366 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106333us
I20260924 14:31:06.439775 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:06.441475 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:06.375103 30732 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:06.239545 30877 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:06.393149 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e81d1761568f40418d9259b74501d677"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:06.459969 27189 raft_consensus.cc:595] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
W20260924 14:31:06.412511 26258 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.004s	sys 0.024s
W20260924 14:31:06.412806 26295 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.044s	sys 0.051s
I20260924 14:31:06.414970 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:06.363169 30385 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.371763 30884 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:06.419431 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 395704us
I20260924 14:31:06.448292 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:06.451812 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:06.415840 30750 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:06.451877 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:06.454793 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:06.461786 26258 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.005s	sys 0.024s
W20260924 14:31:06.462469 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47552.6us
I20260924 14:31:06.386142 30576 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:06.476141 30576 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:06.476290 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:06.478554 30861 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:06.482013 26403 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:06.484710 26398 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: []
W20260924 14:31:06.463246 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 659635us
I20260924 14:31:06.474264 26253 raft_consensus.cc:2396] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260924 14:31:06.419283 30688 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:06.415925 30900 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:06.430802 30445 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.017s	sys 0.010s
I20260924 14:31:06.500512 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260924 14:31:06.501032 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:06.493788 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:06.494232 26403 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:06.467013 30766 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:06.507766 26323 raft_consensus.cc:2379] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:06.509127 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:06.462448 30374 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:31:06.512153 30697 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:06.516991 30877 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:06.504633 30445 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.017s	sys 0.011s
I20260924 14:31:06.493145 30888 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:06.531879 30888 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:06.532374 30888 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:06.533640 30385 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.520468 30826 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:06.522562 30374 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260924 14:31:06.505424 30688 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:06.474602 30752 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.010s	sys 0.021s
I20260924 14:31:06.551316 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:06.568653 27190 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 749) took 1599 ms (client timeout 1786 ms). Trace:
I20260924 14:31:06.553210 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:06.569232 27190 rpcz_store.cc:269] 0924 14:31:04.968951 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:05.029648 (+ 60697us) service_pool.cc:227] Handling call
0924 14:31:06.568423 (+1538775us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b540000d554. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:06.568577 (+   154us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2058012928}
I20260924 14:31:06.554829 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:06.531913 30870 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.561645 30048 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.562690 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 969) took 1139 ms. Trace:
I20260924 14:31:06.525489 30690 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:06.563320 26325 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:06.573808 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:06.582209 26323 rpcz_store.cc:276] 0924 14:31:05.422848 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:05.447417 (+ 24569us) service_pool.cc:227] Handling call
0924 14:31:06.562447 (+1115030us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54005d8c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:06.562618 (+   171us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2037916928}
I20260924 14:31:06.583201 27181 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:06.561067 30628 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:06.576733 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:31:06.580066 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:06.580847 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:06.585175 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:06.585531 27192 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:06.581415 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:06.581562 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:06.518986 30647 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:06.545596 30826 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:06.571208 30752 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.011s	sys 0.021s
W20260924 14:31:06.574824 30741 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.724s	user 0.013s	sys 0.000s
I20260924 14:31:06.608095 26397 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:06.568094 30883 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:06.602216 30765 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.000s	sys 0.024s
I20260924 14:31:06.545107 30876 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260924 14:31:06.540860 30576 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:06.617656 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 430820us
I20260924 14:31:06.607417 30826 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:06.610392 26081 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:06.588232 30637 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:06.589313 30842 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:06.637629 26077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55200 (request call id 129) took 17860 ms (client timeout 15000 ms). Trace:
W20260924 14:31:06.617015 30765 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.000s	sys 0.025s
I20260924 14:31:06.613969 30891 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:06.635385 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:06.622587 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:06.524411 30900 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:06.607122 30383 consensus_queue.cc:1046] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260924 14:31:06.647125 30884 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:31:06.636399 26081 catalog_manager.cc:5495] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 4 to 7, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 7 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:06.610891 30741 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:06.651118 26250 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
W20260924 14:31:06.634824 26326 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.025s
W20260924 14:31:06.652776 26326 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.009s	sys 0.025s
W20260924 14:31:06.638310 26077 rpcz_store.cc:269] 0924 14:30:48.773536 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:48.774315 (+   779us) service_pool.cc:227] Handling call
0924 14:31:06.633736 (+17859421us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:30:59.242395 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:30:59.381193 (+138798us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:30:59.381280 (+    87us) write_op.cc:435] Acquired schema lock
0924 14:30:59.381311 (+    31us) tablet.cc:660] Decoding operations
0924 14:30:59.447710 (+ 66399us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:30:59.447840 (+   130us) write_op.cc:641] Partition lock acquired for write op
0924 14:30:59.447868 (+    28us) tablet.cc:683] Acquiring locks for 20 operations
0924 14:30:59.449403 (+  1535us) tablet.cc:699] Row locks acquired
0924 14:30:59.449440 (+    37us) write_op.cc:260] PREPARE: finished
0924 14:30:59.449612 (+   172us) write_op.cc:270] Start()
0924 14:30:59.449749 (+   137us) write_op.cc:276] Timestamp: P: 1790260259449541 usec, L: 0
0924 14:30:59.449780 (+    31us) op_driver.cc:331] REPLICATION: starting
0924 14:30:59.450544 (+   764us) log.cc:842] Serialized 7973 byte log entry
0924 14:31:00.163545 (+713001us) op_driver.cc:444] REPLICATION: finished
0924 14:31:00.981117 (+817572us) write_op.cc:301] APPLY: starting
0924 14:31:01.002923 (+ 21806us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:01.262986 (+260063us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:01.263279 (+   293us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:01.626339 (+363060us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:01.626808 (+   469us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=40
0924 14:31:01.626878 (+    70us) write_op.cc:312] APPLY: finished
0924 14:31:01.627409 (+   531us) log.cc:842] Serialized 138 byte log entry
0924 14:31:01.803047 (+175638us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:01.853371 (+ 50324us) write_op.cc:454] Released schema lock
0924 14:31:01.900214 (+ 46843us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:01.949066 (+ 48852us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:01.957900 (+  8834us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":288210,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":692020,"apply.run_cpu_time_us":29392,"apply.run_wall_time_us":1092246,"num_ops":20,"prepare.queue_time_us":311462,"prepare.run_cpu_time_us":19764,"prepare.run_wall_time_us":527184,"raft.queue_time_us":283459,"raft.run_cpu_time_us":19860,"raft.run_wall_time_us":393033,"replication_time_us":705738,"thread_start_us":1010850,"threads_started":5,"wal-append.queue_time_us":671251,"wal-append.run_cpu_time_us":40973,"wal-append.run_wall_time_us":2148513}]]}
I20260924 14:31:06.690212 26081 catalog_manager.cc:5495] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 9 to 11, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 11 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:06.665400 30883 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:06.699561 27182 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:06.665594 30884 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:06.701830 27188 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.000s	sys 0.028s
I20260924 14:31:06.703336 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:06.637331 30697 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260924 14:31:06.673717 30683 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:31:06.707975 30683 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:06.708848 26406 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
W20260924 14:31:06.709992 27188 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.000s	sys 0.028s
I20260924 14:31:06.718551 26253 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
W20260924 14:31:06.723349 26331 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.000s	sys 0.021s
W20260924 14:31:06.724611 26331 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.001s	sys 0.021s
I20260924 14:31:06.702546 30883 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:06.709800 30884 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:06.776348 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:06.777277 27186 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:06.683599 30414 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:06.793154 26399 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.016s	sys 0.019s
W20260924 14:31:06.793799 26399 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.016s	sys 0.019s
I20260924 14:31:06.708493 30843 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:06.766242 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.000s	sys 0.023s
I20260924 14:31:06.768838 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1253) took 1471 ms. Trace:
I20260924 14:31:06.776216 30883 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:06.803419 27186 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:06.806774 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:06.804231 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 546722us
I20260924 14:31:06.804345 27188 rpcz_store.cc:276] 0924 14:31:05.297332 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:05.947886 (+650554us) service_pool.cc:227] Handling call
0924 14:31:05.989395 (+ 41509us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:06.113956 (+124561us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:06.113990 (+    34us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:06.114568 (+   578us) log.cc:842] Serialized 52 byte log entry
0924 14:31:06.711501 (+596933us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:06.711606 (+   105us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:06.711670 (+    64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:06.768536 (+ 56866us) raft_consensus.cc:1671] finished
0924 14:31:06.768568 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:06.768762 (+   194us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":197511,"raft.queue_time_us":125663,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":18367,"thread_start_us":530373,"threads_started":1,"wal-append.queue_time_us":605462}
I20260924 14:31:06.813153 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:06.814067 30807 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:06.814725 30807 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:06.815867 30807 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.819164 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:06.780758 30647 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:06.778317 30628 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:06.778821 30048 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.832219 26295 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:06.833158 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:06.765787 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.011s	sys 0.012s
W20260924 14:31:06.835340 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.012s	sys 0.012s
I20260924 14:31:06.839170 29682 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:06.839638 29682 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:06.839962 29682 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:06.845305 30688 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:06.823005 30576 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:06.850276 30843 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.850076 30888 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.828207 30876 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:06.846864 29682 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.853550 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1460) took 1884 ms. Trace:
W20260924 14:31:06.858271 30918 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260924 14:31:06.858773 30798 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260924 14:31:06.859308 30798 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.860526 26405 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 18. Current term is 19. Ops: []
I20260924 14:31:06.862331 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:06.863128 26326 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:06.863008 29682 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:06.837149 30828 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:06.860049 30793 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Stepping down as leader of term 12
W20260924 14:31:06.864890 30755 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260924 14:31:06.806106 30683 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.838742 30637 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.880934 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:06.882339 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:06.854377 30876 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:06.898144 30741 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:06.908779 30828 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:06.908519 30826 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:06.909291 30828 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:06.911104 30826 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:06.912406 30876 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:06.912794 30888 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.884228 30842 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:06.876657 30793 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:06.890918 30929 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:06.883975 30752 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:06.931349 30883 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.931350 30884 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:06.933981 30883 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.935228 30884 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.937098 30876 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:06.939250 30876 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:06.938968 30828 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:06.975488 26256 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.006s	sys 0.011s
W20260924 14:31:06.976771 26256 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.006s	sys 0.012s
W20260924 14:31:06.982483 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 150204us
W20260924 14:31:06.983860 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.007s	sys 0.011s
W20260924 14:31:06.984632 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.007s	sys 0.011s
W20260924 14:31:06.984716 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 477679us
W20260924 14:31:06.985388 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 509124us
W20260924 14:31:06.990175 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 765060us
I20260924 14:31:06.986827 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:06.995074 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 905) took 1128 ms. Trace:
I20260924 14:31:06.995561 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:06.995803 26328 rpcz_store.cc:276] 0924 14:31:05.866297 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:06.384158 (+517861us) service_pool.cc:227] Handling call
0924 14:31:06.392218 (+  8060us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:06.392444 (+   226us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:06.771610 (+379166us) log.cc:842] Serialized 14 byte log entry
0924 14:31:06.993941 (+222331us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:06.994101 (+   160us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:06.994277 (+   176us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:06.994485 (+   208us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:06.994735 (+   250us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":403763,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":579971,"threads_started":2,"wal-append.queue_time_us":153212}
I20260924 14:31:06.994350 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:07.000603 26323 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:07.001276 26323 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:07.002760 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1470) took 1179 ms. Trace:
I20260924 14:31:07.003297 26331 rpcz_store.cc:276] 0924 14:31:05.833715 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:06.030203 (+196488us) service_pool.cc:227] Handling call
0924 14:31:06.091800 (+ 61597us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:06.092635 (+   835us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:06.092667 (+    32us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:06.116044 (+ 23377us) log.cc:842] Serialized 52 byte log entry
0924 14:31:06.726039 (+609995us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:07.001368 (+275329us) log.cc:842] Serialized 14 byte log entry
0924 14:31:07.001920 (+   552us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:07.002090 (+   170us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:07.002171 (+    81us) raft_consensus.cc:1671] finished
0924 14:31:07.002189 (+    18us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:07.002558 (+   369us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12003,"prepare.queue_time_us":121833,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":98,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":21155,"thread_start_us":777380,"threads_started":2,"wal-append.queue_time_us":618425}
I20260924 14:31:07.000622 26328 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:07.008893 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 128889us
I20260924 14:31:07.016037 26369 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:07.015065 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 909) took 1147 ms. Trace:
I20260924 14:31:07.017683 26332 rpcz_store.cc:276] 0924 14:31:05.867431 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:06.404298 (+536867us) service_pool.cc:227] Handling call
0924 14:31:06.410107 (+  5809us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:06.410336 (+   229us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:06.812714 (+402378us) log.cc:842] Serialized 14 byte log entry
0924 14:31:07.014014 (+201300us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:07.014142 (+   128us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:07.014239 (+    97us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:07.014379 (+   140us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:07.014772 (+   393us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4666,"prepare.queue_time_us":414890,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"thread_start_us":542637,"threads_started":2,"wal-append.queue_time_us":131808}
I20260924 14:31:06.991910 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.024874 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.003227 26323 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.036461 30793 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.038918 26323 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:07.040599 30540 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:06.920781 30683 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:07.041738 30828 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:06.919188 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1251) took 1645 ms. Trace:
W20260924 14:31:07.046434 30750 consensus_meta.cc:287] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.000s
I20260924 14:31:07.044389 27194 rpcz_store.cc:276] 0924 14:31:05.274016 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:05.946309 (+672293us) service_pool.cc:227] Handling call
0924 14:31:06.269068 (+322759us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:06.346287 (+ 77219us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:06.346334 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:06.346907 (+   573us) log.cc:842] Serialized 26 byte log entry
0924 14:31:06.829839 (+482932us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:06.829900 (+    61us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:06.829996 (+    96us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:06.854237 (+ 24241us) raft_consensus.cc:1671] finished
0924 14:31:06.854256 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:06.915875 (+ 61619us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54002008d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:06.919129 (+  3254us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":96936,"raft.queue_time_us":69132,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5767,"spinlock_wait_cycles":741401856,"thread_start_us":417726,"threads_started":1,"wal-append.queue_time_us":464567}
I20260924 14:31:07.043462 30374 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:07.049847 30374 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:07.050377 30374 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:07.056679 30842 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:07.057714 30750 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.056905 30946 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:07.064002 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 771) took 1750 ms. Trace:
I20260924 14:31:07.064594 30946 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:07.065335 30946 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:07.071871 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.079171 30826 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:07.080682 30826 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.082515 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.060536 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1620) took 1046 ms. Trace:
I20260924 14:31:07.062325 30750 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:07.063226 26327 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
W20260924 14:31:07.093186 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 101621us
I20260924 14:31:07.064630 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.088265 26406 rpcz_store.cc:276] 0924 14:31:06.014588 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:06.243972 (+229384us) service_pool.cc:227] Handling call
0924 14:31:06.244264 (+   292us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:06.711336 (+467072us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:07.058660 (+347324us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54002fc6d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:07.060323 (+  1663us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1059548416}
I20260924 14:31:07.064711 27182 rpcz_store.cc:276] 0924 14:31:05.313019 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:05.948515 (+635496us) service_pool.cc:227] Handling call
0924 14:31:05.948829 (+   314us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:06.702768 (+753939us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:07.063281 (+360513us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b540000c654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:07.063743 (+   462us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1714225920}
W20260924 14:31:07.100862 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 215947us
I20260924 14:31:07.105094 30876 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:07.106132 30876 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.106613 30445 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:07.108744 30883 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:07.109019 30850 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:07.109476 30850 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:07.119589 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 103907us
I20260924 14:31:07.109555 30883 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:07.126531 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.130504 30798 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:07.130613 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 104603us
I20260924 14:31:07.137225 30826 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:07.139484 26224 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:07.142346 30766 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:07.142606 26224 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.142939 30766 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:07.137557 27126 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:07.144137 30766 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:07.152025 30895 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:07.152863 30895 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:07.153450 30895 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:07.155089 30637 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:31:07.152160 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1621) took 1122 ms. Trace:
I20260924 14:31:07.155857 30637 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:07.155687 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.156044 30414 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:07.156306 30637 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:31:07.156818 27186 raft_consensus.cc:2379] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:07.165817 26405 rpcz_store.cc:276] 0924 14:31:06.029587 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:06.255690 (+226103us) service_pool.cc:227] Handling call
0924 14:31:06.256028 (+   338us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:06.860969 (+604941us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:07.151512 (+290543us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b5400477054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:07.151926 (+   414us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1389759872}
I20260924 14:31:07.167637 30943 consensus_queue.cc:1046] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:31:07.168211 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.169409 26249 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:07.169667 30900 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:07.169835 30688 raft_consensus.cc:3032] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:07.170518 30688 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:07.132668 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 108163us
I20260924 14:31:07.172107 30688 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:07.173235 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 910) took 1261 ms. Trace:
I20260924 14:31:07.131039 30850 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.173704 26327 rpcz_store.cc:276] 0924 14:31:05.912444 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:06.542668 (+630224us) service_pool.cc:227] Handling call
0924 14:31:06.546564 (+  3896us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:07.117448 (+570884us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:07.172842 (+ 55394us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b54002765d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:07.173144 (+   302us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1187213184}
I20260924 14:31:07.176201 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:07.177410 30445 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:07.179576 30850 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:07.134407 30883 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:07.185961 30958 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260924 14:31:07.172518 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.186386 30793 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:07.188150 30826 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:07.188871 30826 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:07.189814 30826 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:07.191663 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 70715.3us
I20260924 14:31:07.193158 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:07.193676 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 63742.4us
I20260924 14:31:06.855628 26329 rpcz_store.cc:276] 0924 14:31:04.969308 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:04.969430 (+   122us) service_pool.cc:227] Handling call
0924 14:31:05.074328 (+104898us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:05.447256 (+372928us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:31:05.460197 (+ 12941us) log.cc:842] Serialized 14 byte log entry
0924 14:31:05.737454 (+277257us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b18000e5540. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd557b 00007f09b935888a
0924 14:31:05.818530 (+ 81076us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b18000e5540. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09bd1f6bcf 00007f09b64267b6 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d
0924 14:31:06.833024 (+1014494us) log.cc:842] Serialized 14 byte log entry
0924 14:31:06.853233 (+ 20209us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:06.853281 (+    48us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:06.853322 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:06.853398 (+    76us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:06.853495 (+    97us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":61696,"prepare.queue_time_us":425501,"prepare.run_cpu_time_us":144,"prepare.run_wall_time_us":13835,"raft.queue_time_us":22620,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":21914,"spinlock_wait_cycles":17942784,"thread_start_us":1046856,"threads_started":2,"wal-append.queue_time_us":998035}
I20260924 14:31:07.196300 30961 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:07.198897 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.213023 30274 consensus_queue.cc:1046] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:07.213815 30668 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:07.214684 30274 raft_consensus.cc:3032] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:07.215130 30274 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.215459 30445 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:07.216027 30445 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.216572 27126 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:07.217288 30445 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:07.218295 30274 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:07.221935 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 904) took 1335 ms. Trace:
I20260924 14:31:07.222571 26330 rpcz_store.cc:276] 0924 14:31:05.866027 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:06.383930 (+517903us) service_pool.cc:227] Handling call
0924 14:31:06.384356 (+   426us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:06.384644 (+   288us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:31:06.833030 (+448386us) log.cc:842] Serialized 14 byte log entry
0924 14:31:07.200866 (+367836us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:07.200994 (+   128us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:07.201093 (+    99us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:07.201211 (+   118us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:07.201416 (+   205us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16116,"prepare.queue_time_us":407904,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":60,"raft.queue_time_us":16299,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":178,"thread_start_us":722407,"threads_started":2,"wal-append.queue_time_us":387780}
W20260924 14:31:07.225756 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53436.8us
I20260924 14:31:07.227592 30688 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:07.228789 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.231878 30445 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:07.213626 26081 catalog_manager.cc:5495] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 5 to 7, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 7 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:07.231935 30876 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:07.236284 26298 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:07.237243 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 107799us
I20260924 14:31:07.238178 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.238898 30876 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:07.255134 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 112726us
I20260924 14:31:07.247993 30668 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:07.258064 30274 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:07.256963 30842 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:07.259334 30961 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260924 14:31:07.279845 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 63478.4us
I20260924 14:31:07.289220 26224 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.289603 30843 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:31:07.291121 30843 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:07.291949 30843 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:07.293115 30374 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:07.296618 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 58566.3us
I20260924 14:31:07.297652 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.270884 30961 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:07.302620 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.303800 26405 raft_consensus.cc:1226] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:31:07.304150 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.305001 26400 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:07.305881 26404 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:07.306574 30929 consensus_queue.cc:1046] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:07.307251 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.298908 30843 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:07.309017 30943 raft_consensus.cc:3032] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:07.309558 30943 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.310611 30943 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:07.300479 30961 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:07.312507 30690 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:31:07.300513 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 71932.2us
I20260924 14:31:07.308372 26397 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:07.318668 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:07.319533 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:07.321372 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:07.322578 26331 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:07.314728 30843 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:07.329779 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.270133 30842 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:07.331583 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.332054 30842 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:07.332831 30940 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:07.333487 30940 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:07.334046 30940 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:07.336638 30897 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:07.337211 26332 raft_consensus.cc:2396] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260924 14:31:07.339423 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:07.340791 30668 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:07.341256 30668 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:07.341671 30668 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:07.342026 30741 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:31:07.343833 29682 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:07.344250 29682 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.345547 29682 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.349577 30766 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.352509 30766 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:07.353642 30963 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:07.355841 26326 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:07.369935 30946 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.336884 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:07.370030 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.339789 30445 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260924 14:31:07.373342 29682 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:07.372745 30637 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:07.374471 30943 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:07.376504 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:07.378357 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:07.379189 30741 consensus_queue.cc:1046] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:31:07.381198 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:07.388183 26404 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:07.408851 26400 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:07.396445 30374 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:07.394695 27125 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:07.419018 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 82304.1us
I20260924 14:31:07.413616 30766 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:07.439738 26256 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 5. Current term is 6. Ops: []
I20260924 14:31:07.433676 26323 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:07.425810 26331 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:07.451822 30752 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:31:07.456931 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.457966 26402 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:07.411688 30895 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.460152 30895 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:07.460785 26253 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:07.444160 30766 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:07.463945 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.465291 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.465744 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.466403 26402 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 9.
I20260924 14:31:07.466676 27191 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:07.467872 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:07.464023 30766 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:07.477591 26405 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:07.478304 27189 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
W20260924 14:31:07.487083 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 92804.3us
I20260924 14:31:07.456986 30637 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:07.497022 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.508962 30739 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:07.509517 26400 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:07.509294 30883 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:31:07.510982 30741 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:07.511463 30741 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:07.511825 30741 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:07.509249 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 177884us
I20260924 14:31:07.519790 26404 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:07.517858 30900 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:31:07.445237 30963 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.529531 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:07.532823 30795 raft_consensus.cc:3032] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:07.533274 30795 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.534157 30795 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.539605 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.547899 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.548396 26253 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:31:07.548949 26258 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:07.551044 30637 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:07.551515 30637 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:07.551882 30637 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:07.554292 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.555537 26256 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 9: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:07.557575 30940 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.558404 30787 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:07.559757 30628 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:07.561210 30877 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Stepping down as leader of term 18
I20260924 14:31:07.561600 30877 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:07.562420 30842 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.565227 30897 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:07.568228 26324 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:07.573788 26254 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:07.559342 30787 consensus_queue.cc:1225] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260924 14:31:07.589322 30971 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:07.592211 30668 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.599776 30971 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.601778 30971 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:07.602418 30971 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:07.608785 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.584344 30795 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:07.531935 30900 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
W20260924 14:31:07.638388 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 43712.7us
I20260924 14:31:07.639040 26399 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:07.584002 30946 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:07.602973 30971 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.644884 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.651176 26258 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260924 14:31:07.651360 26249 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:07.601788 30668 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:07.652074 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:07.654371 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.665552 26256 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:07.646368 30690 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:31:07.671900 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 159952us
I20260924 14:31:07.562493 30877 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.658778 30946 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:07.681731 26253 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:07.691888 30964 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:07.692633 30964 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.644867 30826 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.694882 30964 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:07.695215 30826 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:07.696139 30374 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:07.696725 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.696946 30766 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.698037 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.699607 30766 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:07.701066 30961 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.705756 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.707229 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:07.708083 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 113920us
I20260924 14:31:07.708607 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.717118 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.718709 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.720099 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.712122 30374 consensus_queue.cc:1046] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260924 14:31:07.721485 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.715886 26257 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:07.731848 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:07.732473 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 154937us
I20260924 14:31:07.733021 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.733170 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.734001 26257 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:07.735630 30668 consensus_queue.cc:1046] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:31:07.737056 30688 raft_consensus.cc:3032] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:31:07.737465 30688 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.737730 26258 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:07.738451 30688 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:07.739658 30274 raft_consensus.cc:3032] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:31:07.740031 30688 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:31:07.740231 30274 raft_consensus.cc:729] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.741505 30274 consensus_queue.cc:260] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:07.743615 30842 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:07.747591 30946 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.748080 30628 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:07.748692 30628 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:07.750180 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:07.750000 30628 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.753937 30940 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:07.752187 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:07.754262 30895 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:07.754904 30895 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.755823 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:07.755903 30895 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.757782 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:07.757890 30895 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:07.758518 30741 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.723476 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.754323 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:07.760614 26255 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:07.724794 27191 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:07.711118 30971 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:07.726545 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.726714 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:07.697014 30970 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260924 14:31:07.726810 30637 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.734014 26254 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:31:07.726119 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 345006us
I20260924 14:31:07.760493 26330 raft_consensus.cc:2362] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10. Re-sending same reply.
I20260924 14:31:07.773658 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.781620 26400 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:07.771380 26324 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:07.774785 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.786407 30900 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:07.786773 26254 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:07.722067 30793 raft_consensus.cc:3032] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:07.778086 26251 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:07.796305 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:07.789410 26252 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:31:07.793777 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.794365 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:07.799603 26323 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:07.806154 26401 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:07.764353 30274 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:31:07.808996 30971 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:07.809666 30971 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.810045 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:07.810895 26330 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:07.812801 27189 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:07.810060 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:07.750013 30946 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:07.816251 26256 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:31:07.817143 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.817898 30628 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:07.818192 26251 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:07.818449 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.819844 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.821329 30750 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:31:07.822052 30971 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:07.761509 30961 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:07.822405 26401 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:31:07.823264 27126 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:07.823622 30877 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:07.824644 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.779314 30963 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:07.823649 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.828140 30978 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:07.828682 30978 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.830058 26224 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:07.829689 30978 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.832576 30974 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:07.832886 30976 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:07.833233 30974 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:07.801692 30668 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260924 14:31:07.834815 30445 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:07.835134 30668 raft_consensus.cc:3032] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:07.835230 30445 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.836023 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:07.836374 30445 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.837129 26251 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:07.837296 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.837729 30445 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:07.838585 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.844125 26253 raft_consensus.cc:2379] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:07.844810 27192 raft_consensus.cc:2379] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:07.847596 27189 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:07.849210 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.849933 26258 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:07.843006 30971 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:31:07.853928 27190 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:07.848846 30900 consensus_queue.cc:1046] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260924 14:31:07.854562 27181 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:07.852893 30741 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:07.855100 30900 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:07.795302 30793 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:07.835675 30668 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.867514 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.868959 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.875201 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:07.878789 26325 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:07.880761 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.881632 26251 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:07.868882 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:07.858888 30900 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:07.894469 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:07.865475 30971 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:07.899572 27126 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:07.909094 27182 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:07.911631 27191 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:07.919291 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:07.920322 26325 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:07.922991 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:07.933490 26369 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:07.889739 30940 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260924 14:31:07.964958 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.959697 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:07.899152 30668 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:07.965399 26258 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:07.970157 27126 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:07.985401 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:07.995254 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.997973 26399 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:08.008054 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:08.008286 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:07.937413 30943 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260924 14:31:08.005797 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:08.025115 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:08.002379 30795 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:31:07.928742 30793 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:07.996125 30402 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.185s
I20260924 14:31:08.035149 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:08.044795 30972 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:08.045157 30903 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:08.046577 30942 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260924 14:31:08.046064 30903 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:08.047478 30842 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:08.048139 30842 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:08.050423 30842 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:08.047546 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 256959us
I20260924 14:31:08.052359 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.053399 30883 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:08.054230 26324 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:31:08.054030 30883 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:08.054440 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.055382 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:08.056064 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.056350 30683 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:08.056838 30683 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:08.057893 30683 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:08.017572 30637 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:08.060359 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:08.060701 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.061556 26325 raft_consensus.cc:2396] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260924 14:31:08.022564 30668 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:07.995262 30741 consensus_queue.cc:1046] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260924 14:31:08.067576 26330 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:08.075740 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:07.994680 30971 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:08.084635 27191 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:08.077148 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.087033 26326 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 10
I20260924 14:31:07.981096 30900 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:08.094193 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:08.097191 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:08.103727 26332 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:08.069676 30787 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:31:08.107587 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 69978.9us
W20260924 14:31:08.111821 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 57488.9us
I20260924 14:31:08.112141 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:08.115633 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 182260us
I20260924 14:31:08.085489 30972 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:08.125465 26399 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:08.131808 26298 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.139365 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:08.113451 30683 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:08.115698 30795 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:08.152513 26331 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260924 14:31:08.159494 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:08.159807 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:08.160274 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:08.155741 26331 pending_rounds.cc:76] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 7
I20260924 14:31:08.163160 26331 pending_rounds.cc:97] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260924 14:31:08.170081 26399 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:08.155244 30900 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:08.171589 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:08.178398 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:08.179683 26400 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:08.178701 30795 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:08.181485 30987 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260924 14:31:08.181936 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:08.182247 30943 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:08.175634 26331 raft_consensus.cc:2866] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:08.183887 30972 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:08.190467 30883 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:08.176436 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:08.178694 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:08.195150 26406 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:31:08.167884 30741 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:08.200716 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:08.200884 30688 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:08.167135 30628 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:08.212529 27189 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:31:08.210583 26326 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:08.178421 30984 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:31:08.220391 26400 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
W20260924 14:31:08.220539 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 107781us
I20260924 14:31:08.225016 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:08.229709 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:08.249783 30752 raft_consensus.cc:3032] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:08.250342 30752 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:08.251300 30752 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:08.252265 30991 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:31:08.206858 30903 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:08.253779 30984 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:08.263950 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:08.263240 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 169474us
W20260924 14:31:08.281224 26221 connection.cc:232] RPC call timeout handler was delayed by -1.0148636460007765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
I20260924 14:31:08.285467 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:08.287528 27181 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:08.264596 30752 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:08.297540 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:08.301476 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:08.303398 26400 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:08.312057 30628 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:31:08.292382 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:08.311718 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 86824.6us
I20260924 14:31:08.318181 30374 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:08.287302 30964 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:08.326659 30974 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:08.328178 30984 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:08.290125 30637 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:08.314441 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:08.295614 30903 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260924 14:31:08.290072 27194 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:08.344545 26400 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:08.345235 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:08.275300 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61428.1us
I20260924 14:31:08.337954 26406 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:31:08.342715 26251 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:08.347234 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.351346 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.351935 26330 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:08.352056 26329 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:08.344733 27182 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
W20260924 14:31:08.359797 30445 consensus_meta.cc:287] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.522s	user 0.015s	sys 0.000s
I20260924 14:31:08.363842 30992 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260924 14:31:08.363955 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49657.1us
I20260924 14:31:08.365585 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.366880 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.366585 30895 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:08.368342 30787 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:08.368578 30374 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260924 14:31:08.369102 27192 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:08.369933 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.333079 30964 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:08.343878 30637 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:08.397869 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 96462.1us
I20260924 14:31:08.399102 26221 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.380759 26332 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:08.355804 30843 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:08.412030 30993 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:08.379755 30974 consensus_queue.cc:1046] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260924 14:31:08.415709 30842 raft_consensus.cc:3032] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:08.416157 30842 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:08.417282 30842 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:08.418764 30842 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:08.424657 29682 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:08.425465 29682 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:08.425954 30940 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260924 14:31:08.427964 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 82812us
I20260924 14:31:08.429728 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.380371 30964 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:08.352671 30628 consensus_queue.cc:1225] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260924 14:31:08.397630 30637 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:08.514212 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 115229us
I20260924 14:31:08.515708 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:08.519153 30877 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:08.522671 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:08.523793 26330 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:08.535079 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 105430us
I20260924 14:31:08.512142 30843 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:08.540974 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 174177us
I20260924 14:31:08.542098 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.506783 26332 pending_rounds.cc:76] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 6
I20260924 14:31:08.544046 26332 pending_rounds.cc:97] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260924 14:31:08.544713 26332 raft_consensus.cc:2866] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:08.549122 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.550432 30993 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:08.552393 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.553586 30993 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:31:08.555392 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.560066 30843 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:08.563941 30993 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:08.565577 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.567209 30993 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:08.579638 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.581061 30903 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:31:08.581809 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.581812 30903 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:08.584276 30903 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:08.587723 30374 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:08.588104 30993 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:08.597538 26406 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260924 14:31:08.599625 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 244659us
I20260924 14:31:08.599745 30752 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:08.600893 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:08.601948 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:08.602797 30374 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:08.603334 30374 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:08.603518 30992 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:08.604417 30374 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:08.608285 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.611156 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:08.616372 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 74345.4us
I20260924 14:31:08.621151 27191 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:08.613015 27189 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:08.548357 26329 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:08.609442 30990 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:31:08.542716 30274 consensus_meta.cc:287] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.709s	user 0.014s	sys 0.000s
I20260924 14:31:08.625157 27188 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:08.628841 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:08.630342 26324 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:08.634382 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:08.626168 27125 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:08.638811 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:08.641059 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.641748 30972 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260924 14:31:08.641669 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 126090us
I20260924 14:31:08.643093 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.643674 30842 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:31:08.644290 30842 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:08.644464 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:08.645305 30077 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:08.645406 30974 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:31:08.642203 30929 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260924 14:31:08.647563 30972 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:08.649029 30964 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:08.651377 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.652846 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:08.653817 26258 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
W20260924 14:31:08.654997 30981 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.845s	user 0.001s	sys 0.006s
I20260924 14:31:08.655135 26257 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260924 14:31:08.651786 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.651649 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:08.633236 30900 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:08.665518 26327 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:08.668488 26324 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:31:08.668990 26324 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:08.667619 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:08.671051 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.661540 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:08.674897 26329 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:08.675977 26323 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:08.657706 30990 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:31:08.656188 26330 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:08.650198 30946 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election won for term 6
I20260924 14:31:08.636067 30877 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:31:08.645896 30077 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:08.665637 30993 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:08.694048 26324 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:08.707042 26325 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:31:08.708431 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:08.699784 26297 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.695s (SENT) [suppressed 25 similar messages]
I20260924 14:31:08.710160 26324 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:08.712386 30946 raft_consensus.cc:686] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:08.680912 30972 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:08.645336 30842 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:08.660862 30986 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260924 14:31:08.740644 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:08.673179 30900 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:08.748629 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 107662us
I20260924 14:31:08.751572 26249 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:08.699512 30688 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260924 14:31:08.753487 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.760826 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:08.747545 26323 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:08.804488 26323 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:08.739012 30974 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:08.879078 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:08.919143 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 93985.8us
I20260924 14:31:08.714032 30946 consensus_queue.cc:237] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:08.906958 30402 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:08.929212 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:08.943979 30274 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:08.944620 30274 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:08.945640 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:08.945643 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:08.947763 26250 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:08.948222 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:08.949199 30940 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:08.949683 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:08.931113 26324 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:31:08.876929 30961 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:08.951150 26323 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:31:08.765959 31004 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:08.946839 30274 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:08.949182 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:08.969493 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:08.949784 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:08.930858 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:08.934839 31001 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
W20260924 14:31:08.932809 31003 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
W20260924 14:31:08.951644 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 271830us
I20260924 14:31:08.994244 26252 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:08.948124 26256 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
W20260924 14:31:08.906628 30991 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260924 14:31:08.765625 30978 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.012s	sys 0.008s
I20260924 14:31:08.961513 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:09.000382 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.000869 26255 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:31:08.958415 30739 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260924 14:31:08.974852 26405 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:31:08.975735 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:08.976153 26250 raft_consensus.cc:2354] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
I20260924 14:31:08.765307 30637 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:09.022717 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:08.950182 30940 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:08.998013 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:09.032135 30940 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:09.032374 26323 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:08.941705 30964 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:08.944036 30402 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.037837 30402 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:09.012879 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:09.015066 26369 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:09.015858 30990 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:31:09.039256 26255 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:31:09.004025 30978 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.171s	user 0.012s	sys 0.008s
W20260924 14:31:09.020323 31002 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260924 14:31:09.020781 26250 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:09.021672 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:09.022122 26402 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:08.958726 30445 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:09.044592 30445 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.031582 26253 raft_consensus.cc:2354] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
I20260924 14:31:09.047972 30765 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:09.048600 30765 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:09.049691 30765 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:09.051940 26400 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 19: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:09.052889 30993 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:31:09.053885 29682 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.057154 30637 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:09.057979 30637 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:09.059297 27182 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:08.876926 30077 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.061086 30077 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:08.912679 30974 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.070268 30688 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:09.071137 30688 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:08.959404 30961 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:09.094551 30961 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:09.003582 30972 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:09.095638 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:09.079231 30688 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:09.080076 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:08.961683 31004 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:09.109136 31004 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:09.087304 30977 raft_consensus.cc:3032] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:31:09.091814 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:09.113523 26256 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:09.104163 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:09.114189 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 99210.4us
I20260924 14:31:09.118866 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:09.119745 26253 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
W20260924 14:31:09.122720 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 124083us
I20260924 14:31:09.130435 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:09.047382 30445 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:09.135216 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:09.137926 30741 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:09.139685 31018 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:09.111438 30765 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:09.100886 30974 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:09.148430 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:09.121423 30977 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:09.155987 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:09.193035 26327 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:08.942685 30992 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.189s
I20260924 14:31:09.196480 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:09.194137 30637 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:09.191877 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 242181us
W20260924 14:31:09.192422 30963 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.365s	user 0.012s	sys 0.021s
W20260924 14:31:09.201122 30963 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.374s	user 0.012s	sys 0.021s
I20260924 14:31:09.200569 30637 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.204042 30637 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:09.204330 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.134749 30986 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election won for term 10
W20260924 14:31:09.210959 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 80737.1us
I20260924 14:31:09.154979 30741 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:09.218811 30402 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:09.220904 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:09.225422 26250 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:09.221124 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:09.226935 26326 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:09.227921 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:09.234828 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:09.234664 26327 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:09.213831 30900 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:09.236512 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 213373us
I20260924 14:31:09.155084 30374 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260924 14:31:09.241199 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:09.215580 30445 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:09.247845 26298 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:09.255681 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 107353us
I20260924 14:31:09.256482 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.234014 30986 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:09.264647 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:09.278151 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.251760 30445 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:09.238426 30402 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:09.287750 26406 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:09.290879 26402 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:09.290931 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:09.293254 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 90799.6us
W20260924 14:31:09.273687 31010 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
I20260924 14:31:09.267879 31018 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:09.273280 30877 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:09.298431 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.252218 30900 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260924 14:31:09.283733 30990 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:09.276525 30787 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:09.283375 30992 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:09.288434 30445 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:09.315703 30972 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260924 14:31:09.319911 31008 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260924 14:31:09.320729 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:09.293000 30688 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260924 14:31:09.322048 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 87303us
I20260924 14:31:09.293175 30961 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260924 14:31:09.325474 31016 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260924 14:31:09.316676 31018 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:09.276722 31024 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260924 14:31:09.336329 30972 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:09.277067 30741 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:09.303258 30940 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:09.337821 30688 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:09.355877 30402 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:09.361676 26252 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:31:09.361954 31023 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260924 14:31:09.383416 30978 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:09.383950 26254 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:09.384737 30741 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:09.384925 30940 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:09.378724 30972 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:09.380347 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:09.363821 26249 consensus_meta.cc:287] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.002s
I20260924 14:31:09.347186 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.345115 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:09.346289 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 79664.1us
W20260924 14:31:09.371631 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 101106us
I20260924 14:31:09.322937 30877 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:09.393080 26249 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:09.377411 26400 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:09.352141 30752 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:09.380358 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.381001 26402 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:09.384016 30978 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.193586 30977 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:09.346204 30637 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260924 14:31:09.404240 26399 raft_consensus.cc:2379] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 6: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:09.401899 26404 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
W20260924 14:31:09.340911 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 99809.8us
I20260924 14:31:09.408183 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.408288 31035 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260924 14:31:09.409173 26402 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:09.409621 31035 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:09.409482 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0756687d99e643c094531124c9438376"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:09.408212 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.410432 27188 raft_consensus.cc:595] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:09.410629 26403 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:09.410828 27188 raft_consensus.cc:603] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:09.414425 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1486) took 1361 ms. Trace:
I20260924 14:31:09.414858 26331 rpcz_store.cc:276] 0924 14:31:08.052917 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:08.059546 (+  6629us) service_pool.cc:227] Handling call
0924 14:31:08.059834 (+   288us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:08.183651 (+123817us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:09.414173 (+1230522us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b5400558cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:09.414343 (+   170us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":212097408}
I20260924 14:31:09.416004 30877 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:09.390751 30688 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:09.410099 31035 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:09.435190 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:09.436837 30971 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.177s	user 0.006s	sys 0.014s
W20260924 14:31:09.437268 30971 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.186s	user 0.006s	sys 0.019s
I20260924 14:31:09.437948 26257 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:09.438764 31038 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:31:09.440112 31038 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:09.440465 31038 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:09.441293 31038 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:09.441514 30445 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.442565 31038 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:09.445664 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:09.446885 30971 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:09.399859 30974 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:31:09.407178 30982 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 1.563s	user 0.015s	sys 0.016s
W20260924 14:31:09.407035 31017 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260924 14:31:09.450728 30843 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260924 14:31:09.453632 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 108642us
I20260924 14:31:09.406661 31022 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
W20260924 14:31:09.456297 31015 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260924 14:31:09.457331 31035 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:09.401016 30752 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:09.474730 31033 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:09.474532 31018 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.366577 30986 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:09.406833 30637 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:09.424749 30940 raft_consensus.cc:2741] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260924 14:31:09.478955 31037 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:09.479128 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:09.479702 30985 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:09.480300 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1494) took 1079 ms. Trace:
I20260924 14:31:09.511181 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:09.480235 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:09.511938 27188 raft_consensus.cc:595] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:09.487860 26329 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260924 14:31:09.476241 30961 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:09.520546 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:09.450860 30946 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260924 14:31:09.491187 27182 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:09.464952 30978 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:09.491852 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 217532us
I20260924 14:31:09.498607 31039 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:09.505807 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.498852 30765 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:09.479250 30990 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:09.477551 30843 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:09.512305 27188 raft_consensus.cc:603] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:09.513512 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:09.511592 26332 rpcz_store.cc:276] 0924 14:31:08.402676 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:08.403396 (+   720us) service_pool.cc:227] Handling call
0924 14:31:08.506569 (+103173us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:08.545688 (+ 39119us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:08.545734 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:08.633751 (+ 88017us) log.cc:842] Serialized 26 byte log entry
0924 14:31:08.957740 (+323989us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:08.957800 (+    60us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:08.957899 (+    99us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:09.480079 (+522180us) raft_consensus.cc:1671] finished
0924 14:31:09.480101 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:09.480237 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":303726,"threads_started":1,"wal-append.queue_time_us":586472}
I20260924 14:31:09.535501 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:09.477873 30445 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:09.512302 26323 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:09.521706 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 789) took 1315 ms. Trace:
I20260924 14:31:09.465922 30739 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260924 14:31:09.524473 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:09.545636 26249 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:09.525023 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.512758 30637 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:31:09.546643 26400 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:09.536218 26254 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 16
I20260924 14:31:09.538002 26329 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:09.522526 30787 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:09.526058 30940 raft_consensus.cc:2741] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260924 14:31:09.542380 26406 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:09.524471 30961 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:09.542133 27192 rpcz_store.cc:276] 0924 14:31:08.194535 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:08.194691 (+   156us) service_pool.cc:227] Handling call
0924 14:31:09.510072 (+1315381us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540073c654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:09.510204 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":366028672}
I20260924 14:31:09.529265 31024 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:09.542399 26326 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:09.551229 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.543109 30843 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:09.573666 31034 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:09.574021 30983 raft_consensus.cc:3032] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:09.574618 30983 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:09.546195 30978 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:09.574318 31034 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.514772 31033 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:09.584024 30739 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:31:09.531562 31038 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260924 14:31:09.595114 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.596081 26399 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:09.540143 30765 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:09.571146 30940 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:09.601671 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "64e0b4058f18400dab940d3832586746"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:09.602375 27189 raft_consensus.cc:595] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:09.611016 31044 log.cc:924] Time spent T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260924 14:31:09.552170 30787 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:09.588486 30978 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:09.539255 30752 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:09.598536 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53454.6us
I20260924 14:31:09.634915 30752 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:09.635100 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:09.565541 31024 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.636705 31033 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:09.638195 31024 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:09.638772 30983 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.640205 30983 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:09.640816 30765 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:09.641157 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.642135 30940 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:09.642485 30940 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:09.642530 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.642946 30940 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:09.643901 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.597891 30741 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:09.590488 30739 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:09.539310 31039 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.630467 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:09.639325 30978 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:09.647948 26404 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260924 14:31:09.619145 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45982.4us
I20260924 14:31:09.660185 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.657621 26399 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:09.661078 26400 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:09.667798 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "70ed8dcd463347619717298c81a42998"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:09.668509 27189 raft_consensus.cc:595] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:09.668888 27189 raft_consensus.cc:603] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:09.673015 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:09.673357 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:09.673713 30903 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:09.674463 30903 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:09.674695 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:09.675354 26258 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:31:09.673810 26250 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:09.679152 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:09.656984 30946 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:31:09.681325 27194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 786) took 1606 ms (client timeout 1835 ms). Trace:
I20260924 14:31:09.659837 30971 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:09.653180 31023 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:09.685024 27125 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:09.691386 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 164073us
W20260924 14:31:09.689483 27194 rpcz_store.cc:269] 0924 14:31:08.075085 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:08.075260 (+   175us) service_pool.cc:227] Handling call
0924 14:31:09.681164 (+1605904us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b54000ce9d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:09.681260 (+    96us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":103741696}
I20260924 14:31:09.708333 27123 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:09.710281 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:09.717196 26252 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:09.631175 30972 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:09.724767 31024 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:09.725322 31024 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:09.725680 31024 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:09.727814 30787 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:09.727907 30739 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:09.729364 30739 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:09.706073 30765 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:31:09.731808 30765 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:09.732216 30765 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:09.733358 31033 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:09.733803 31033 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:09.734143 31033 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:09.739580 31049 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260924 14:31:09.742750 30793 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:09.745245 31031 consensus_queue.cc:1046] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.196s
W20260924 14:31:09.746562 31042 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260924 14:31:09.756253 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1517) took 1019 ms. Trace:
W20260924 14:31:09.756757 26371 leader_election.cc:334] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.706s (SENT)
I20260924 14:31:09.757261 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.659209 30741 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.756783 26251 rpcz_store.cc:276] 0924 14:31:08.736695 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:08.736869 (+   174us) service_pool.cc:227] Handling call
0924 14:31:08.754195 (+ 17326us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:08.754540 (+   345us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:08.754567 (+    27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:09.353367 (+598800us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b2c001c3658. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:09.353720 (+   353us) log.cc:842] Serialized 26 byte log entry
0924 14:31:09.450265 (+ 96545us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:09.450318 (+    53us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:09.450416 (+    98us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:09.748991 (+298575us) raft_consensus.cc:1671] finished
0924 14:31:09.749011 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:09.756162 (+  7151us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1858,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":8213,"spinlock_wait_cycles":9121280,"thread_start_us":96106,"threads_started":1,"wal-append.queue_time_us":154669}
I20260924 14:31:09.769093 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.769096 26254 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:09.774583 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:09.767244 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:09.715902 30943 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:09.776712 26250 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:09.804060 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.783665 26399 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:09.831835 27192 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:09.799624 26401 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:09.836032 27192 raft_consensus.cc:595] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:09.755903 30961 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:09.765816 30929 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:09.837889 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.838397 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:09.843014 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.774762 30985 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:31:09.849300 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 92562.3us
I20260924 14:31:09.783265 30943 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:09.857861 26371 leader_election.cc:334] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.717s (SENT)
I20260924 14:31:09.862099 30861 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:09.869469 26371 leader_election.cc:334] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.785s (SENT)
I20260924 14:31:09.870116 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:09.870620 31050 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260924 14:31:09.872314 31047 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260924 14:31:09.749236 30903 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:09.874994 30903 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:09.875669 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 231851us
I20260924 14:31:09.876535 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:09.869164 30861 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:09.878593 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.880915 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.881709 26399 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
W20260924 14:31:09.882828 26297 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.522s (SENT)
I20260924 14:31:09.883316 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:09.884541 31043 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:09.885519 26331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:31:09.886112 26331 raft_consensus.cc:595] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:09.886439 26331 raft_consensus.cc:603] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:09.888623 31023 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:09.889117 31023 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:09.889688 31023 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:09.782382 30984 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260924 14:31:09.891131 31039 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:09.892125 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:09.892664 30741 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:09.889214 30637 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:09.838636 31034 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:09.845821 30795 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:09.939145 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 47101.7us
I20260924 14:31:09.863759 30943 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:09.863863 30929 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.963837 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.835354 30946 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260924 14:31:09.971863 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260924 14:31:09.969304 30688 consensus_meta.cc:287] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.001s
I20260924 14:31:09.926806 31043 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:09.981952 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:31:09.992446 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.995570 26328 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:09.995510 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:09.996320 26405 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:31:09.998239 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:09.998353 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:09.999389 30978 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:10.001979 27126 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:10.004474 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:10.008199 26328 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
W20260924 14:31:10.013075 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 119586us
I20260924 14:31:09.971362 30971 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:09.975672 31039 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:10.051920 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 182487us
W20260924 14:31:10.052845 26371 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.809s (SENT)
I20260924 14:31:09.967630 31060 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:09.976271 31004 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:10.055117 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:10.035764 31058 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260924 14:31:10.039201 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 48251.2us
I20260924 14:31:10.061076 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:10.040086 31071 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:10.048732 30843 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:10.019364 30688 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:09.939668 30795 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:10.126991 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 66002.3us
I20260924 14:31:10.049964 30668 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260924 14:31:10.132052 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 357520us
I20260924 14:31:10.135285 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.135962 26252 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:10.051640 31034 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260924 14:31:10.136693 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 73571.8us
W20260924 14:31:10.059860 30975 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.011s	sys 0.019s
I20260924 14:31:10.145170 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.150816 26252 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:10.131260 31004 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:10.148536 26329 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:10.132543 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:10.150857 30975 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.012s	sys 0.020s
I20260924 14:31:10.162055 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:10.163995 30971 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:10.164858 30971 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:10.165393 30971 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:10.166468 30971 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.168409 26249 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 6. Current term is 7. Ops: []
I20260924 14:31:10.169186 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.172691 26252 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 5.
I20260924 14:31:10.063547 30972 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:10.190001 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 137173us
I20260924 14:31:10.133426 31060 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260924 14:31:10.170428 31070 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260924 14:31:10.187335 27125 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:10.144847 30668 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:10.206758 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:10.144181 30990 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:10.212268 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:10.213143 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:10.020367 31043 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.061192 31039 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:10.215503 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 208323us
I20260924 14:31:10.182788 30978 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:10.215538 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:10.187947 31018 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:10.206241 31034 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.207782 31071 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.214958 30668 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:10.248588 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 54472.4us
I20260924 14:31:10.259546 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:10.260999 27126 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:10.263687 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 101699us
I20260924 14:31:10.227603 30752 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260924 14:31:10.289986 27194 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:10.247576 30903 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260924 14:31:10.293974 30903 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.294478 30903 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:10.297190 31064 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260924 14:31:10.299947 31056 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.316s	user 0.001s	sys 0.000s
I20260924 14:31:10.302174 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:10.304016 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:10.323661 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:10.329133 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 137422us
I20260924 14:31:10.264081 31066 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:10.227553 30990 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:10.336746 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:10.240942 31034 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.285996 30637 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:10.301323 30895 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:10.248577 31018 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:10.362901 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 149817us
I20260924 14:31:10.319401 30842 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:10.252766 31071 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:10.156296 31004 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:10.370134 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.371102 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.338771 30972 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.343863 30741 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:10.372583 26252 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
W20260924 14:31:10.410915 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 149945us
W20260924 14:31:10.390568 31056 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260924 14:31:10.399119 31047 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260924 14:31:10.373559 31073 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:31:10.362059 30929 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:10.390442 30688 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:10.374087 30637 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:10.362491 30895 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:10.405649 31033 consensus_meta.cc:287] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.668s	user 0.007s	sys 0.005s
I20260924 14:31:10.394477 30843 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:10.383051 31060 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election won for term 13
W20260924 14:31:10.428176 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 104606us
W20260924 14:31:10.405988 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 151877us
I20260924 14:31:10.363904 30978 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260924 14:31:10.482271 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:10.483129 26252 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:10.412402 30795 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:10.516625 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:10.516963 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:10.517638 27194 raft_consensus.cc:2379] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:10.516646 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:10.427592 31073 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:10.516394 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 843) took 1043 ms. Trace:
I20260924 14:31:10.553258 27191 rpcz_store.cc:276] 0924 14:31:09.473310 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:09.473444 (+   134us) service_pool.cc:227] Handling call
0924 14:31:09.473921 (+   477us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:09.490484 (+ 16563us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:31:09.507255 (+ 16771us) log.cc:842] Serialized 14 byte log entry
0924 14:31:09.678197 (+170942us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:09.699873 (+ 21676us) log.cc:842] Serialized 26 byte log entry
0924 14:31:09.711572 (+ 11699us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:09.711623 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:09.711714 (+    91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:10.516184 (+804470us) raft_consensus.cc:1671] finished
0924 14:31:10.516204 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:10.516335 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16034,"prepare.queue_time_us":33281,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":1959,"raft.queue_time_us":24898,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":17161,"thread_start_us":170482,"threads_started":1,"wal-append.queue_time_us":340556}
W20260924 14:31:10.627177 26297 leader_election.cc:334] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.143s (SENT)
I20260924 14:31:10.637111 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:10.476514 30895 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:10.404198 30972 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:10.647558 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:10.648685 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:10.658052 27188 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
W20260924 14:31:10.662775 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 114729us
W20260924 14:31:10.663522 26297 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.989s (SENT)
I20260924 14:31:10.663894 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:10.629256 31073 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:10.518605 31088 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:10.673985 31088 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:10.674669 31088 consensus_queue.cc:1046] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:31:10.675647 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:10.676267 31088 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:10.648897 31068 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:10.677390 31068 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:10.677531 31088 raft_consensus.cc:3032] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:10.677819 31068 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:10.678128 31088 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:10.552431 31060 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:10.680953 31074 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:10.688966 31074 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:10.689989 31022 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.689908 31074 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:10.690546 31022 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.654615 30077 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.691933 30077 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:10.692492 31089 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
I20260924 14:31:10.652761 31041 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:10.693500 31041 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:10.693876 31041 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:10.694155 30402 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:10.695040 31067 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260924 14:31:10.486182 31033 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:10.696623 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1506) took 1012 ms. Trace:
I20260924 14:31:10.673430 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:10.647536 30842 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:10.708470 27181 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:10.697063 26326 rpcz_store.cc:276] 0924 14:31:09.684426 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:09.684556 (+   130us) service_pool.cc:227] Handling call
0924 14:31:09.709639 (+ 25083us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:09.752036 (+ 42397us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:09.752081 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:09.767239 (+ 15158us) log.cc:842] Serialized 26 byte log entry
0924 14:31:10.135624 (+368385us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:10.135679 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:10.135780 (+   101us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:10.696418 (+560638us) raft_consensus.cc:1671] finished
0924 14:31:10.696437 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:10.696561 (+   124us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":720,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":6818,"thread_start_us":290478,"threads_started":1,"wal-append.queue_time_us":483825}
I20260924 14:31:10.664779 31039 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:10.744072 29682 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:10.748255 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 231682us
I20260924 14:31:10.748667 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:10.694697 31041 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:10.748133 26221 connection.cc:232] RPC call timeout handler was delayed by -1.041811578999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
I20260924 14:31:10.754611 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:10.678607 31043 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:10.763602 31103 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:10.764674 30940 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.121s	user 0.012s	sys 0.000s
I20260924 14:31:10.765190 30940 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:10.765931 31069 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260924 14:31:10.767983 31071 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:10.687484 31054 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:10.755347 26404 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:10.707722 30903 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:10.761562 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:10.787207 26399 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:10.763142 26406 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:10.758103 30990 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:10.764230 31103 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:10.751416 26221 connection.cc:232] RPC call timeout handler was delayed by -1.0139115789997959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
I20260924 14:31:10.792547 27194 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260924 14:31:10.722043 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:10.796350 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.797183 26254 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:10.688138 31079 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:10.777655 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:10.769353 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:10.780078 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.693504 30850 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.780090 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:10.785307 26252 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
W20260924 14:31:10.753750 30765 consensus_meta.cc:287] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.021s	user 0.009s	sys 0.004s
I20260924 14:31:10.694221 30077 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:10.802441 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:10.695051 30402 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.803689 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:10.803900 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:10.783485 31054 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:10.806082 31103 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:10.807335 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.784626 30940 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:10.809309 31024 consensus_meta.cc:287] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.083s	user 0.011s	sys 0.001s
I20260924 14:31:10.791640 31043 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260924 14:31:10.810915 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:10.794910 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.811966 31094 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.816227 26251 raft_consensus.cc:2379] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:10.852013 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:10.852758 26251 raft_consensus.cc:2379] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:10.853863 26251 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260924 14:31:10.815587 26399 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:10.816876 26405 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 19: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:10.817231 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:10.817579 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:10.823746 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.824563 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.781231 31031 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.824857 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.826396 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:10.821352 27126 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:10.866324 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45280.8us
I20260924 14:31:10.805020 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 947) took 1319 ms. Trace:
I20260924 14:31:10.866974 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:10.780613 31034 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:10.873854 26258 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:10.828187 26398 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:10.832363 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:10.809003 31023 consensus_meta.cc:287] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.919s	user 0.013s	sys 0.000s
I20260924 14:31:10.873523 27181 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:10.745182 30842 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:10.838400 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:10.792255 31088 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.831382 31071 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:10.839865 31022 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:10.687693 31060 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.823902 30688 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.906733 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:10.907120 26255 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:10.907287 30688 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.824519 31018 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:10.841035 31024 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.885912 31023 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.908447 30861 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:10.842017 31033 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:10.911468 31040 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:10.911922 31040 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:10.912315 31023 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:10.912752 31040 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:10.914217 31040 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:10.916311 30668 consensus_meta.cc:287] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.000s
I20260924 14:31:10.916977 30964 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:10.916782 30668 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:10.917418 30964 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:10.918315 30964 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.839066 31096 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.919277 30668 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:10.881083 30895 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:10.919723 31096 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:10.920732 27192 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:10.921826 31096 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:10.922057 26401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1636) took 2948 ms (client timeout 2076 ms). Trace:
I20260924 14:31:10.922384 30895 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:10.922639 26401 rpcz_store.cc:269] 0924 14:31:07.974099 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:07.974253 (+   154us) service_pool.cc:227] Handling call
0924 14:31:10.921794 (+2947541us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54006e6ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:10.921945 (+   151us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4131561728}
W20260924 14:31:10.847539 30943 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.861s	user 0.008s	sys 0.004s
I20260924 14:31:10.843375 31068 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:10.848481 31095 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.849220 31085 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:10.850011 31086 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.950529 31086 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:10.850520 31094 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:10.855216 26251 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260924 14:31:10.871583 26252 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:10.877140 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:10.910429 31024 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:10.832070 26250 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:10.867492 26324 rpcz_store.cc:276] 0924 14:31:09.486017 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:09.487544 (+  1527us) service_pool.cc:227] Handling call
0924 14:31:09.487926 (+   382us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:09.527868 (+ 39942us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:09.527898 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:09.537393 (+  9495us) log.cc:842] Serialized 26 byte log entry
0924 14:31:09.634217 (+ 96824us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:09.788397 (+154180us) log.cc:842] Serialized 14 byte log entry
0924 14:31:09.790568 (+  2171us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:10.804580 (+1014012us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b5400498254. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:10.804631 (+    51us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:10.804706 (+    75us) raft_consensus.cc:1671] finished
0924 14:31:10.804816 (+   110us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:10.804961 (+   145us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":86731,"prepare.queue_time_us":221783,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":1983,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":1493,"spinlock_wait_cycles":80378112,"thread_start_us":209012,"threads_started":2,"wal-append.queue_time_us":140339,"wal-append.run_cpu_time_us":5021,"wal-append.run_wall_time_us":1100962}
I20260924 14:31:10.831324 30903 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.884622 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.957646 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:10.886065 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 549410us
I20260924 14:31:10.958297 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:10.885474 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 204799us
I20260924 14:31:10.891692 31034 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.878522 31031 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:10.846316 31091 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260924 14:31:10.893072 30402 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:10.893476 31099 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:10.895005 31071 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:10.895762 30842 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:10.828490 30765 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:10.896207 31083 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:10.896647 26254 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:31:10.823765 30850 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.898566 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1605) took 1232 ms. Trace:
I20260924 14:31:10.900483 30977 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:10.902714 31103 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:31:10.903412 31100 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260924 14:31:10.879184 30741 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:11.009063 30741 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:11.009569 30741 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:10.907699 31066 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.555s	user 0.001s	sys 0.009s
I20260924 14:31:10.924072 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1518) took 2168 ms. Trace:
I20260924 14:31:10.909193 26255 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:10.834285 31054 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:11.011797 31066 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:11.011803 26257 rpcz_store.cc:276] 0924 14:31:08.755421 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:08.755727 (+   306us) service_pool.cc:227] Handling call
0924 14:31:08.846930 (+ 91203us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:09.438329 (+591399us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:10.923759 (+1485430us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54002cbfd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:10.923971 (+   212us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1358700800}
I20260924 14:31:10.919363 30964 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:11.015594 26257 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260924 14:31:11.015996 31073 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:11.017019 26257 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:11.018205 31073 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:11.018906 26257 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:10.840767 30972 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:10.941399 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:10.879908 30983 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.011s	sys 0.054s
W20260924 14:31:11.020244 30983 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.011s	sys 0.055s
I20260924 14:31:10.949445 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:10.949163 31085 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:11.022454 26257 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:10.949638 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.023128 31085 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:10.949447 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.024010 27191 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:11.024441 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.025245 27190 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:11.025141 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:10.954551 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 77636.7us
I20260924 14:31:10.877465 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.028695 31083 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:11.028829 31023 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:10.909611 30688 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:11.030227 30688 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:11.029757 31083 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:11.030649 30688 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:11.031010 30688 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:10.959842 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:10.885354 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.954685 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.032922 26251 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:10.960939 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:10.967936 27194 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:31:10.949833 31095 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:10.992660 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:10.956532 30974 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:10.995427 26249 rpcz_store.cc:276] 0924 14:31:09.666403 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:09.666644 (+   241us) service_pool.cc:227] Handling call
0924 14:31:09.666967 (+   323us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:10.181178 (+514211us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:10.898242 (+717064us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b540015e0d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:10.898469 (+   227us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1131300608}
I20260924 14:31:10.998729 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:11.042340 26326 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:11.043078 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.043493 26405 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260924 14:31:10.988173 31096 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:10.999590 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.000530 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.044857 27186 raft_consensus.cc:2354] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:31:10.962764 30739 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:11.005635 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:10.945544 30943 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:11.007151 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:10.909803 31092 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election won for term 11
W20260924 14:31:11.029361 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 151973us
I20260924 14:31:11.048468 27189 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: []
I20260924 14:31:10.997632 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:11.028892 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:11.012125 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.039871 26329 raft_consensus.cc:2379] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:11.053895 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:11.004937 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.045910 26325 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:11.054767 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:11.055385 26406 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:11.055842 30972 consensus_queue.cc:1046] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:31:11.056960 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:11.057595 26401 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:11.047662 30739 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:11.059608 30739 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:11.059731 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:11.060153 30739 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:11.051354 26328 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:11.060729 30739 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.060246 26256 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:11.062731 30739 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:11.063839 30402 consensus_queue.cc:1046] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:11.064940 30943 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:11.065114 30402 raft_consensus.cc:3032] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:11.065575 30402 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:11.066453 30402 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:11.068337 31086 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:11.069181 27182 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260924 14:31:11.071089 26332 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:11.052424 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.003134 31106 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.080015 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.002247 31099 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:10.966854 30903 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:11.078845 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:11.079061 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50251.3us
I20260924 14:31:11.084892 26257 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 5 FOLLOWER]: Advancing to term 7
I20260924 14:31:11.085327 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.086843 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.080673 26404 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:31:11.089452 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.090848 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.091943 26249 raft_consensus.cc:2396] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260924 14:31:11.093190 26406 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:31:11.093979 26328 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:11.082223 26325 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:11.095557 26251 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:11.083597 26401 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:11.080353 27126 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:11.090173 26404 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:11.099119 27126 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.101344 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.102097 26256 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:11.102142 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.101470 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.063235 30739 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:11.106077 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:11.107371 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.107899 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.108506 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:10.991367 31034 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.104732 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.057046 30972 raft_consensus.cc:3032] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:31:11.115053 27126 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.081423 31104 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:11.070187 31094 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:11.139979 26251 raft_consensus.cc:2354] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
I20260924 14:31:11.124547 26249 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 6: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:11.155980 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.088339 30842 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:31:11.170668 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:11.180289 26256 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:11.181545 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.182308 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.182982 31068 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:11.183974 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.184619 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.184851 26253 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:11.185424 27191 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:11.186053 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.186430 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:11.119355 30739 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.189429 30739 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:11.191699 27125 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.171770 31074 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:11.193094 31074 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:11.193563 31074 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:11.194393 31074 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:11.182334 31034 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:11.197507 26403 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:11.198735 30765 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:11.103122 31100 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.199872 31099 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:11.200850 31100 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:11.203095 31100 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:11.101007 31086 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:11.110951 30402 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:11.119701 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.091811 31092 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:11.163272 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.120239 26400 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:11.113075 31066 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:31:11.159601 27126 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.168530 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.221108 27190 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:11.096701 31023 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.087670 31103 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:11.156415 30990 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.182334 26256 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:11.213376 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:11.224879 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65376.8us
I20260924 14:31:11.232681 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:11.232967 27126 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.119947 30972 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:11.100841 30983 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:11.106150 31018 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:11.260617 31018 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:11.261204 31018 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:11.218406 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.264034 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.076344 31096 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:11.265100 26256 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 12
I20260924 14:31:11.158067 31024 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:11.267771 30842 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:11.268023 31024 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:11.268872 31060 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:11.269361 31060 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:11.269726 31060 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:11.269170 31024 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:11.161808 31004 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:31:11.270363 31004 raft_consensus.cc:686] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:11.159902 31091 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:11.236224 31103 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.217882 31037 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:11.230612 31023 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:11.207247 31111 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:31:11.250226 30990 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:11.226840 31064 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260924 14:31:11.268448 30842 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:11.208005 31104 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.264111 26254 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:11.336778 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.337703 26254 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:11.235076 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:11.271773 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 231004us
I20260924 14:31:11.339974 27190 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:11.341377 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.342931 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.343433 30990 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:11.343981 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:11.344341 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:11.119971 31071 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:11.344924 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:11.346184 30971 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:11.211669 26250 raft_consensus.cc:2362] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14. Re-sending same reply.
I20260924 14:31:11.347898 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.349141 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.348950 30972 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:11.218851 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.350641 26251 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:11.350966 30850 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:11.223183 31108 log.cc:924] Time spent T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Append to log took a long time: real 0.232s	user 0.001s	sys 0.001s
W20260924 14:31:11.295946 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 138006us
W20260924 14:31:11.285561 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 325828us
W20260924 14:31:11.352625 26371 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.660s (SENT)
I20260924 14:31:11.353076 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:11.353564 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:11.354620 26371 leader_election.cc:334] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.806s (SENT)
I20260924 14:31:11.354949 26297 leader_election.cc:398] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
I20260924 14:31:11.355229 26297 leader_election.cc:401] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260924 14:31:11.301663 26257 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:31:11.223242 31073 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.356410 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:11.296887 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 63992.9us
I20260924 14:31:11.357041 26257 raft_consensus.cc:2362] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10. Re-sending same reply.
I20260924 14:31:11.357859 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:11.302090 31095 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:11.358666 30739 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:11.293638 26256 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:11.274801 31094 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:11.267109 31096 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:11.295291 31117 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260924 14:31:11.288163 31091 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:11.221410 31066 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:11.212556 26403 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260924 14:31:11.360298 31066 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:11.303793 30983 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:11.356124 31073 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.364223 30983 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:11.365115 31079 consensus_meta.cc:287] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.004s
I20260924 14:31:11.365790 31079 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:11.366242 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1650) took 1624 ms. Trace:
I20260924 14:31:11.256032 31092 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:11.097175 31106 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:11.366621 26402 rpcz_store.cc:276] 0924 14:31:09.741867 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:09.755462 (+ 13595us) service_pool.cc:227] Handling call
0924 14:31:09.781590 (+ 26128us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:09.863855 (+ 82265us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:09.863894 (+    39us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:09.863940 (+    46us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:09.863977 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:11.366007 (+1502030us) spinlock_profiling.cc:243] Waited 82 ms on lock 0x7b5400336654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:11.366055 (+    48us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:11.366167 (+   112us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":188618624}
I20260924 14:31:11.368994 31106 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:11.369279 31078 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:11.370136 31078 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:11.370620 31078 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:11.300618 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:11.345214 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 845) took 1581 ms. Trace:
I20260924 14:31:11.372574 27189 rpcz_store.cc:276] 0924 14:31:09.763693 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:09.763840 (+   147us) service_pool.cc:227] Handling call
0924 14:31:09.781708 (+ 17868us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:11.070894 (+1289186us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:11.344949 (+274055us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400326254. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:11.345158 (+   209us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2894065920}
I20260924 14:31:11.375250 31078 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:11.377733 31104 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:11.299263 31077 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.381107 31077 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:11.382493 31041 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:11.357964 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:11.384853 31114 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260924 14:31:11.385109 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.388787 31103 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:11.359894 31091 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:11.365206 31073 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:11.391878 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:11.393307 31120 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:11.393759 31120 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:11.394114 31120 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:11.346236 26331 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:11.396512 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.345368 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:11.351022 30972 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:11.399720 30971 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:31:11.363286 27186 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:11.400254 31035 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:11.350190 26250 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:31:11.399266 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53983.7us
I20260924 14:31:11.409291 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:11.345160 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:11.345048 26325 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:11.272697 31100 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:31:11.408465 26371 leader_election.cc:334] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.055s (SENT)
I20260924 14:31:11.411147 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:11.398391 31031 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:11.416258 26331 raft_consensus.cc:2396] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260924 14:31:11.302321 31043 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:11.419566 26251 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260924 14:31:11.305204 31033 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260924 14:31:11.423272 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 78502.5us
W20260924 14:31:11.430878 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 73093.4us
I20260924 14:31:11.436197 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.436439 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:11.271297 31004 consensus_queue.cc:237] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:11.438014 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.441334 26398 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:31:11.451018 26254 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:31:11.453956 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:11.454908 26403 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:11.437232 26404 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:11.438887 26406 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:11.433251 26325 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:11.446206 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.412820 30842 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:11.383255 31077 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:11.465428 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 110594us
I20260924 14:31:11.476994 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:11.427907 30739 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:11.422253 31038 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:11.478463 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:11.466605 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.453625 30903 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:11.424998 31033 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:11.427541 30971 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:31:11.487618 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 132137us
I20260924 14:31:11.461889 31095 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:11.443593 31121 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:11.304333 31037 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.431802 31040 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260924 14:31:11.466142 31094 raft_consensus.cc:686] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:11.552249 26404 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260924 14:31:11.380405 31054 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:11.487413 26399 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:11.541656 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.542152 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.548066 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.493498 31122 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:11.558753 27190 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:11.491703 26403 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
W20260924 14:31:11.565778 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 87411.8us
I20260924 14:31:11.566623 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:11.568085 26297 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:11.569440 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:11.462267 30961 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
W20260924 14:31:11.550791 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 141576us
I20260924 14:31:11.563946 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:11.543942 31117 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260924 14:31:11.431221 31043 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:11.574128 26399 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:11.600410 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:11.576386 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:11.578190 26295 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:11.497038 26371 leader_election.cc:334] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.098s (SENT)
I20260924 14:31:11.468196 30940 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.603976 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.604890 26332 raft_consensus.cc:2379] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 8: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:11.604736 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.605408 26330 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:11.604519 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.609347 26330 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:11.612025 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.612857 26329 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:11.614460 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.614850 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.615120 26326 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:11.555222 31104 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:31:11.616510 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.542963 31023 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:11.631976 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 89885.7us
W20260924 14:31:11.621627 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 43552.7us
I20260924 14:31:11.637866 26369 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.629406 27191 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:11.554010 31106 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:11.630916 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:11.554940 31080 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.574844 31100 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.499313 30741 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:11.658026 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 110056us
I20260924 14:31:11.659075 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.588742 31111 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:11.588290 31079 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:11.643822 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:11.665688 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.574530 31118 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:11.540585 31060 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:11.494051 30688 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.630686 30940 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:11.658089 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 81815.8us
I20260924 14:31:11.646575 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.689612 26324 raft_consensus.cc:2396] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260924 14:31:11.661903 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 165009us
I20260924 14:31:11.632009 30946 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:11.688454 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:11.635661 31122 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:11.633256 31083 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:11.631273 31095 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:11.496853 31033 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:11.651691 31058 consensus_queue.cc:1046] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260924 14:31:11.671576 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.677889 27181 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:11.679762 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.667143 31038 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:11.657374 30972 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:11.684369 31004 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:11.699437 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.691213 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.667246 31111 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:11.707033 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:11.708014 27190 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:11.711191 26258 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:11.665485 31079 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:11.728363 26330 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:11.729378 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.730216 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.700354 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:11.680491 31082 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:11.653545 31106 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.657048 31080 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:11.695784 30402 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
W20260924 14:31:11.822475 31124 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260924 14:31:11.825349 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.826270 26329 raft_consensus.cc:2379] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:11.826853 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.827029 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.828054 26329 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:11.828526 31060 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:11.829010 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:11.829595 31054 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:11.829733 26329 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 12
I20260924 14:31:11.830129 31043 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:11.831904 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.832588 26326 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:11.833346 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 846) took 1676 ms. Trace:
I20260924 14:31:11.833482 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:11.833786 27182 rpcz_store.cc:276] 0924 14:31:10.157468 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:10.157622 (+   154us) service_pool.cc:227] Handling call
0924 14:31:10.194340 (+ 36718us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:11.069641 (+875301us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:11.833106 (+763465us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b540000c654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:11.833293 (+   187us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2011355008}
I20260924 14:31:11.834419 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.835005 26327 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:11.835067 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.659920 31100 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:11.793112 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48930.6us
I20260924 14:31:11.674139 31077 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260924 14:31:11.849920 26253 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:11.845733 26329 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:11.714668 31083 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:11.835932 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.836081 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.861781 26327 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:11.861987 26330 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:11.728910 31099 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:11.878917 31099 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.880138 31100 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:11.791029 30977 consensus_meta.cc:287] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.792s	user 0.006s	sys 0.014s
I20260924 14:31:11.837450 31080 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:11.838575 31107 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:11.737818 30971 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.900228 26251 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:11.902400 26258 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260924 14:31:11.827219 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.706642 31040 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260924 14:31:11.740255 31023 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:11.802382 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 95417.8us
I20260924 14:31:11.743839 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:11.851840 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:11.909005 26249 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:11.706897 31122 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:11.915331 26297 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.695s (SENT)
I20260924 14:31:11.915558 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:11.852181 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.859146 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1474) took 1061 ms. Trace:
I20260924 14:31:11.915984 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:11.686305 31094 consensus_queue.cc:237] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:11.876179 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.878180 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.889995 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:11.794677 30274 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:11.895160 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:11.926424 26327 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:11.792567 31079 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:11.706624 31032 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.140s
I20260924 14:31:11.802749 31082 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:11.736683 30963 consensus_queue.cc:1046] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.145s
I20260924 14:31:11.792639 30972 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:11.755043 31111 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:11.922788 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:11.923400 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 179659us
I20260924 14:31:11.869292 31038 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260924 14:31:11.924736 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:11.966135 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:11.922940 31041 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.891353 31099 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:11.914283 31023 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:11.841410 30940 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:11.859606 30402 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:11.884761 31083 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:11.738165 31078 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.161s
I20260924 14:31:11.938495 27123 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:12.055063 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 116669us
I20260924 14:31:11.741097 31103 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.172s
I20260924 14:31:11.916570 26397 rpcz_store.cc:276] 0924 14:31:10.797796 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:10.797952 (+   156us) service_pool.cc:227] Handling call
0924 14:31:10.812034 (+ 14082us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:10.812282 (+   248us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:10.812623 (+   341us) log.cc:842] Serialized 14 byte log entry
0924 14:31:11.464666 (+652043us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b1800489880. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd557b 00007f09b935888a
0924 14:31:11.858776 (+394110us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:11.858844 (+    68us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:11.858892 (+    48us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:11.858967 (+    75us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:11.859075 (+   108us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26042,"prepare.queue_time_us":3929,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":140177,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":10801,"spinlock_wait_cycles":4845312,"thread_start_us":367739,"threads_started":2}
I20260924 14:31:11.942425 27190 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:12.063795 27190 raft_consensus.cc:595] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:12.064157 27190 raft_consensus.cc:603] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:12.011432 31040 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:11.736879 31004 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:11.922641 31118 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.956771 26249 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:12.080941 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:12.082300 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:11.960909 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:11.850594 30946 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:11.967329 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:11.984002 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:11.921947 30971 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:11.985144 31106 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:11.986334 30787 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:11.987674 30961 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260924 14:31:11.987974 31066 consensus_meta.cc:287] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.000s
I20260924 14:31:11.989071 30688 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:11.930665 30974 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:11.923164 31122 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:11.908636 31080 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:11.993870 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:12.018755 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 304839us
I20260924 14:31:11.941180 31119 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:12.118932 26404 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:31:11.948062 31082 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:12.025159 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 57476.5us
I20260924 14:31:12.096798 31106 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:31:12.035516 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 71278.1us
I20260924 14:31:12.100023 30946 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:12.126398 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:11.956418 31077 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:12.047091 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:12.048341 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:12.048906 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:12.050184 26251 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260924 14:31:12.132066 26406 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:11.970239 31037 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:12.022644 30793 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:31:11.941609 31074 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260924 14:31:11.930589 30975 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260924 14:31:12.057909 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:11.908303 31005 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:12.060969 26401 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:12.070899 26221 leader_election.cc:398] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:31:12.147655 26221 leader_election.cc:401] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:12.083866 27182 raft_consensus.cc:2354] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15.
I20260924 14:31:12.084520 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:12.079329 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:12.062407 31123 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.406s	user 0.000s	sys 0.001s
W20260924 14:31:12.156437 31120 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.762s	user 0.005s	sys 0.009s
I20260924 14:31:12.156895 31120 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.159092 31120 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:11.945767 31095 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:12.162760 31018 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.001s
I20260924 14:31:12.178025 31018 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.083380 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:12.178654 30842 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.692s	user 0.003s	sys 0.005s
I20260924 14:31:12.063069 31034 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:12.179708 31034 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:12.180120 31034 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:12.181646 26403 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:12.181526 31018 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:12.182156 26403 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:12.026130 31041 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.184073 26329 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:12.184924 31041 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:12.185542 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:12.186244 26329 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:12.187206 26402 raft_consensus.cc:2354] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:12.024642 30983 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260924 14:31:12.054009 30741 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:12.102458 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 135197us
I20260924 14:31:12.076629 31128 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:12.087606 31040 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:12.103708 31126 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260924 14:31:11.948220 30765 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:12.119096 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:12.202740 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:12.206175 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 87203.2us
I20260924 14:31:12.206604 26328 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:12.182873 26403 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:12.148521 31129 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.086s	user 0.001s	sys 0.001s
I20260924 14:31:11.952008 31100 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:12.124610 31109 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260924 14:31:12.128170 31077 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:12.143636 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 85826.2us
I20260924 14:31:12.222663 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:12.145605 26369 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:12.148402 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 86596.6us
W20260924 14:31:12.225647 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 80113.8us
I20260924 14:31:12.148078 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:12.227417 26328 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:12.096485 30787 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.125068 31092 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:12.233062 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 849) took 1301 ms. Trace:
W20260924 14:31:12.153313 31139 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260924 14:31:12.153858 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:12.107522 31066 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:12.243271 27181 rpcz_store.cc:276] 0924 14:31:10.932067 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:10.940968 (+  8901us) service_pool.cc:227] Handling call
0924 14:31:12.232857 (+1291889us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b540073df54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:12.233007 (+   150us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1512181120}
I20260924 14:31:12.088349 31118 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.154357 31149 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7.
I20260924 14:31:12.154516 30688 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
W20260924 14:31:12.155225 31131 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260924 14:31:12.207201 31128 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:12.244372 30765 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:12.287415 30765 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:12.288328 31040 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:12.289618 30765 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:12.155964 31130 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260924 14:31:11.984705 30964 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:12.159605 26323 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:12.160408 26295 leader_election.cc:398] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15.
I20260924 14:31:12.162662 30971 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:12.095403 30972 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:12.291419 30964 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.292889 31066 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:12.293610 31147 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:12.293891 30964 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:12.168721 31107 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:12.294245 31147 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:12.294678 31147 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:12.129184 30402 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.295558 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:12.245651 31039 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:12.155725 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:12.144668 31140 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260924 14:31:12.245427 30741 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:12.297636 30741 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:12.298293 27194 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:12.298542 27186 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: []
I20260924 14:31:12.291742 26295 leader_election.cc:401] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260924 14:31:12.299275 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 139008us
I20260924 14:31:12.299751 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 806) took 1116 ms. Trace:
I20260924 14:31:12.300110 27186 rpcz_store.cc:276] 0924 14:31:11.183025 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.401238 (+218213us) service_pool.cc:227] Handling call
0924 14:31:11.401561 (+   323us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:12.298979 (+897418us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:12.299546 (+   567us) spinlock_profiling.cc:243] Waited 897 ms on lock 0x7b5400071a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:12.299701 (+   155us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2062381312}
I20260924 14:31:12.174403 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1155) took 1003 ms. Trace:
I20260924 14:31:12.144315 31103 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:12.301200 26256 rpcz_store.cc:276] 0924 14:31:11.170722 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.361564 (+190842us) service_pool.cc:227] Handling call
0924 14:31:11.362201 (+   637us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:11.363187 (+   986us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:11.363219 (+    32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:11.466891 (+103672us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b2c001c3658. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:11.475982 (+  9091us) log.cc:842] Serialized 26 byte log entry
0924 14:31:11.711308 (+235326us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:11.746146 (+ 34838us) log.cc:842] Serialized 14 byte log entry
0924 14:31:11.802530 (+ 56384us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:11.802612 (+    82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:12.174086 (+371474us) raft_consensus.cc:1671] finished
0924 14:31:12.174194 (+   108us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:12.174337 (+   143us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":71041,"prepare.queue_time_us":78896,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":103583,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":27716,"spinlock_wait_cycles":55759616,"thread_start_us":160132,"threads_started":1,"wal-append.queue_time_us":646310}
I20260924 14:31:12.302496 31103 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:12.302950 31103 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:12.175169 31035 consensus_meta.cc:287] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.774s	user 0.012s	sys 0.000s
I20260924 14:31:12.178416 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1539) took 1074 ms. Trace:
I20260924 14:31:12.303777 29682 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:12.303871 26253 rpcz_store.cc:276] 0924 14:31:11.104211 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.185727 (+ 81516us) service_pool.cc:227] Handling call
0924 14:31:12.178243 (+992516us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54002882d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:12.178359 (+   116us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":480035200}
I20260924 14:31:12.304495 31120 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 15
I20260924 14:31:12.233501 31077 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:12.305091 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:12.305728 26253 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:12.306545 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:12.071947 31071 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:12.268700 30940 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260924 14:31:12.307730 31071 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:12.309825 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:12.310547 26400 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:12.191493 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:12.312357 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:12.187575 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:12.313513 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:12.313918 26329 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:12.190536 26255 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260924 14:31:12.314977 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 807) took 1014 ms. Trace:
I20260924 14:31:12.316088 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:12.300123 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:12.316923 27181 raft_consensus.cc:2396] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260924 14:31:12.187646 31005 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:12.202663 31091 consensus_meta.cc:287] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.812s	user 0.009s	sys 0.001s
I20260924 14:31:12.203550 31079 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:12.179165 30842 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:12.147222 31134 log.cc:924] Time spent T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260924 14:31:12.209080 26403 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:12.101389 30961 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:12.147047 31088 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:12.168469 31023 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:12.298296 31140 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:12.190404 30274 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:12.360487 30274 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:12.213033 26406 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:12.191633 31097 raft_consensus.cc:3032] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Stepping down as leader of term 5
I20260924 14:31:12.363935 31097 raft_consensus.cc:729] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:12.364874 31097 consensus_queue.cc:260] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:12.192063 31068 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:12.150986 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:12.152725 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 68304.2us
I20260924 14:31:12.233259 27189 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260924 14:31:12.353497 31140 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:12.235112 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:12.335392 30961 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:12.332916 30741 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:12.140599 31024 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260924 14:31:12.211696 31078 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:12.272778 27190 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:12.323881 30688 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:12.275041 31092 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:12.276717 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:12.286811 31128 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:12.163247 30946 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:12.304286 29682 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:12.177114 31104 consensus_meta.cc:287] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.003s
I20260924 14:31:12.080422 31004 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:12.315074 26329 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:12.315465 27192 rpcz_store.cc:276] 0924 14:31:11.300339 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.464623 (+164284us) service_pool.cc:227] Handling call
0924 14:31:11.465087 (+   464us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:11.465391 (+   304us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:11.465428 (+    37us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:11.485864 (+ 20436us) log.cc:842] Serialized 26 byte log entry
0924 14:31:11.819804 (+333940us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:11.819856 (+    52us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:11.819962 (+   106us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:12.314783 (+494821us) raft_consensus.cc:1671] finished
0924 14:31:12.314803 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:12.314924 (+   121us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":231177,"threads_started":1,"wal-append.queue_time_us":779282}
I20260924 14:31:12.195860 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:12.187356 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1621) took 1259 ms. Trace:
I20260924 14:31:12.317023 26257 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:12.319139 27186 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:12.213049 31100 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:12.320122 26399 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260924 14:31:12.321280 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:12.303337 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:12.433339 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:12.367497 31083 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:12.327071 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:12.331493 26327 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:12.297147 31039 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:12.337553 27185 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:31:12.123708 31080 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:12.293699 30787 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:12.344666 26250 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:12.348101 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:12.331070 30739 consensus_meta.cc:287] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.835s	user 0.006s	sys 0.006s
I20260924 14:31:12.356710 31122 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:12.359601 31032 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:12.366391 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:12.369333 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 134302us
I20260924 14:31:12.370800 26251 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:31:12.375231 27190 raft_consensus.cc:595] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:12.386582 26254 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:12.349259 27182 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:12.381366 26403 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
W20260924 14:31:12.367374 31135 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.532s	user 0.001s	sys 0.004s
I20260924 14:31:12.324051 31018 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260924 14:31:12.423965 26252 rpcz_store.cc:276] 0924 14:31:10.927882 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:10.957373 (+ 29491us) service_pool.cc:227] Handling call
0924 14:31:12.187147 (+1229774us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54007828d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:12.187275 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2633792896}
I20260924 14:31:12.368973 31068 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:12.515141 30739 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11.
I20260924 14:31:12.435636 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:12.516429 30741 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260924 14:31:12.516556 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 81049.9us
I20260924 14:31:12.516738 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:12.517572 31152 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:12.517632 27125 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:12.518154 31152 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:12.518252 30741 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:12.518821 30741 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:12.518893 31068 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:12.519362 30795 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:31:12.519090 31152 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.523124 26330 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:12.523849 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:12.524958 31122 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:12.519845 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 171870us
I20260924 14:31:12.553339 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:12.526175 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:12.554217 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:12.515233 31039 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:12.434854 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:12.561491 26256 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:12.561635 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:12.525471 31032 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:12.525370 27182 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "64e0b4058f18400dab940d3832586746"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:12.571796 27182 raft_consensus.cc:595] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:12.571996 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 376190us
I20260924 14:31:12.572510 26400 raft_consensus.cc:2354] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15.
W20260924 14:31:12.574854 26297 leader_election.cc:334] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.215s (SENT)
I20260924 14:31:12.576200 27188 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:12.544600 26326 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:12.555802 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:12.577041 27123 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:12.556855 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:12.578220 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:12.557931 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:12.376839 31071 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:12.374967 31078 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:12.597383 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 276179us
I20260924 14:31:12.598197 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:12.370227 31023 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:12.539952 26403 pending_rounds.cc:76] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 6
I20260924 14:31:12.599015 26403 pending_rounds.cc:97] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260924 14:31:12.599295 26403 raft_consensus.cc:2866] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:31:12.420223 31058 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.004s	sys 0.011s
W20260924 14:31:12.600383 31058 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.004s	sys 0.011s
I20260924 14:31:12.324610 31091 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.602326 31091 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:12.602568 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:12.602815 31091 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:12.577539 26406 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:31:12.379539 31079 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:12.604135 30077 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:12.592312 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:12.604602 30077 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:12.419217 31097 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 5 FOLLOWER]: Advancing to term 6
I20260924 14:31:12.604993 30077 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:12.383332 31120 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15.
I20260924 14:31:12.597414 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:12.597440 26257 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:12.609596 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:12.611864 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:12.612804 31058 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:12.613256 31058 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:12.613620 31058 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:12.603284 31091 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:12.620970 31091 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:12.624085 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:12.629891 26329 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:12.630333 27181 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:12.519390 30741 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:12.430632 31104 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260924 14:31:12.639372 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 64547.3us
W20260924 14:31:12.595306 31033 consensus_meta.cc:287] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.855s	user 0.009s	sys 0.003s
I20260924 14:31:12.420701 30765 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.206s
I20260924 14:31:12.561312 31148 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:12.646098 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:12.637938 26298 leader_election.cc:398] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15.
I20260924 14:31:12.625458 31078 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:12.727972 26331 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:31:12.636341 31107 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260924 14:31:12.380376 30688 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:12.663847 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 436195us
I20260924 14:31:12.625258 31023 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:12.593302 31066 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:12.596591 30787 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:31:12.596777 30976 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:12.624222 31121 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:12.748301 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:12.639822 31122 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:12.742240 31104 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:12.624089 31060 consensus_queue.cc:1046] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.121s
I20260924 14:31:12.772192 27181 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:12.782126 26369 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:12.773149 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 141015us
I20260924 14:31:12.783962 26295 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:12.784142 26298 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:12.785949 31105 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:12.786855 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:12.754662 31023 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:12.632642 30861 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
I20260924 14:31:12.786737 31105 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:12.796717 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:12.797405 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:12.808300 27185 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 851) took 1862 ms (client timeout 1908 ms). Trace:
W20260924 14:31:12.866123 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 82074.2us
I20260924 14:31:12.867143 26298 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:12.816210 31092 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:12.816429 29682 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:12.817898 31121 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
W20260924 14:31:12.820398 31138 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.710s	user 0.000s	sys 0.001s
I20260924 14:31:12.882764 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:12.820520 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:12.884598 26398 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:12.886176 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 971) took 1047 ms. Trace:
I20260924 14:31:12.886682 26330 rpcz_store.cc:276] 0924 14:31:11.838609 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.877863 (+ 39254us) service_pool.cc:227] Handling call
0924 14:31:12.885959 (+1008096us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54007800d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:12.886111 (+   152us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1371746176}
I20260924 14:31:12.888705 26330 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260924 14:31:12.889369 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 975) took 1049 ms. Trace:
I20260924 14:31:12.889729 26330 rpcz_store.cc:276] 0924 14:31:11.839802 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.887878 (+1048076us) service_pool.cc:227] Handling call
0924 14:31:12.888296 (+   418us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:12.889092 (+   796us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:12.889313 (+   221us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:12.799315 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:12.830847 30793 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:12.891633 30793 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:31:12.831867 27188 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 797) took 2086 ms (client timeout 1990 ms). Trace:
W20260924 14:31:12.899811 27188 rpcz_store.cc:269] 0924 14:31:10.746060 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:10.746729 (+   669us) service_pool.cc:227] Handling call
0924 14:31:12.831690 (+2084961us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400155c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:12.831797 (+   107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3679458048}
I20260924 14:31:12.836437 31083 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:12.837134 31160 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:12.780300 31035 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:12.636062 31150 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.158s
W20260924 14:31:12.863926 27185 rpcz_store.cc:269] 0924 14:31:10.945616 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:10.945784 (+   168us) service_pool.cc:227] Handling call
0924 14:31:12.808073 (+1862289us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400074c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:12.808217 (+   144us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3027298560}
I20260924 14:31:12.731689 31033 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.870327 26406 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:31:12.882386 26324 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.015s	sys 0.070s
I20260924 14:31:12.902410 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:12.926890 26324 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.015s	sys 0.071s
I20260924 14:31:12.904084 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:12.844564 31066 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:12.775137 31107 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.106s
I20260924 14:31:12.949440 31032 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:12.958808 31034 consensus_meta.cc:287] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.778s	user 0.008s	sys 0.000s
I20260924 14:31:12.864310 30971 raft_consensus.cc:3032] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Stepping down as leader of term 6
I20260924 14:31:12.923511 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:12.779867 30972 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260924 14:31:12.914765 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:13.000186 26327 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:12.757586 31148 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:12.883466 30787 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:31:12.946463 26250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1635) took 1535 ms (client timeout 1976 ms). Trace:
W20260924 14:31:12.947124 31103 consensus_meta.cc:287] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.000s
I20260924 14:31:12.926095 31035 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:12.905440 31149 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:12.965078 31033 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:12.965453 31082 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.832s	user 0.010s	sys 0.000s
I20260924 14:31:12.926786 31159 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:12.928329 31118 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:12.793715 30077 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:12.926003 31121 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:12.743438 31100 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:12.919685 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:13.103629 27188 raft_consensus.cc:595] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:13.104009 27188 raft_consensus.cc:603] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:12.882025 30983 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:12.901937 30842 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:12.906612 31122 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:12.820446 30274 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:12.936348 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:13.004506 26401 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:12.806979 31140 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:13.010535 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:13.006237 26250 rpcz_store.cc:269] 0924 14:31:11.410643 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.410925 (+   282us) service_pool.cc:227] Handling call
0924 14:31:12.946229 (+1535304us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b54000c21d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:12.946395 (+   166us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2032566016}
W20260924 14:31:13.064200 26258 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.015s	sys 0.061s
W20260924 14:31:13.129079 26258 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.015s	sys 0.061s
I20260924 14:31:13.096063 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1513) took 1523 ms. Trace:
W20260924 14:31:13.011502 31147 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.001s
W20260924 14:31:13.006923 31111 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.017s	user 0.007s	sys 0.004s
I20260924 14:31:12.993166 30971 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:13.105290 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 493849us
I20260924 14:31:13.200212 27190 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:12.955025 31039 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:13.003863 31032 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:13.117249 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.012s	sys 0.010s
I20260924 14:31:13.122341 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1517) took 1518 ms. Trace:
W20260924 14:31:13.204022 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.012s	sys 0.010s
I20260924 14:31:13.129977 26325 rpcz_store.cc:276] 0924 14:31:11.572223 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.572427 (+   204us) service_pool.cc:227] Handling call
0924 14:31:11.572877 (+   450us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:11.589640 (+ 16763us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:11.589671 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:11.590122 (+   451us) log.cc:842] Serialized 26 byte log entry
0924 14:31:11.763669 (+173547us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:11.763727 (+    58us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:11.763844 (+   117us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:12.819840 (+1055996us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:13.063151 (+243311us) raft_consensus.cc:1671] finished
0924 14:31:13.063184 (+    33us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:13.095984 (+ 32800us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":8422,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":16000,"spinlock_wait_cycles":6517504,"thread_start_us":117369,"threads_started":1,"wal-append.queue_time_us":431284}
I20260924 14:31:13.096370 31085 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:13.098461 31035 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:13.097987 31155 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:13.004074 31034 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:13.204406 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 610663us
W20260924 14:31:13.206076 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 560004us
I20260924 14:31:13.062636 31103 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:13.115154 31080 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:13.227679 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:13.262107 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:13.280174 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:13.204156 26323 rpcz_store.cc:276] 0924 14:31:11.604022 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.604190 (+   168us) service_pool.cc:227] Handling call
0924 14:31:13.122104 (+1517914us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54002774d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:13.122239 (+   135us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1272792448}
I20260924 14:31:13.263875 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:12.992751 31066 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:13.123224 31122 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:13.096294 31082 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:13.267263 30990 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.010s	sys 0.027s
W20260924 14:31:13.294364 26254 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.009s	sys 0.013s
W20260924 14:31:13.294734 26254 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.009s	sys 0.013s
W20260924 14:31:13.264775 31088 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.889s	user 0.009s	sys 0.001s
I20260924 14:31:13.222304 31033 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:13.220723 30765 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:13.222594 31024 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:13.300127 26325 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:13.296873 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:13.300781 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:13.308665 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.042s	sys 0.099s
W20260924 14:31:13.392912 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.042s	sys 0.099s
W20260924 14:31:13.393318 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 768146us
I20260924 14:31:13.224821 31041 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:13.291364 31058 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.005s
I20260924 14:31:13.104974 31149 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:13.302412 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.012s	sys 0.008s
I20260924 14:31:13.310442 27186 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:13.396378 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:13.474622 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.012s	sys 0.008s
I20260924 14:31:13.475126 26327 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:13.262964 31073 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.009s	sys 0.014s
I20260924 14:31:13.317085 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:13.409399 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:13.479978 27188 raft_consensus.cc:595] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
W20260924 14:31:13.331568 31097 consensus_meta.cc:287] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.000s
I20260924 14:31:13.284497 31091 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:13.412330 27191 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:13.400362 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1545) took 2293 ms. Trace:
I20260924 14:31:13.268815 31150 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:13.300347 30990 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.011s	sys 0.027s
W20260924 14:31:13.475649 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 897419us
W20260924 14:31:13.388046 31128 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.006s	sys 0.033s
I20260924 14:31:13.226264 30793 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:13.293624 31079 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:13.302573 31152 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:13.199772 31147 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:13.410670 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:13.229913 31032 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:13.262779 31085 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:13.493952 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.004s	sys 0.033s
W20260924 14:31:13.600278 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.005s	sys 0.034s
W20260924 14:31:13.600873 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 775327us
I20260924 14:31:13.601348 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:13.515133 26253 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.008s	sys 0.032s
W20260924 14:31:13.602136 26253 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.064s	user 0.009s	sys 0.032s
I20260924 14:31:13.200266 31111 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:13.316764 30964 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:13.486248 26327 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:13.520918 31083 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:13.483414 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 71503.3us
I20260924 14:31:13.295401 31122 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:13.278332 30842 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.200s
I20260924 14:31:13.308729 31118 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:13.310163 30739 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:13.405902 31077 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.101s	user 0.008s	sys 0.003s
I20260924 14:31:13.523880 26255 rpcz_store.cc:276] 0924 14:31:11.106897 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.359850 (+252953us) service_pool.cc:227] Handling call
0924 14:31:12.190182 (+830332us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:12.314570 (+124388us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:13.400089 (+1085519us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b54002882d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:13.400284 (+   195us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1908761344}
I20260924 14:31:13.308255 30787 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:13.554147 26406 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:13.392397 31088 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:13.531374 26369 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.019s	sys 0.028s
W20260924 14:31:13.689291 26369 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.022s	sys 0.028s
W20260924 14:31:13.689646 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 706219us
W20260924 14:31:13.556665 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.009s	sys 0.031s
W20260924 14:31:13.692869 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.009s	sys 0.031s
W20260924 14:31:13.511003 31073 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.010s	sys 0.014s
I20260924 14:31:13.565864 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:13.566644 26402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1659) took 1840 ms (client timeout 1521 ms). Trace:
I20260924 14:31:13.587867 31080 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:13.573942 26252 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.016s	sys 0.024s
W20260924 14:31:13.521003 30946 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.015s	sys 0.033s
W20260924 14:31:13.583937 27192 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.017s	sys 0.043s
W20260924 14:31:13.734651 27192 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.017s	sys 0.044s
W20260924 14:31:13.596736 30963 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.011s	sys 0.033s
I20260924 14:31:13.300082 31158 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:13.756605 31158 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260924 14:31:13.743525 30963 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.012s	sys 0.033s
I20260924 14:31:13.757711 30985 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:31:13.758287 30985 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:13.612989 27123 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:13.760471 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 147543us
I20260924 14:31:13.761796 27123 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:13.280555 31155 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:13.764950 31155 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:13.393529 31033 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:13.521557 31092 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:13.767784 31092 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:13.588361 30961 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.217s	user 0.007s	sys 0.003s
I20260924 14:31:13.771353 30961 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:13.699622 30787 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:13.631682 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:31:13.529279 31154 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.687s	user 0.000s	sys 0.001s
I20260924 14:31:13.775859 31039 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:13.776750 31039 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260924 14:31:13.778275 26297 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.935s (SENT) [suppressed 14 similar messages]
I20260924 14:31:13.778532 31039 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:13.513988 31109 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:13.566059 30637 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:13.779649 31109 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:13.780009 30637 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:13.782335 31109 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:13.396900 31024 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:13.664131 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 1313) took 1360 ms. Trace:
I20260924 14:31:13.783288 27190 rpcz_store.cc:276] 0924 14:31:12.303737 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.523549 (+219812us) service_pool.cc:227] Handling call
0924 14:31:13.663807 (+1140258us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b540000c154. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:13.664041 (+   234us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1499546880}
I20260924 14:31:13.784991 31024 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:13.786912 27190 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:13.471263 31041 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:13.788748 31170 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.501s	user 0.000s	sys 0.001s
I20260924 14:31:13.789858 31041 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:13.658668 31156 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.745s	user 0.005s	sys 0.002s
I20260924 14:31:13.679762 27185 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260924 14:31:13.672559 31120 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:13.793817 31120 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:13.793905 27185 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260924 14:31:13.679819 26332 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:13.794296 31120 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:13.669399 30971 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:13.600083 31152 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:13.797086 31152 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:13.797163 30971 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
W20260924 14:31:13.601665 31094 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.070s	sys 0.132s
W20260924 14:31:13.799733 31094 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.071s	sys 0.132s
I20260924 14:31:13.726289 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:13.676918 31005 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:13.800395 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 74191.4us
I20260924 14:31:13.390336 31121 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:13.615533 31173 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:13.802903 31173 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:13.791061 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1499) took 1704 ms. Trace:
I20260924 14:31:13.805557 31173 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:13.731941 26252 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.017s	sys 0.033s
I20260924 14:31:13.806731 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:13.807639 26406 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:31:13.808025 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:13.808454 31033 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:13.808732 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:13.810294 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:13.811178 26406 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:13.813609 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 805) took 2691 ms. Trace:
I20260924 14:31:13.814127 27189 rpcz_store.cc:276] 0924 14:31:11.122599 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.373466 (+250867us) service_pool.cc:227] Handling call
0924 14:31:12.233000 (+859534us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:12.384497 (+151497us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:13.813325 (+1428828us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b540073df50. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:13.813512 (+   187us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1975803776}
I20260924 14:31:13.816594 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 958) took 1992 ms. Trace:
I20260924 14:31:13.817157 26328 rpcz_store.cc:276] 0924 14:31:11.824116 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.271753 (+447637us) service_pool.cc:227] Handling call
0924 14:31:12.276881 (+  5128us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:12.325455 (+ 48574us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:31:12.345180 (+ 19725us) log.cc:842] Serialized 14 byte log entry
0924 14:31:12.731100 (+385920us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:12.731170 (+    70us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:12.731233 (+    63us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:13.816288 (+1085055us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b54007cf6d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:13.816354 (+    66us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:13.816523 (+   169us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":20088,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":1289,"raft.queue_time_us":5804,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":15867,"spinlock_wait_cycles":111100160,"thread_start_us":384930,"threads_started":1,"wal-append.queue_time_us":573597}
I20260924 14:31:13.818945 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1506) took 1263 ms. Trace:
I20260924 14:31:13.820230 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 974) took 1980 ms. Trace:
W20260924 14:31:13.820729 31166 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.694s	user 0.000s	sys 0.001s
I20260924 14:31:13.822376 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1503) took 1513 ms. Trace:
I20260924 14:31:13.824625 29682 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:13.831070 31032 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:13.558543 31091 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:13.834451 31091 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:13.735975 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 857) took 2116 ms. Trace:
I20260924 14:31:13.835063 27191 rpcz_store.cc:276] 0924 14:31:11.619572 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.826570 (+206998us) service_pool.cc:227] Handling call
0924 14:31:13.735771 (+1909201us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b540014ffd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:13.735911 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3619342080}
W20260924 14:31:13.709262 31163 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.699s	user 0.000s	sys 0.002s
W20260924 14:31:13.729615 31167 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.409s	user 0.000s	sys 0.001s
I20260924 14:31:13.756883 31159 raft_consensus.cc:465] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Not starting pre-election -- already a leader
I20260924 14:31:13.777086 26325 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:13.699982 26402 rpcz_store.cc:269] 0924 14:31:11.690779 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.690950 (+   171us) service_pool.cc:227] Handling call
0924 14:31:13.530733 (+1839783us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54006e78d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:13.531095 (+   362us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1033560320}
I20260924 14:31:13.819521 26399 rpcz_store.cc:276] 0924 14:31:12.555091 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.555532 (+   441us) service_pool.cc:227] Handling call
0924 14:31:12.555910 (+   378us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:12.571645 (+ 15735us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:12.571678 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:12.640809 (+ 69131us) log.cc:842] Serialized 26 byte log entry
0924 14:31:13.127195 (+486386us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:31:13.127261 (+    66us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:13.127366 (+   105us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:13.818648 (+691282us) raft_consensus.cc:1671] finished
0924 14:31:13.818705 (+    57us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:13.818872 (+   167us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":377,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":33844,"spinlock_wait_cycles":42240,"thread_start_us":371219,"threads_started":1,"wal-append.queue_time_us":1152560}
I20260924 14:31:13.482285 30765 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:13.729802 30975 consensus_queue.cc:1046] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260924 14:31:13.622135 31079 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:13.395704 31105 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:13.733274 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:13.867614 26255 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:13.868053 27126 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:13.734474 30946 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.015s	sys 0.033s
W20260924 14:31:13.843165 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 81459.5us
I20260924 14:31:13.477787 31058 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:13.774139 30961 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:13.782680 30637 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:13.565366 31150 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:13.700994 31148 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:13.708124 31088 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:13.856472 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:13.859382 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:13.952826 26399 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:13.952872 26398 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:13.805624 26397 rpcz_store.cc:276] 0924 14:31:12.086334 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.086477 (+   143us) service_pool.cc:227] Handling call
0924 14:31:12.101119 (+ 14642us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:12.109042 (+  7923us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:12.109084 (+    42us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:12.109853 (+   769us) log.cc:842] Serialized 52 byte log entry
0924 14:31:12.386903 (+277050us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:12.926011 (+539108us) log.cc:842] Serialized 14 byte log entry
0924 14:31:12.926255 (+   244us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:12.926332 (+    77us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:13.790763 (+864431us) raft_consensus.cc:1671] finished
0924 14:31:13.790793 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:13.790983 (+   190us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":436618,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":11131,"raft.queue_time_us":187,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":41907,"thread_start_us":753385,"threads_started":2,"wal-append.queue_time_us":628420}
I20260924 14:31:13.865813 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:13.955612 27191 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:13.632513 30964 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:13.874915 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:13.958773 26371 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.033s	sys 0.072s
W20260924 14:31:13.960327 26371 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.033s	sys 0.072s
I20260924 14:31:13.961848 27188 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:31:13.957149 26254 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.048s	sys 0.064s
W20260924 14:31:13.965785 26254 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.048s	sys 0.064s
I20260924 14:31:13.876132 26402 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:31:13.967340 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:13.770434 31092 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:13.641024 31083 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:13.730103 30274 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:13.822831 26405 rpcz_store.cc:276] 0924 14:31:12.309026 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.317299 (+  8273us) service_pool.cc:227] Handling call
0924 14:31:12.317634 (+   335us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:12.317964 (+   330us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:12.317991 (+    27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:12.326151 (+  8160us) log.cc:842] Serialized 26 byte log entry
0924 14:31:12.764235 (+438084us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:12.764290 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:12.764358 (+    68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:13.822120 (+1057762us) raft_consensus.cc:1671] finished
0924 14:31:13.822149 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:13.822306 (+   157us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2781,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":7657,"thread_start_us":395187,"threads_started":1,"wal-append.queue_time_us":649460}
W20260924 14:31:13.922127 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 48157.3us
I20260924 14:31:13.680306 31118 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:13.932876 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 129910us
W20260924 14:31:13.929744 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 61812us
W20260924 14:31:13.932951 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 301315us
I20260924 14:31:14.008342 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:13.693089 30739 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:13.692565 31077 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:13.854761 30963 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:13.971336 31097 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:13.961366 31082 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:13.820745 26329 rpcz_store.cc:276] 0924 14:31:11.839507 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.679867 (+840360us) service_pool.cc:227] Handling call
0924 14:31:12.761191 (+ 81324us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:12.780561 (+ 19370us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:12.795063 (+ 14502us) log.cc:842] Serialized 14 byte log entry
0924 14:31:13.308927 (+513864us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:13.308981 (+    54us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:13.309032 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:13.819942 (+510910us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b5400237654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:13.820009 (+    67us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:13.820162 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":23451,"prepare.queue_time_us":8269,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":31994,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":30649,"spinlock_wait_cycles":43945728,"thread_start_us":490013,"threads_started":1}
I20260924 14:31:13.957125 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:13.974584 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 817) took 1447 ms. Trace:
I20260924 14:31:13.869611 30077 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:13.981282 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:14.101394 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:13.869110 30983 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:13.754958 31080 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:13.937101 30961 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:13.972997 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:13.943054 31039 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:14.010375 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:13.932502 31091 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:14.002650 26399 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:14.030617 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:14.030853 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:13.950781 31103 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:13.945356 30793 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:13.947630 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:13.959900 31034 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:13.919279 31079 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:14.199980 31079 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:14.200176 31071 consensus_queue.cc:1046] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:14.201472 26397 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:14.203613 31058 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:14.166725 30637 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
W20260924 14:31:14.011291 30741 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.343s	user 0.011s	sys 0.002s
I20260924 14:31:13.853152 30787 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:14.100355 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:14.255139 27188 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:14.143913 26403 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:14.109825 27189 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:14.102133 27186 rpcz_store.cc:276] 0924 14:31:12.527422 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.609400 (+ 81978us) service_pool.cc:227] Handling call
0924 14:31:12.609732 (+   332us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:13.474752 (+865020us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:13.974305 (+499553us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b540073c654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:13.974520 (+   215us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1590471936}
I20260924 14:31:14.259294 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:14.103093 26323 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:14.044584 31148 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:13.759354 30985 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:14.152019 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50501.1us
W20260924 14:31:14.153008 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 374748us
I20260924 14:31:14.155212 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 967) took 2328 ms. Trace:
I20260924 14:31:14.261550 26324 rpcz_store.cc:276] 0924 14:31:11.826913 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.332501 (+505588us) service_pool.cc:227] Handling call
0924 14:31:12.372940 (+ 40439us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:12.373295 (+   355us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:12.373324 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:12.373815 (+   491us) log.cc:842] Serialized 26 byte log entry
0924 14:31:12.942473 (+568658us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:12.942527 (+    54us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:12.942614 (+    87us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:14.155002 (+1212388us) raft_consensus.cc:1671] finished
0924 14:31:14.155023 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:14.155155 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1099,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":553113,"threads_started":1,"wal-append.queue_time_us":825959}
I20260924 14:31:14.263783 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:14.265327 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:14.039381 30963 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:14.102932 26328 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:14.272423 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:14.152565 29682 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:31:14.039077 31092 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:14.337682 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1514) took 2680 ms. Trace:
I20260924 14:31:14.102197 31155 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.099588 31195 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.166698 26406 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 6.
I20260924 14:31:14.100322 31149 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:14.138247 31109 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.099933 31184 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:14.191967 27185 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:31:14.145936 30077 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.200551 31079 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:13.973937 30974 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:14.193295 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:14.247747 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1164) took 2016 ms. Trace:
I20260924 14:31:14.155774 30274 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:14.289744 27186 raft_consensus.cc:2379] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:14.273224 26328 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:14.338177 26331 rpcz_store.cc:276] 0924 14:31:11.572611 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.572737 (+   126us) service_pool.cc:227] Handling call
0924 14:31:11.573047 (+   310us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:12.738572 (+1165525us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.253186 (+1514614us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540001efd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:14.253408 (+   222us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2515666816}
I20260924 14:31:14.130210 31097 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:14.186714 31039 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:14.170429 31041 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:14.241334 31040 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.842s	user 0.015s	sys 0.055s
I20260924 14:31:14.399529 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:14.396308 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1534) took 1101 ms. Trace:
I20260924 14:31:14.656673 26325 rpcz_store.cc:276] 0924 14:31:13.295100 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:13.474132 (+179032us) service_pool.cc:227] Handling call
0924 14:31:13.485662 (+ 11530us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:13.865234 (+379572us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.395937 (+530703us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b5400558cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:14.396228 (+   291us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":669617920}
I20260924 14:31:14.658031 26325 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:31:14.658805 26325 raft_consensus.cc:595] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:14.659210 26325 raft_consensus.cc:603] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:14.339174 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:14.410704 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:14.194119 31111 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:14.150604 30983 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:14.674301 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:14.427197 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 152399us
I20260924 14:31:14.676188 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:14.445182 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:14.677297 26324 raft_consensus.cc:2396] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260924 14:31:14.448540 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 170723us
I20260924 14:31:14.245523 31147 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:14.450294 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:14.273726 31128 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.028s	sys 0.051s
I20260924 14:31:14.521428 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1550) took 2460 ms. Trace:
I20260924 14:31:14.523463 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1521) took 2912 ms. Trace:
I20260924 14:31:14.344113 31103 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:14.394721 31058 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:14.707283 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1021) took 1004 ms. Trace:
W20260924 14:31:14.655433 31040 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.269s	user 0.016s	sys 0.055s
I20260924 14:31:14.639186 31039 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:14.227525 30961 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:14.639210 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1509) took 1242 ms. Trace:
W20260924 14:31:14.725942 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 47931.4us
I20260924 14:31:14.726016 26401 rpcz_store.cc:276] 0924 14:31:13.396877 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:13.397034 (+   157us) service_pool.cc:227] Handling call
0924 14:31:13.397378 (+   344us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:13.692447 (+295069us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:13.692478 (+    31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:13.692529 (+    51us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:13.692567 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.638875 (+946308us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54001c1954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:14.638947 (+    72us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:14.639134 (+   187us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":678044032}
I20260924 14:31:14.424226 31091 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:14.405560 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:14.633001 26252 rpcz_store.cc:276] 0924 14:31:12.230932 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.521362 (+290430us) service_pool.cc:227] Handling call
0924 14:31:12.521617 (+   255us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:12.521898 (+   281us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:12.940542 (+418644us) log.cc:842] Serialized 14 byte log entry
0924 14:31:14.246979 (+1306437us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:14.247096 (+   117us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:14.247209 (+   113us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.247349 (+   140us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:14.247630 (+   281us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":245846,"prepare.queue_time_us":756904,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":7907,"raft.queue_time_us":175869,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":8830,"thread_start_us":1585977,"threads_started":3,"wal-append.queue_time_us":860221}
W20260924 14:31:14.640201 26404 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.017s	sys 0.036s
W20260924 14:31:14.734685 26404 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.018s	sys 0.036s
I20260924 14:31:14.656183 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1556) took 1378 ms. Trace:
I20260924 14:31:14.735596 26255 rpcz_store.cc:276] 0924 14:31:13.277466 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.250194 (+972728us) service_pool.cc:227] Handling call
0924 14:31:14.656085 (+405891us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:14.341188 31179 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.423s
I20260924 14:31:14.419835 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:14.680363 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1535) took 1385 ms. Trace:
I20260924 14:31:14.424019 31071 raft_consensus.cc:3032] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:14.341576 31196 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:14.635924 27191 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.022s	sys 0.027s
W20260924 14:31:14.884064 27191 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.023s	sys 0.027s
I20260924 14:31:14.684471 26332 rpcz_store.cc:276] 0924 14:31:11.611217 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.611335 (+   118us) service_pool.cc:227] Handling call
0924 14:31:11.611637 (+   302us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:13.794797 (+2183160us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.523144 (+728347us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b5400498c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:14.523381 (+   237us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4755818368}
I20260924 14:31:14.425115 31085 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:14.447604 31206 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.672281 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:14.624397 30974 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:14.632418 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:14.707894 26324 rpcz_store.cc:276] 0924 14:31:13.702984 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.337308 (+634324us) service_pool.cc:227] Handling call
0924 14:31:14.707188 (+369880us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:14.709134 27125 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.017s	sys 0.029s
W20260924 14:31:14.897080 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.017s	sys 0.029s
W20260924 14:31:14.628643 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 435546us
I20260924 14:31:14.683071 26254 rpcz_store.cc:276] 0924 14:31:12.060679 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.516090 (+455411us) service_pool.cc:227] Handling call
0924 14:31:12.517598 (+  1508us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:12.555309 (+ 37711us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:13.295067 (+739758us) log.cc:842] Serialized 14 byte log entry
0924 14:31:14.030019 (+734952us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:14.030080 (+    61us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:14.030138 (+    58us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.521064 (+490926us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b5400288f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:14.521140 (+    76us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:14.521343 (+   203us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":63729,"prepare.queue_time_us":866364,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":183508,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":12203,"spinlock_wait_cycles":86122240,"thread_start_us":1368053,"threads_started":2,"wal-append.queue_time_us":1103324}
I20260924 14:31:14.444356 31148 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.520334 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:14.632477 31149 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.522044 31141 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.409538 31092 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:14.269074 30741 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:14.711107 26295 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.021s	sys 0.040s
I20260924 14:31:14.443269 31079 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:14.916775 31079 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:14.918037 31122 consensus_meta.cc:287] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.226s	user 0.010s	sys 0.000s
I20260924 14:31:14.903831 31148 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:14.918527 31122 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:14.903512 31080 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:14.708462 31128 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.028s	sys 0.052s
I20260924 14:31:14.667181 30765 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:14.795578 31188 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.458s	user 0.000s	sys 0.001s
I20260924 14:31:14.707695 30946 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.920718 31122 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:14.921068 30946 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:14.922833 30971 consensus_meta.cc:287] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.125s	user 0.013s	sys 0.005s
W20260924 14:31:14.818713 26369 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.053s	sys 0.042s
W20260924 14:31:14.923617 26369 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.054s	sys 0.047s
W20260924 14:31:14.923995 31068 consensus_meta.cc:287] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 2.404s	user 0.012s	sys 0.003s
I20260924 14:31:14.684121 31088 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:14.924414 31068 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:14.717043 30787 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:14.926473 31068 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:14.926703 27194 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:14.453586 30077 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:14.928886 30961 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:14.929183 30077 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:14.930078 31035 consensus_meta.cc:287] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.653s	user 0.010s	sys 0.001s
W20260924 14:31:14.810551 27188 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.000s
I20260924 14:31:14.930485 31035 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:14.930925 27188 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:14.931514 27181 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:14.932046 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:14.932551 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 810) took 2960 ms. Trace:
I20260924 14:31:14.932861 26254 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:14.932927 27194 rpcz_store.cc:276] 0924 14:31:11.972057 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.299046 (+326989us) service_pool.cc:227] Handling call
0924 14:31:12.299340 (+   294us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:14.927128 (+2627788us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.932332 (+  5204us) spinlock_profiling.cc:243] Waited 2.63 s on lock 0x7b540070f8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:14.932494 (+   162us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6041869184}
I20260924 14:31:14.934038 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:14.934198 31128 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:14.923161 30946 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:14.893872 31118 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:14.677309 30963 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:14.924201 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 913865us
I20260924 14:31:14.934648 26254 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:14.649098 30274 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:14.853569 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:14.833475 26256 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.997s	user 0.005s	sys 0.025s
W20260924 14:31:14.940413 26256 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.104s	user 0.007s	sys 0.027s
W20260924 14:31:14.853478 26402 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.023s	sys 0.051s
W20260924 14:31:14.791253 31182 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.721s	user 0.000s	sys 0.001s
I20260924 14:31:14.817817 31083 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:14.707894 31111 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:14.816705 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:14.952962 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1656) took 1260 ms. Trace:
I20260924 14:31:14.953516 26252 rpcz_store.cc:276] 0924 14:31:13.692791 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.816314 (+1123523us) service_pool.cc:227] Handling call
0924 14:31:14.952860 (+136546us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:14.443733 31195 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:14.450619 31184 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:14.618012 31109 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:14.935016 31004 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 2.376s	user 0.009s	sys 0.004s
I20260924 14:31:14.872285 26323 rpcz_store.cc:276] 0924 14:31:13.295386 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:13.410380 (+114994us) service_pool.cc:227] Handling call
0924 14:31:14.680147 (+1269767us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b540012eb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:14.680292 (+   145us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1090363136}
I20260924 14:31:14.668283 31041 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:14.738080 31073 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.789206 31102 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:14.812384 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1537) took 1516 ms. Trace:
I20260924 14:31:14.967093 26325 rpcz_store.cc:276] 0924 14:31:13.295980 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.741238 (+1445258us) service_pool.cc:227] Handling call
0924 14:31:14.811691 (+ 70453us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:14.811947 (+   256us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:14.811983 (+    36us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:14.812042 (+    59us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:14.812085 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.812164 (+    79us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:14.812310 (+   146us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:14.880643 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 146898us
I20260924 14:31:14.969117 27126 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:14.791599 31175 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.799602 31186 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:14.808275 31077 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:14.788858 31194 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:14.899148 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:14.980005 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 80958.2us
I20260924 14:31:14.896018 26328 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:14.721268 31103 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:14.869766 30842 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:14.900840 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1538) took 1207 ms. Trace:
W20260924 14:31:14.903393 26295 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.021s	sys 0.040s
I20260924 14:31:14.880414 31071 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:14.912285 26296 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.016s	sys 0.016s
I20260924 14:31:14.912055 31141 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:14.912842 30983 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:14.899989 30974 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:14.896366 31097 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:14.888002 31037 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.011s	sys 0.028s
I20260924 14:31:14.914021 26324 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260924 14:31:14.914343 31092 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:14.892395 31196 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:14.914975 31160 consensus_meta.cc:287] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.978s	user 0.016s	sys 0.000s
I20260924 14:31:14.903230 31149 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:14.977676 31102 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:14.916329 30741 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:14.920555 31148 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:14.869913 31150 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:14.953604 31091 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:14.426913 31155 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:14.944062 26221 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.016s	sys 0.010s
W20260924 14:31:15.035158 26221 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.017s	sys 0.011s
W20260924 14:31:15.035485 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 697409us
I20260924 14:31:15.036209 30972 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:15.037992 31164 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.610s	user 0.000s	sys 0.001s
I20260924 14:31:15.038374 30972 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:14.944363 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:14.944566 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.016s	sys 0.032s
W20260924 14:31:15.039884 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.016s	sys 0.032s
I20260924 14:31:15.040555 31149 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:14.979049 31175 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:15.042429 31175 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:14.954763 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.043682 31149 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.043845 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.044576 31141 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:14.961066 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:15.044799 31155 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:15.045364 26400 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:15.046306 26323 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:15.046700 31103 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:15.047489 31102 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:15.015581 31160 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:31:14.956133 31204 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260924 14:31:14.953118 30963 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:14.975064 26325 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:14.976152 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.052176 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:14.923220 31120 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:14.981164 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1020) took 1278 ms. Trace:
I20260924 14:31:14.976562 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:14.981611 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 1.02452e+06us
I20260924 14:31:14.981444 26332 rpcz_store.cc:276] 0924 14:31:13.693616 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.894129 (+1200513us) service_pool.cc:227] Handling call
0924 14:31:14.900169 (+  6040us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:14.900423 (+   254us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:14.900460 (+    37us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:14.900515 (+    55us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:14.900569 (+    54us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.900642 (+    73us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:14.900773 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:14.982872 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1670) took 2382 ms. Trace:
I20260924 14:31:14.980705 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:14.973487 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.010s	sys 0.012s
W20260924 14:31:14.997757 26296 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.016s	sys 0.020s
W20260924 14:31:14.973059 31158 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.006s	sys 0.020s
I20260924 14:31:14.996488 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1166) took 2764 ms. Trace:
I20260924 14:31:15.065709 31092 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.066565 31103 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:31:15.067020 31103 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:15.066943 31092 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.067538 31103 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:15.069532 31092 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:15.070360 26328 rpcz_store.cc:276] 0924 14:31:13.702643 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.632078 (+929435us) service_pool.cc:227] Handling call
0924 14:31:14.981101 (+349023us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:15.072144 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1028) took 1107 ms. Trace:
I20260924 14:31:15.072551 31109 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:15.073282 30985 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:15.073423 31148 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:15.052569 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1025) took 1182 ms. Trace:
W20260924 14:31:15.074168 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 97715.2us
I20260924 14:31:15.074361 26325 rpcz_store.cc:276] 0924 14:31:13.869776 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.974096 (+1104320us) service_pool.cc:227] Handling call
0924 14:31:14.974616 (+   520us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:15.052307 (+ 77691us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.052512 (+   205us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:15.072490 26328 rpcz_store.cc:276] 0924 14:31:13.965074 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.071103 (+1106029us) service_pool.cc:227] Handling call
0924 14:31:15.071485 (+   382us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:15.071810 (+   325us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:15.071841 (+    31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:15.071883 (+    42us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:15.071920 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.071977 (+    57us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:15.072090 (+   113us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:15.075886 26325 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260924 14:31:15.078857 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.011s	sys 0.013s
W20260924 14:31:15.080265 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 814967us
W20260924 14:31:15.075780 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 95156.1us
I20260924 14:31:15.006315 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1019) took 1304 ms. Trace:
I20260924 14:31:14.976428 31186 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:14.993723 31078 consensus_meta.cc:287] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 2.230s	user 0.012s	sys 0.002s
I20260924 14:31:14.983559 31173 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260924 14:31:14.915848 26223 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.179s	user 0.019s	sys 0.012s
W20260924 14:31:15.083350 26223 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.389s	user 0.019s	sys 0.012s
W20260924 14:31:15.003278 31037 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.012s	sys 0.028s
W20260924 14:31:15.014194 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 157894us
I20260924 14:31:14.904263 31206 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:15.019776 26254 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
W20260924 14:31:14.941193 26402 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.023s	sys 0.052s
I20260924 14:31:14.970058 31073 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:14.997818 31217 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260924 14:31:15.023758 26372 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.010s	sys 0.026s
W20260924 14:31:15.024451 31192 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.640s	user 0.000s	sys 0.000s
W20260924 14:31:15.024792 31181 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.449s	user 0.001s	sys 0.000s
I20260924 14:31:15.025084 31171 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:15.025631 31195 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:15.024469 31035 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:15.026389 31184 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:14.997921 30842 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:15.012151 31104 consensus_meta.cc:287] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 2.224s	user 0.008s	sys 0.007s
I20260924 14:31:14.971860 31004 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:15.039528 31193 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260924 14:31:14.879472 30739 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:14.960429 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:15.044261 31149 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:14.934643 31128 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:15.017360 31121 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.009s	sys 0.007s
I20260924 14:31:15.049465 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1508) took 1654 ms. Trace:
W20260924 14:31:15.032507 31023 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 2.168s	user 0.019s	sys 0.001s
I20260924 14:31:15.049950 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1640) took 3107 ms. Trace:
I20260924 14:31:15.272476 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.081574 26324 rpcz_store.cc:276] 0924 14:31:13.702253 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.896839 (+1194586us) service_pool.cc:227] Handling call
0924 14:31:14.913662 (+ 16823us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:15.005407 (+ 91745us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.006239 (+   832us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:15.271253 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1677) took 1085 ms. Trace:
I20260924 14:31:15.076728 26403 rpcz_store.cc:276] 0924 14:31:12.600374 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.611607 (+ 11233us) service_pool.cc:227] Handling call
0924 14:31:14.982692 (+2371085us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540023f854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:14.982802 (+   110us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3440681856}
I20260924 14:31:15.085172 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1029) took 1112 ms. Trace:
I20260924 14:31:15.273481 27186 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:15.273918 26325 rpcz_store.cc:276] 0924 14:31:13.973252 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.075271 (+1102019us) service_pool.cc:227] Handling call
0924 14:31:15.075663 (+   392us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:15.084734 (+  9071us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.085094 (+   360us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:15.087682 26331 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:15.119227 31175 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:15.032531 31039 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.031329 31079 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.037436 31071 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:15.005398 31194 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:15.146920 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:15.260432 31104 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:31:15.180011 26372 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.021s	user 0.013s	sys 0.029s
I20260924 14:31:15.136884 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:15.300809 26406 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:15.178582 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46346.6us
W20260924 14:31:15.076474 31158 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.006s	sys 0.022s
I20260924 14:31:15.302898 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:15.303148 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:15.182665 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260924 14:31:15.182677 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 864) took 1160 ms. Trace:
I20260924 14:31:15.186105 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1559) took 1143 ms. Trace:
I20260924 14:31:15.306138 26326 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:15.306206 26254 rpcz_store.cc:276] 0924 14:31:14.042707 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.933771 (+891064us) service_pool.cc:227] Handling call
0924 14:31:15.186015 (+252244us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:15.114984 31078 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:15.189610 31171 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:15.211246 31195 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:15.217396 31198 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
W20260924 14:31:15.145285 31152 consensus_meta.cc:287] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.347s	user 0.012s	sys 0.000s
W20260924 14:31:15.139963 31219 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260924 14:31:15.180274 31217 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:15.331684 31217 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:15.139719 31223 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.082994 31186 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:15.085866 31111 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.074038 31148 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:15.067669 31100 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:15.089355 31041 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:15.146144 30274 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:15.148538 31118 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260924 14:31:15.065974 31150 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.188196 31083 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.216387 31218 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.040601 30972 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:15.261761 26252 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:15.260408 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:15.264882 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 816) took 2737 ms. Trace:
I20260924 14:31:15.058151 26400 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:15.178910 31173 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:15.281014 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:15.212800 31120 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.270704 26404 rpcz_store.cc:276] 0924 14:31:13.394787 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:13.394994 (+   207us) service_pool.cc:227] Handling call
0924 14:31:13.395534 (+   540us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:13.865242 (+469708us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:13.865274 (+    32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:13.865765 (+   491us) log.cc:842] Serialized 26 byte log entry
0924 14:31:14.735226 (+869461us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:14.735284 (+    58us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.735379 (+    95us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:15.043516 (+308137us) raft_consensus.cc:1671] finished
0924 14:31:15.043539 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:15.049404 (+  5865us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6933,"raft.queue_time_us":192220,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9800,"thread_start_us":1155710,"threads_started":2,"wal-append.queue_time_us":1038453}
I20260924 14:31:15.283655 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1536) took 1988 ms. Trace:
I20260924 14:31:15.283499 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.284121 27182 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:15.273545 26402 rpcz_store.cc:276] 0924 14:31:14.186023 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.186174 (+   151us) service_pool.cc:227] Handling call
0924 14:31:14.186543 (+   369us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:14.186783 (+   240us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:15.180591 (+993808us) log.cc:842] Serialized 14 byte log entry
0924 14:31:15.269285 (+ 88694us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:15.269357 (+    72us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:15.271007 (+  1650us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.271082 (+    75us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:15.271197 (+   115us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":681566,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":3954,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":3999,"thread_start_us":1074817,"threads_started":2}
I20260924 14:31:15.285411 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:15.258617 30961 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.272970 26258 rpcz_store.cc:276] 0924 14:31:11.942585 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:11.942753 (+   168us) service_pool.cc:227] Handling call
0924 14:31:11.968007 (+ 25254us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:12.021487 (+ 53480us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:12.021642 (+   155us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:12.055829 (+ 34187us) log.cc:842] Serialized 26 byte log entry
0924 14:31:12.310644 (+254815us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b1800007600. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:31:13.129544 (+818900us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:13.129597 (+    53us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:13.129683 (+    86us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:15.039213 (+1909530us) raft_consensus.cc:1671] finished
0924 14:31:15.039233 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:15.049888 (+ 10655us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":66920,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":11934,"spinlock_wait_cycles":4954368,"thread_start_us":869074,"threads_started":2,"wal-append.queue_time_us":1977666}
W20260924 14:31:15.270601 31121 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.009s	sys 0.007s
I20260924 14:31:15.077337 26253 rpcz_store.cc:276] 0924 14:31:12.231691 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.534906 (+303215us) service_pool.cc:227] Handling call
0924 14:31:12.535295 (+   389us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:12.536304 (+  1009us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:12.536392 (+    88us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:12.537691 (+  1299us) log.cc:842] Serialized 26 byte log entry
0924 14:31:13.602595 (+1064904us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:13.638394 (+ 35799us) log.cc:842] Serialized 14 byte log entry
0924 14:31:13.638645 (+   251us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:13.638737 (+    92us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:14.996205 (+1357468us) raft_consensus.cc:1671] finished
0924 14:31:14.996243 (+    38us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:14.996423 (+   180us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":137678,"prepare.queue_time_us":77763,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":74875,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":11382,"thread_start_us":1064166,"threads_started":1,"wal-append.queue_time_us":1187052}
I20260924 14:31:15.135257 31092 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.138s
I20260924 14:31:15.292744 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.297140 27194 raft_consensus.cc:2354] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9.
I20260924 14:31:15.260169 30688 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:15.088560 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1030) took 1095 ms. Trace:
I20260924 14:31:15.405963 26328 rpcz_store.cc:276] 0924 14:31:13.993043 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.077007 (+1083964us) service_pool.cc:227] Handling call
0924 14:31:15.077439 (+   432us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:15.088154 (+ 10715us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:15.088180 (+    26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:15.088227 (+    47us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:15.088261 (+    34us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.088328 (+    67us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:15.088480 (+   152us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":41515,"raft.queue_time_us":11170,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":26985}
I20260924 14:31:15.407876 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:15.210615 31210 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.307593 26405 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:15.307387 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:15.303773 26324 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:31:15.309885 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:15.304909 27191 rpcz_store.cc:276] 0924 14:31:14.022783 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.022911 (+   128us) service_pool.cc:227] Handling call
0924 14:31:14.054802 (+ 31891us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:14.055047 (+   245us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:14.058862 (+  3815us) log.cc:842] Serialized 14 byte log entry
0924 14:31:15.182313 (+1123451us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:15.182387 (+    74us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:15.182432 (+    45us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.182501 (+    69us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:15.182620 (+   119us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":55686,"prepare.queue_time_us":99060,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":69871,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":7190,"thread_start_us":1025523,"threads_started":2,"wal-append.queue_time_us":816670}
W20260924 14:31:15.312282 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 236006us
I20260924 14:31:15.314965 26331 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260924 14:31:15.288516 31038 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:15.329178 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 818) took 2043 ms. Trace:
I20260924 14:31:15.333012 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.281514 31023 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:15.342123 31111 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:15.412550 26330 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:15.336129 31206 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:15.349064 31148 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:15.315634 31104 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:15.286841 31128 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:15.352304 31230 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260924 14:31:15.334785 26406 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:15.430233 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:15.333664 30787 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:15.433384 30787 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:15.433794 30787 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:15.433982 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:15.429903 26325 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:15.434931 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:15.319063 31088 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.358259 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:15.438598 26257 raft_consensus.cc:2354] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13.
I20260924 14:31:15.353051 31223 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:15.370024 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:15.428200 31232 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260924 14:31:15.381682 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 121356us
I20260924 14:31:15.392601 27186 raft_consensus.cc:2354] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14.
I20260924 14:31:15.392439 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.317637 31195 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.394339 26327 rpcz_store.cc:276] 0924 14:31:13.295697 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:13.631227 (+335530us) service_pool.cc:227] Handling call
0924 14:31:13.631599 (+   372us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:13.800203 (+168604us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:13.800249 (+    46us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:13.800303 (+    54us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:13.800347 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.283404 (+1483057us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540001dbd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:15.283481 (+    77us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:15.283603 (+   122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":387204864}
I20260924 14:31:15.385437 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1022) took 1662 ms. Trace:
I20260924 14:31:15.400552 26251 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260924 14:31:15.402520 26403 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:15.399708 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1645) took 3021 ms. Trace:
I20260924 14:31:15.359762 31073 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:15.411046 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.411942 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1026) took 1518 ms. Trace:
I20260924 14:31:15.411839 27181 rpcz_store.cc:276] 0924 14:31:13.285686 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:13.290934 (+  5248us) service_pool.cc:227] Handling call
0924 14:31:13.319619 (+ 28685us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:14.931892 (+1612273us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.328919 (+397027us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400001c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:15.329108 (+   189us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3706488192}
I20260924 14:31:15.412642 26332 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:15.334937 31152 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:15.413283 26399 raft_consensus.cc:2396] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:31:15.413352 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:15.365276 31118 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:15.381381 31173 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:15.416760 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:15.418538 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:15.381990 31155 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:15.420274 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:15.398269 30972 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:15.423106 26252 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:15.425670 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.432776 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.413838 31122 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.316836 30739 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:15.367758 31083 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:15.439056 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 80960us
I20260924 14:31:15.308458 31184 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260924 14:31:15.416059 31038 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:15.367653 27192 rpcz_store.cc:276] 0924 14:31:12.527048 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.624102 (+ 97054us) service_pool.cc:227] Handling call
0924 14:31:12.624438 (+   336us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:12.624834 (+   396us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:12.624862 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:12.635902 (+ 11040us) log.cc:842] Serialized 26 byte log entry
0924 14:31:13.735129 (+1099227us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:14.140215 (+405086us) log.cc:842] Serialized 14 byte log entry
0924 14:31:14.152169 (+ 11954us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.152248 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:15.264664 (+1112416us) raft_consensus.cc:1671] finished
0924 14:31:15.264706 (+    42us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:15.264822 (+   116us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":39220,"prepare.queue_time_us":80564,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":124124,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":26051,"thread_start_us":1358354,"threads_started":2,"wal-append.queue_time_us":1351530}
I20260924 14:31:15.659133 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.448211 26400 raft_consensus.cc:2379] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:15.451527 31228 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:15.658588 26406 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:15.455663 31149 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:15.458941 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1018) took 3488 ms. Trace:
I20260924 14:31:15.460577 31024 raft_consensus.cc:3032] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:15.416177 30993 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:15.372390 30974 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:15.404758 31120 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:15.409101 30961 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.407996 31103 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:15.413038 30688 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.464293 31079 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:15.766357 26255 raft_consensus.cc:2354] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 15. Current term is 16.
I20260924 14:31:15.364810 31150 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:15.370028 31218 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:15.655727 26252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1177) took 1494 ms (client timeout 1668 ms). Trace:
I20260924 14:31:15.656085 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:15.657526 26253 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:15.565971 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.657146 26331 rpcz_store.cc:276] 0924 14:31:13.921644 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.048909 (+1127265us) service_pool.cc:227] Handling call
0924 14:31:15.087495 (+ 38586us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:15.411677 (+324182us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.411875 (+   198us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:15.661760 26323 rpcz_store.cc:276] 0924 14:31:13.723223 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.965031 (+1241808us) service_pool.cc:227] Handling call
0924 14:31:14.965479 (+   448us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:15.046707 (+ 81228us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.385119 (+338412us) spinlock_profiling.cc:243] Waited 80.4 ms on lock 0x7b540012e654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:15.385362 (+   243us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":185024896}
I20260924 14:31:15.443969 31229 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:15.680023 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.681495 26326 rpcz_store.cc:276] 0924 14:31:11.970327 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.891882 (+921555us) service_pool.cc:227] Handling call
0924 14:31:12.892221 (+   339us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:15.306643 (+2414422us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.458576 (+151933us) spinlock_profiling.cc:243] Waited 2.41 s on lock 0x7b540001f4d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:15.458836 (+   260us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5550543232}
I20260924 14:31:15.682346 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.665004 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.654837 26256 rpcz_store.cc:276] 0924 14:31:12.378568 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.562766 (+184198us) service_pool.cc:227] Handling call
0924 14:31:12.563056 (+   290us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:12.624250 (+ 61194us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:12.624283 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:12.811412 (+187129us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b18003cfc40. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a
0924 14:31:13.835117 (+1023705us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b2c001c3658. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:13.835776 (+   659us) log.cc:842] Serialized 26 byte log entry
0924 14:31:14.940836 (+1105060us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:14.940899 (+    63us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:14.940994 (+    95us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:15.399424 (+458430us) raft_consensus.cc:1671] finished
0924 14:31:15.399468 (+    44us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:15.399650 (+   182us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":172,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":32543,"spinlock_wait_cycles":27389056,"thread_start_us":1104588,"threads_started":1,"wal-append.queue_time_us":1123621}
I20260924 14:31:15.684482 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.687098 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:15.687198 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:15.637142 30963 consensus_meta.cc:287] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.524s	user 0.003s	sys 0.008s
I20260924 14:31:15.416260 31210 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:15.692255 31155 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:31:15.799687 26253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1176) took 1645 ms (client timeout 1977 ms). Trace:
I20260924 14:31:15.668466 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:15.800266 26253 rpcz_store.cc:269] 0924 14:31:14.154196 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.405582 (+1251386us) service_pool.cc:227] Handling call
0924 14:31:15.799628 (+394046us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:15.655422 31078 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:15.444756 31128 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:15.766765 31158 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260924 14:31:15.465547 31088 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:15.800551 31184 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:15.806058 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:15.815948 26326 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:15.816066 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:15.439600 31004 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:15.694410 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:15.592183 31039 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:15.429874 31104 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.828058 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:15.716861 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.829216 26253 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:15.771136 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:15.585961 30842 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:15.720558 31066 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.470s	user 0.008s	sys 0.011s
W20260924 14:31:15.838703 31066 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.597s	user 0.011s	sys 0.016s
W20260924 14:31:15.839612 31231 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.473s	user 0.001s	sys 0.001s
W20260924 14:31:15.841565 31236 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260924 14:31:15.843482 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.465139 31175 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:15.781967 26254 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:31:15.791968 26406 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:31:15.792954 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.793059 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.793479 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:15.794661 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:15.795825 26397 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:31:15.797191 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 110194us
I20260924 14:31:15.798350 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:15.798971 26252 rpcz_store.cc:269] 0924 14:31:14.173189 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.960023 (+786834us) service_pool.cc:227] Handling call
0924 14:31:15.655662 (+695639us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:15.659494 31176 raft_consensus.cc:979] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:31:15.657142 31118 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:15.801720 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:15.802825 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:15.852427 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:15.663944 30972 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:15.805720 31210 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:15.802579 31229 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:15.760694 31228 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:15.806254 26258 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:15.814411 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.864120 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1676) took 1708 ms. Trace:
I20260924 14:31:15.864403 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.865397 27191 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:15.865185 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.805742 31218 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:15.821043 26332 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:15.772807 31194 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:15.685425 31213 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:15.821030 26403 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:15.825424 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:15.696084 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:15.807914 31237 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260924 14:31:15.831920 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 60819.6us
I20260924 14:31:15.876667 27192 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:15.681360 31122 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:15.810626 30985 consensus_meta.cc:287] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.737s	user 0.008s	sys 0.004s
I20260924 14:31:15.813189 31023 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:15.834527 30974 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:15.799130 31141 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260924 14:31:15.693878 30993 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:15.846670 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 281820us
I20260924 14:31:15.852761 26327 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:15.658455 30739 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.872701 31210 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:15.925021 26253 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:15.855125 26404 raft_consensus.cc:2396] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260924 14:31:15.849421 26256 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
W20260924 14:31:15.947722 26256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1175) took 1797 ms (client timeout 2048 ms). Trace:
W20260924 14:31:15.948282 26256 rpcz_store.cc:269] 0924 14:31:14.153776 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.794432 (+1640656us) service_pool.cc:227] Handling call
0924 14:31:15.947657 (+153225us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:15.881202 31080 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.718858 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:15.858852 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.805898 31103 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:15.807044 31083 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:15.859923 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:15.857317 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.864744 26405 rpcz_store.cc:276] 0924 14:31:14.155284 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.155555 (+   271us) service_pool.cc:227] Handling call
0924 14:31:14.155900 (+   345us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:15.410311 (+1254411us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:15.863783 (+453472us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400290754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:15.864036 (+   253us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2647925504}
I20260924 14:31:15.870146 26326 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:15.876935 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:15.877764 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 181742us
I20260924 14:31:15.685441 31024 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:15.881564 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:15.978973 26406 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 9
I20260924 14:31:15.884438 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:15.887144 26403 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:15.847700 31158 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:15.854457 30842 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:15.814070 31217 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.927639 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:15.872625 31039 raft_consensus.cc:1067] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:31:15.852377 26323 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:15.942886 26327 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:31:15.944341 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 313330us
I20260924 14:31:15.995971 27188 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:15.861064 31175 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:15.828029 31235 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.459s	user 0.001s	sys 0.000s
I20260924 14:31:15.893082 31004 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:15.946089 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.950779 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:16.027212 27192 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:16.027364 26253 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:15.955591 26258 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:15.968400 30739 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:15.968703 31038 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:15.970937 26332 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:15.919926 31111 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:15.991255 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.037552 27186 raft_consensus.cc:2396] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260924 14:31:15.988117 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:15.874047 31228 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.949326 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.039212 26256 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:16.040295 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.041640 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:15.999552 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:16.043185 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 43800.1us
I20260924 14:31:16.044157 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:16.007139 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.008989 26331 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:15.974462 31080 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:16.009943 26400 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260924 14:31:16.049325 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1523) took 1066 ms. Trace:
I20260924 14:31:16.049651 26400 rpcz_store.cc:276] 0924 14:31:14.982712 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.977656 (+994944us) service_pool.cc:227] Handling call
0924 14:31:16.009728 (+ 32072us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:16.049049 (+ 39321us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.049268 (+   219us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":15104}
W20260924 14:31:16.011031 31197 log.cc:924] Time spent T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.770s	user 0.001s	sys 0.002s
I20260924 14:31:16.010442 31229 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:15.970196 31152 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:16.013653 31195 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:16.014765 31060 raft_consensus.cc:3032] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:16.015620 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1165) took 3784 ms. Trace:
I20260924 14:31:16.062662 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:16.062440 26251 rpcz_store.cc:276] 0924 14:31:12.231249 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.534855 (+303606us) service_pool.cc:227] Handling call
0924 14:31:12.944940 (+410085us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:31:15.400165 (+2455225us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:15.654279 (+254114us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.015381 (+361102us) spinlock_profiling.cc:243] Waited 2.45 s on lock 0x7b54006d2ed0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:16.015556 (+   175us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5675390720}
I20260924 14:31:16.063556 26331 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:16.015872 30974 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:15.880858 31213 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:15.947237 31118 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:15.948804 31155 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:16.015661 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 138861us
I20260924 14:31:16.020174 27188 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260924 14:31:16.021311 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1526) took 1025 ms. Trace:
W20260924 14:31:16.030424 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 245149us
I20260924 14:31:16.039079 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.101727 26332 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:16.002501 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:15.976540 31210 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:16.055030 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:16.107333 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52395.2us
I20260924 14:31:15.989045 30842 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:16.056370 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:16.060572 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:16.110671 26323 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:16.111348 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 50857.6us
I20260924 14:31:16.112501 27123 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:16.064632 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.114653 26400 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:16.008600 31158 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:16.025622 31223 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:16.066044 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1648) took 2845 ms. Trace:
I20260924 14:31:16.109164 26326 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:16.057857 30739 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:16.025262 30688 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:16.061070 26253 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:16.031507 31004 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:16.058127 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:16.045609 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:16.090340 26406 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
W20260924 14:31:16.017671 27182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 1318) took 3094 ms (client timeout 3219 ms). Trace:
I20260924 14:31:16.053474 31111 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:16.105824 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.106050 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.118309 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 987) took 1031 ms. Trace:
I20260924 14:31:16.212600 26326 rpcz_store.cc:276] 0924 14:31:15.087126 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.825011 (+737885us) service_pool.cc:227] Handling call
0924 14:31:16.118251 (+293240us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.214159 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:16.215090 26326 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:16.215018 27182 rpcz_store.cc:269] 0924 14:31:12.922996 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.923154 (+   158us) service_pool.cc:227] Handling call
0924 14:31:16.017438 (+3094284us) spinlock_profiling.cc:243] Waited 2.27 s on lock 0x7b540000cb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:16.017579 (+   141us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5228093824}
I20260924 14:31:16.097569 26405 rpcz_store.cc:276] 0924 14:31:14.996395 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.000759 (+1004364us) service_pool.cc:227] Handling call
0924 14:31:16.001197 (+   438us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.020929 (+ 19732us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:16.020963 (+    34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:16.021015 (+    52us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:16.021056 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.021133 (+    77us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:16.021254 (+   121us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":19271,"raft.queue_time_us":24113,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":17136}
I20260924 14:31:16.216856 26332 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:16.121135 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.175073 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.120882 27186 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260924 14:31:16.220670 26331 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:16.224906 26256 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:16.225677 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1661) took 1009 ms. Trace:
I20260924 14:31:16.226199 26253 rpcz_store.cc:276] 0924 14:31:15.215846 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.950466 (+734620us) service_pool.cc:227] Handling call
0924 14:31:16.225599 (+275133us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.227669 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:16.228047 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.228838 26331 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:16.093495 31120 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:16.062131 31024 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:16.172982 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:16.188496 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.235455 26400 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:16.117779 26250 rpcz_store.cc:276] 0924 14:31:13.220901 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:13.222199 (+  1298us) service_pool.cc:227] Handling call
0924 14:31:13.276586 (+ 54387us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:13.277024 (+   438us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:13.277065 (+    41us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:13.314566 (+ 37501us) log.cc:842] Serialized 26 byte log entry
0924 14:31:13.861565 (+546999us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:31:13.861611 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:13.861686 (+    75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:16.010329 (+2148643us) spinlock_profiling.cc:243] Waited 71.3 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:16.061386 (+ 51057us) raft_consensus.cc:1671] finished
0924 14:31:16.061413 (+    27us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:16.065962 (+  4549us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":636,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":17048,"spinlock_wait_cycles":163877248,"thread_start_us":332158,"threads_started":1,"wal-append.queue_time_us":623663}
I20260924 14:31:16.236634 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.062026 31060 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:16.231629 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.128132 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.186112 26323 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:16.238714 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 992) took 1054 ms. Trace:
I20260924 14:31:16.239223 26323 rpcz_store.cc:276] 0924 14:31:15.184493 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.056084 (+871591us) service_pool.cc:227] Handling call
0924 14:31:16.238620 (+182536us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:16.239321 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 193795us
I20260924 14:31:16.240597 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.241370 26323 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
I20260924 14:31:16.241475 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:16.171571 31186 consensus_meta.cc:287] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.819s	user 0.011s	sys 0.001s
I20260924 14:31:16.242420 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.243687 26323 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:16.236558 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.246241 26400 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 6.
W20260924 14:31:16.222211 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 199312us
I20260924 14:31:16.237597 26405 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:16.249022 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:16.249202 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.251217 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.144917 31118 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:16.253264 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 182427us
I20260924 14:31:16.253942 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.250152 26251 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:31:16.257532 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1658) took 1046 ms. Trace:
I20260924 14:31:16.201502 30972 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:16.117791 31033 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:16.269193 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.270572 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:16.255051 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 82179.1us
I20260924 14:31:16.129179 31150 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:16.255038 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.258071 26251 rpcz_store.cc:276] 0924 14:31:15.211323 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.221378 (+1010055us) service_pool.cc:227] Handling call
0924 14:31:16.257474 (+ 36096us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.255424 26406 raft_consensus.cc:2396] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260924 14:31:16.257048 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:16.172132 31206 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260924 14:31:16.231984 31004 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:16.144065 30961 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:16.144743 31104 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:16.252838 31210 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:16.263902 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.264950 31244 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:16.129120 31088 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:16.254534 31079 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:16.119019 30842 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:16.267995 26256 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:16.268122 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.280315 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.222756 30739 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:16.129308 31039 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:16.129222 31032 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:16.128886 31083 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.164s
I20260924 14:31:16.179930 31218 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260924 14:31:16.252452 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:16.295878 26250 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:16.428383 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1562) took 1244 ms. Trace:
W20260924 14:31:16.425855 26249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1548) took 4365 ms (client timeout 1935 ms). Trace:
W20260924 14:31:16.427124 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 114975us
W20260924 14:31:16.429229 26249 rpcz_store.cc:269] 0924 14:31:12.060026 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.083149 (+ 23123us) service_pool.cc:227] Handling call
0924 14:31:16.425509 (+4342360us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b54002a1a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:16.425718 (+   209us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5875225472}
I20260924 14:31:16.428867 26256 rpcz_store.cc:276] 0924 14:31:15.183787 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.057876 (+874089us) service_pool.cc:227] Handling call
0924 14:31:16.428317 (+370441us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.428876 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1660) took 1216 ms. Trace:
I20260924 14:31:16.431397 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.431797 26331 raft_consensus.cc:2396] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260924 14:31:16.431785 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:16.434785 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 182459us
I20260924 14:31:16.435694 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:16.434701 26326 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:16.438556 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 997) took 1084 ms. Trace:
I20260924 14:31:16.438797 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1657) took 1250 ms. Trace:
I20260924 14:31:16.439524 26249 rpcz_store.cc:276] 0924 14:31:15.188850 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.431542 (+1242692us) service_pool.cc:227] Handling call
0924 14:31:16.438734 (+  7192us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.431823 26250 rpcz_store.cc:276] 0924 14:31:15.212024 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.237932 (+1025908us) service_pool.cc:227] Handling call
0924 14:31:16.428819 (+190887us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.440874 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:16.441663 26253 raft_consensus.cc:2396] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260924 14:31:16.435048 27126 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:16.442371 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.442706 26326 rpcz_store.cc:276] 0924 14:31:15.354458 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.213812 (+859354us) service_pool.cc:227] Handling call
0924 14:31:16.438496 (+224684us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:16.411078 31245 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
W20260924 14:31:16.347219 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 216189us
I20260924 14:31:16.442005 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1568) took 1074 ms. Trace:
W20260924 14:31:16.412139 31246 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260924 14:31:16.258841 31223 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:16.452665 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1181) took 1181 ms. Trace:
I20260924 14:31:16.453131 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:16.443805 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:16.446909 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1040) took 1097 ms. Trace:
I20260924 14:31:16.449038 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.323750 30993 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:16.432855 30739 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:16.268421 31024 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:16.446873 26249 rpcz_store.cc:276] 0924 14:31:15.367612 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.440536 (+1072924us) service_pool.cc:227] Handling call
0924 14:31:16.441950 (+  1414us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.255811 31111 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.195s
I20260924 14:31:16.433342 31039 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:16.428404 31079 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:16.449137 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.454336 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.569548 26250 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
I20260924 14:31:16.570160 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1183) took 1289 ms. Trace:
I20260924 14:31:16.570520 26250 rpcz_store.cc:276] 0924 14:31:15.280976 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.441487 (+1160511us) service_pool.cc:227] Handling call
0924 14:31:16.570091 (+128604us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.453384 26253 rpcz_store.cc:276] 0924 14:31:15.271286 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.250866 (+979580us) service_pool.cc:227] Handling call
0924 14:31:16.451937 (+201071us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.455035 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:16.579071 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:16.582904 26405 raft_consensus.cc:2396] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260924 14:31:16.572822 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1667) took 1141 ms. Trace:
I20260924 14:31:16.584019 26251 rpcz_store.cc:276] 0924 14:31:15.430911 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.454778 (+1023867us) service_pool.cc:227] Handling call
0924 14:31:16.572749 (+117971us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.455315 31104 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:16.511070 31078 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:16.597422 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.454516 26331 rpcz_store.cc:276] 0924 14:31:15.349338 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.244311 (+894973us) service_pool.cc:227] Handling call
0924 14:31:16.446814 (+202503us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.436652 31217 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:16.508095 27125 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:16.513252 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.435700 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1564) took 1171 ms. Trace:
I20260924 14:31:16.524147 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.602967 26256 rpcz_store.cc:276] 0924 14:31:15.264251 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.431102 (+1166851us) service_pool.cc:227] Handling call
0924 14:31:16.435637 (+  4535us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.527598 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.530203 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.605515 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.532466 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.534838 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.427742 31060 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:16.538247 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:16.536446 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.555042 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.558682 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:16.561348 26258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1179) took 2335 ms (client timeout 1828 ms). Trace:
I20260924 14:31:16.612985 26327 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 6.
W20260924 14:31:16.613458 26258 rpcz_store.cc:269] 0924 14:31:14.225942 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.397483 (+1171541us) service_pool.cc:227] Handling call
0924 14:31:16.561121 (+1163638us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54004f9f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:16.561260 (+   139us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":344724224}
I20260924 14:31:16.614754 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.615559 26258 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:31:16.616616 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:16.447578 31080 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:16.452736 31223 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:16.456205 31226 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:31:16.450511 31244 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:16.255076 31186 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:16.142522 31041 consensus_queue.cc:1159] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e09fbebe29a49d291d23f9d395fe335" responder_term: 13 status { last_received { term: 11 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:31:16.507586 31088 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:16.590981 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.593286 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.594431 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.636727 26251 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:16.540483 31032 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:16.639809 31032 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:16.639896 30964 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:16.640399 31032 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:16.598317 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1184) took 1311 ms. Trace:
I20260924 14:31:16.643164 26250 rpcz_store.cc:276] 0924 14:31:15.287378 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.571336 (+1283958us) service_pool.cc:227] Handling call
0924 14:31:16.598257 (+ 26921us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.644613 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:16.621979 30993 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:16.645483 26250 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:16.645723 30993 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:16.574071 31039 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:16.647960 30993 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:16.508888 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:16.648208 31100 consensus_meta.cc:287] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.290s	user 0.012s	sys 0.000s
I20260924 14:31:16.652938 26401 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:16.653376 30793 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:16.653906 30793 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:16.604703 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1524) took 1621 ms. Trace:
I20260924 14:31:16.582041 31079 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:16.655043 26404 rpcz_store.cc:276] 0924 14:31:14.982985 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.977592 (+994607us) service_pool.cc:227] Handling call
0924 14:31:16.009686 (+ 32094us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.009894 (+   208us) raft_consensus.cc:1520] Early marking committed up to index 11
0924 14:31:16.255951 (+246057us) log.cc:842] Serialized 14 byte log entry
0924 14:31:16.603639 (+347688us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:16.603786 (+   147us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:16.603937 (+   151us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.604365 (+   428us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:16.604576 (+   211us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":331199,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":10562,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":13878,"spinlock_wait_cycles":1920,"thread_start_us":560890,"threads_started":2,"wal-append.queue_time_us":371988}
I20260924 14:31:16.657106 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.602648 26249 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:16.658170 31226 raft_consensus.cc:686] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:16.658905 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1565) took 1348 ms. Trace:
I20260924 14:31:16.610366 27181 raft_consensus.cc:2379] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:31:16.659593 30787 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.225s	user 0.010s	sys 0.003s
I20260924 14:31:16.608580 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.660411 26326 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:16.631606 31060 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:16.661338 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.445247 30961 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:16.619710 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.662848 26256 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:16.663770 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.617317 26258 raft_consensus.cc:2362] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16. Re-sending same reply.
I20260924 14:31:16.624056 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.589392 30963 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:16.589107 31104 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:16.589890 31228 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:16.631349 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.651939 26332 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:16.665006 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.505229 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.665871 26258 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:16.666319 26406 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:16.666702 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:16.603188 31173 consensus_meta.cc:287] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.915s	user 0.012s	sys 0.002s
I20260924 14:31:16.665220 26331 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:16.607978 31111 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:16.637468 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1563) took 1423 ms. Trace:
I20260924 14:31:16.676007 26251 rpcz_store.cc:276] 0924 14:31:15.213985 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.594099 (+1380114us) service_pool.cc:227] Handling call
0924 14:31:16.637385 (+ 43286us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.676916 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.637985 26253 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:16.678345 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1185) took 1379 ms. Trace:
W20260924 14:31:16.678884 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 224436us
I20260924 14:31:16.678718 26253 rpcz_store.cc:276] 0924 14:31:15.298530 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.572245 (+1273715us) service_pool.cc:227] Handling call
0924 14:31:16.678279 (+106034us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.680991 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1569) took 1257 ms. Trace:
W20260924 14:31:16.649211 31248 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260924 14:31:16.650452 26324 raft_consensus.cc:2379] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:16.559268 31213 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:16.686797 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.686995 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.687069 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:16.687948 26331 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:16.595691 31078 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:16.658200 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.635849 26405 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:31:16.659255 26249 rpcz_store.cc:276] 0924 14:31:15.310711 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.512951 (+1202240us) service_pool.cc:227] Handling call
0924 14:31:16.658844 (+145893us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.673427 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.673602 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.671815 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1570) took 1035 ms. Trace:
I20260924 14:31:16.628952 31223 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:16.675477 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.676916 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:16.660027 30787 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:16.681459 26251 rpcz_store.cc:276] 0924 14:31:15.423761 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.676673 (+1252912us) service_pool.cc:227] Handling call
0924 14:31:16.680910 (+  4237us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.722734 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:16.723623 26251 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:16.659730 31226 consensus_queue.cc:237] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:16.637445 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.634872 31254 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:16.612200 31218 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260924 14:31:16.665241 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1038) took 1316 ms. Trace:
I20260924 14:31:16.650158 31186 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:16.689890 26369 leader_election.cc:398] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14.
I20260924 14:31:16.738011 26369 leader_election.cc:401] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260924 14:31:16.687592 26253 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:16.740506 26328 raft_consensus.cc:2354] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14.
I20260924 14:31:16.741042 29682 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:16.741546 29682 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:16.738547 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50522.4us
I20260924 14:31:16.742340 29682 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:16.714483 31251 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:16.675208 31104 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:16.716670 27186 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:16.675062 30963 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:16.690961 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:16.716703 26404 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 6.
I20260924 14:31:16.718724 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:16.719436 26256 rpcz_store.cc:276] 0924 14:31:15.638429 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.663546 (+1025117us) service_pool.cc:227] Handling call
0924 14:31:16.671755 (+  8209us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.720340 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.725252 31262 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:16.727358 26400 raft_consensus.cc:2396] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260924 14:31:16.679037 31228 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:31:16.734164 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:16.792695 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.734189 26332 rpcz_store.cc:276] 0924 14:31:15.348644 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.449705 (+1101061us) service_pool.cc:227] Handling call
0924 14:31:16.665187 (+215482us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.737548 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1674) took 3824 ms. Trace:
I20260924 14:31:16.743402 31155 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:16.672688 31088 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:16.688174 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.674798 31173 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:16.691177 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 238142us
I20260924 14:31:16.852381 26369 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:16.750976 26295 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:16.854311 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 103432us
I20260924 14:31:16.757990 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1650) took 3536 ms. Trace:
I20260924 14:31:16.855079 26253 rpcz_store.cc:276] 0924 14:31:13.221516 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.757147 (+3535631us) service_pool.cc:227] Handling call
0924 14:31:16.757493 (+   346us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.757671 (+   178us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:16.757695 (+    24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:16.757734 (+    39us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:16.757765 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.757825 (+    60us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:16.757927 (+   102us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.759409 26224 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:31:16.768011 26325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 983) took 1682 ms (client timeout 1609 ms). Trace:
W20260924 14:31:16.856850 26325 rpcz_store.cc:269] 0924 14:31:15.085691 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.280727 (+195036us) service_pool.cc:227] Handling call
0924 14:31:16.767839 (+1487112us) spinlock_profiling.cc:243] Waited 60 ms on lock 0x7b54007d37d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:16.767946 (+   107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":138013440}
I20260924 14:31:16.771888 26251 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
W20260924 14:31:16.779876 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 88975.2us
I20260924 14:31:16.857860 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1664) took 1501 ms. Trace:
I20260924 14:31:16.858178 26251 rpcz_store.cc:276] 0924 14:31:15.356751 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.722394 (+1365643us) service_pool.cc:227] Handling call
0924 14:31:16.857792 (+135398us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.858577 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:16.859315 26251 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 5. Current term is 6. Ops: []
I20260924 14:31:16.775960 26249 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:16.864984 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1659) took 1653 ms. Trace:
I20260924 14:31:16.866243 26249 rpcz_store.cc:276] 0924 14:31:15.211702 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.718386 (+1506684us) service_pool.cc:227] Handling call
0924 14:31:16.864906 (+146520us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:16.794380 26257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1653) took 3556 ms (client timeout 1609 ms). Trace:
W20260924 14:31:16.867483 26257 rpcz_store.cc:269] 0924 14:31:13.237857 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:13.293000 (+ 55143us) service_pool.cc:227] Handling call
0924 14:31:16.794143 (+3501143us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b5400098654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:16.794279 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4696560384}
I20260924 14:31:16.683773 31111 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:16.813580 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1521) took 1831 ms. Trace:
I20260924 14:31:16.869047 26397 rpcz_store.cc:276] 0924 14:31:14.982117 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.859981 (+877864us) service_pool.cc:227] Handling call
0924 14:31:15.891215 (+ 31234us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:15.891757 (+   542us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:15.891804 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:15.892447 (+   643us) log.cc:842] Serialized 26 byte log entry
0924 14:31:16.257857 (+365410us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:16.257899 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.257975 (+    76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:16.813362 (+555387us) raft_consensus.cc:1671] finished
0924 14:31:16.813385 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:16.813516 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6273,"raft.queue_time_us":1020,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":30178,"thread_start_us":300421,"threads_started":1,"wal-append.queue_time_us":482404}
I20260924 14:31:16.785245 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:16.814549 26331 raft_consensus.cc:2362] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15. Re-sending same reply.
I20260924 14:31:16.815156 26398 rpcz_store.cc:276] 0924 14:31:12.907664 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.907795 (+   131us) service_pool.cc:227] Handling call
0924 14:31:12.942717 (+ 34922us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:13.953674 (+1010957us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.732011 (+2778337us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400336654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:16.732207 (+   196us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2304458624}
I20260924 14:31:16.871215 26404 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 16
W20260924 14:31:16.817533 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 258914us
I20260924 14:31:16.767318 31140 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:16.822854 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 314822us
I20260924 14:31:16.777724 26250 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:16.784566 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.880093 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1186) took 1112 ms. Trace:
I20260924 14:31:16.872860 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:16.880283 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.819137 26406 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:31:16.873172 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1167) took 3602 ms. Trace:
I20260924 14:31:16.880584 26250 rpcz_store.cc:276] 0924 14:31:15.768269 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.644184 (+875915us) service_pool.cc:227] Handling call
0924 14:31:16.880018 (+235834us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.881067 26251 rpcz_store.cc:276] 0924 14:31:13.270619 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.858820 (+3588201us) service_pool.cc:227] Handling call
0924 14:31:16.859152 (+   332us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.872928 (+ 13776us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.873120 (+   192us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.881918 26250 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260924 14:31:16.882206 26251 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
W20260924 14:31:16.872337 26298 leader_election.cc:334] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 1.521s (SENT)
I20260924 14:31:16.884171 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.884792 26253 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:16.885037 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.885421 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1652) took 3663 ms. Trace:
I20260924 14:31:16.885648 26400 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:16.885772 26253 rpcz_store.cc:276] 0924 14:31:13.222007 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.856579 (+3634572us) service_pool.cc:227] Handling call
0924 14:31:16.884602 (+ 28023us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.885187 (+   585us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.885371 (+   184us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.887173 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1172) took 3614 ms. Trace:
I20260924 14:31:16.887566 26251 rpcz_store.cc:276] 0924 14:31:13.272379 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.881787 (+3609408us) service_pool.cc:227] Handling call
0924 14:31:16.882060 (+   273us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.886971 (+  4911us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.887127 (+   156us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.888641 26251 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:16.889283 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1554) took 3616 ms. Trace:
I20260924 14:31:16.889623 26251 rpcz_store.cc:276] 0924 14:31:13.273071 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.888220 (+3615149us) service_pool.cc:227] Handling call
0924 14:31:16.888485 (+   265us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.889042 (+   557us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.889230 (+   188us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.891595 26327 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:16.891934 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:16.892266 26251 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:31:16.893060 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1654) took 3618 ms. Trace:
I20260924 14:31:16.893169 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:16.893445 26251 rpcz_store.cc:276] 0924 14:31:13.274719 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.891628 (+3616909us) service_pool.cc:227] Handling call
0924 14:31:16.891895 (+   267us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.892860 (+   965us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.893020 (+   160us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.894714 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:16.895550 26405 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:16.796900 31033 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:16.895344 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:16.895274 26251 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:16.793255 30963 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:16.898069 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.898463 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:16.899688 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1557) took 3616 ms. Trace:
I20260924 14:31:16.900082 26331 raft_consensus.cc:2362] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12. Re-sending same reply.
I20260924 14:31:16.900025 26251 rpcz_store.cc:276] 0924 14:31:13.283540 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.894798 (+3611258us) service_pool.cc:227] Handling call
0924 14:31:16.895059 (+   261us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.899469 (+  4410us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.899644 (+   175us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.897742 26332 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:16.901784 27185 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 8: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:16.794548 31260 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:16.902529 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:16.903421 26406 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:16.903940 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:16.902562 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:16.906178 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:16.908180 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 148854us
I20260924 14:31:16.910266 26224 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:16.910670 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:16.912535 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:16.913373 27181 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:31:16.917711 26224 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:16.914312 26253 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:16.911063 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:16.878254 31206 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:16.925354 26398 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:16.873404 31195 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:16.878197 31041 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:16.918849 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:16.919260 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1173) took 3646 ms. Trace:
I20260924 14:31:16.921357 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:16.916080 27191 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260924 14:31:16.926221 26224 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:16.950011 26253 rpcz_store.cc:276] 0924 14:31:13.272744 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.886843 (+3614099us) service_pool.cc:227] Handling call
0924 14:31:16.914144 (+ 27301us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.918945 (+  4801us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.919188 (+   243us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.952389 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:16.950402 26325 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:16.882510 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1170) took 3610 ms. Trace:
I20260924 14:31:16.897729 30795 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:16.902880 31262 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:31:16.721783 31256 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.239s
I20260924 14:31:16.915812 30974 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:16.738679 31254 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:16.956004 26250 rpcz_store.cc:276] 0924 14:31:13.271683 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.881383 (+3609700us) service_pool.cc:227] Handling call
0924 14:31:16.881706 (+   323us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:16.882269 (+   563us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.882463 (+   194us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:16.963060 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:16.987275 27181 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260924 14:31:16.988317 27186 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 13
I20260924 14:31:16.925060 30983 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:16.776176 31251 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:16.925485 31206 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:16.989977 29682 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:16.994702 26398 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:16.779282 31078 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:16.953894 31186 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:16.953676 31083 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:16.962025 31228 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:16.957553 30964 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:31:17.008780 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:16.925081 30963 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:16.986285 31262 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:17.010047 26331 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:17.059932 29682 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:16.744432 31244 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:16.998618 30795 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:17.026139 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 173868us
I20260924 14:31:16.943773 31260 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:16.992379 31264 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:31:16.958330 31195 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:17.077699 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:17.080214 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1033) took 1733 ms. Trace:
I20260924 14:31:16.954754 31265 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:17.083137 29682 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:17.084304 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:31:17.017532 30983 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:17.078531 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 69879.9us
I20260924 14:31:17.088007 26258 raft_consensus.cc:2396] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260924 14:31:17.087005 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:16.960374 31041 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:17.088955 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 167719us
I20260924 14:31:17.089177 26250 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:17.099781 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 195917us
I20260924 14:31:17.092342 27191 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:17.092720 26331 rpcz_store.cc:276] 0924 14:31:15.346949 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.908563 (+1561614us) service_pool.cc:227] Handling call
0924 14:31:17.078873 (+170310us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:17.135227 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 48552.3us
I20260924 14:31:17.079579 31078 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:17.065234 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1538) took 1008 ms. Trace:
I20260924 14:31:17.138418 26398 rpcz_store.cc:276] 0924 14:31:16.057096 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.954005 (+896909us) service_pool.cc:227] Handling call
0924 14:31:16.994263 (+ 40258us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.059418 (+ 65155us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.065169 (+  5751us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:17.139122 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:17.058466 31226 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:17.139806 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48511.5us
I20260924 14:31:17.139959 27191 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:17.140633 27125 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:17.093425 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1528) took 2083 ms. Trace:
I20260924 14:31:17.080616 30787 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.141360 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:17.136795 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:17.089874 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 194605us
I20260924 14:31:17.084802 31186 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:17.142048 26400 rpcz_store.cc:276] 0924 14:31:15.010097 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.901287 (+1891190us) service_pool.cc:227] Handling call
0924 14:31:16.926970 (+ 25683us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.927173 (+   203us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:16.986845 (+ 59672us) log.cc:842] Serialized 14 byte log entry
0924 14:31:17.093080 (+106235us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:17.093135 (+    55us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:17.093178 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.093245 (+    67us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:17.093359 (+   114us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":36270,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"thread_start_us":105913,"threads_started":1}
I20260924 14:31:17.083197 31228 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:17.144021 31004 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.699s	user 0.008s	sys 0.002s
I20260924 14:31:17.079726 31262 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:17.144171 26405 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260924 14:31:17.144277 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:17.061751 30974 consensus_queue.cc:1046] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260924 14:31:17.090585 30993 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260924 14:31:17.063494 30975 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:17.064543 31269 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:17.155849 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1662) took 1866 ms. Trace:
I20260924 14:31:17.192348 26250 rpcz_store.cc:276] 0924 14:31:15.289091 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.019941 (+1730850us) service_pool.cc:227] Handling call
0924 14:31:17.074061 (+ 54120us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.139584 (+ 65523us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.155741 (+ 16157us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:17.093961 31024 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.003s
I20260924 14:31:17.132952 30983 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:17.164382 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:17.200577 26398 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:17.165166 29682 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.203917 29682 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:17.172834 31111 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.173089 26405 pending_rounds.cc:76] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 5
I20260924 14:31:17.173659 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:17.174400 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:17.175920 30787 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:17.139350 31088 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.176864 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:17.134949 31041 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:17.180709 31263 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260924 14:31:17.180984 31148 raft_consensus.cc:3032] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:17.181960 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 833) took 1919 ms. Trace:
I20260924 14:31:17.182984 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 984) took 2096 ms. Trace:
I20260924 14:31:17.183804 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1651) took 3962 ms. Trace:
W20260924 14:31:17.184017 26254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1178) took 2991 ms (client timeout 1696 ms). Trace:
I20260924 14:31:17.185005 31278 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:31:17.185741 31261 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:17.185999 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 829) took 3139 ms. Trace:
I20260924 14:31:17.160795 31080 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:17.186990 31195 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.188400 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:17.184092 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:17.092535 31264 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:17.134768 31039 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:17.091737 30963 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:17.166193 31186 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:17.166704 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:17.165331 31270 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:17.243661 26405 pending_rounds.cc:97] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260924 14:31:17.178828 30993 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:17.167903 31210 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260924 14:31:17.130745 31206 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:17.133765 31265 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:17.156555 31226 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:17.179095 31229 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260924 14:31:17.206768 27182 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:17.336185 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1668) took 1883 ms. Trace:
W20260924 14:31:17.283115 27194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 874) took 2068 ms (client timeout 1908 ms). Trace:
I20260924 14:31:17.322978 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1016) took 5354 ms. Trace:
I20260924 14:31:17.321673 26404 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
W20260924 14:31:17.227517 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 90784.4us
W20260924 14:31:17.350616 26328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1548) took 2062 ms (client timeout 1608 ms). Trace:
W20260924 14:31:17.354216 26255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1171) took 4082 ms (client timeout 2229 ms). Trace:
I20260924 14:31:17.285048 26406 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260924 14:31:17.292557 26402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1688) took 1925 ms (client timeout 1521 ms). Trace:
I20260924 14:31:17.489040 26332 rpcz_store.cc:276] 0924 14:31:15.086157 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.851163 (+1765006us) service_pool.cc:227] Handling call
0924 14:31:17.182800 (+331637us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b54005bb3d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:17.182927 (+   127us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":11420416}
I20260924 14:31:17.177673 31173 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:17.489048 26405 raft_consensus.cc:2866] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:17.543795 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:17.488683 26329 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:17.545079 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:17.545279 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:17.542665 26398 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:31:17.547318 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:17.547396 26402 rpcz_store.cc:269] 0924 14:31:15.369174 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.392056 (+ 22882us) service_pool.cc:227] Handling call
0924 14:31:17.284518 (+1892462us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54006e78d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:17.292466 (+  7948us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2479518592}
W20260924 14:31:17.529469 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 364952us
I20260924 14:31:17.548731 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:17.491981 27194 rpcz_store.cc:269] 0924 14:31:15.215035 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.215253 (+   218us) service_pool.cc:227] Handling call
0924 14:31:17.282847 (+2067594us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b5400074c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:17.283021 (+   174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":54441344}
W20260924 14:31:17.494149 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 305840us
I20260924 14:31:17.553586 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:17.231890 27123 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:17.489862 27190 rpcz_store.cc:276] 0924 14:31:14.047039 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.720218 (+673179us) service_pool.cc:227] Handling call
0924 14:31:14.720701 (+   483us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:14.903066 (+182365us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:14.903099 (+    33us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:16.757997 (+1854898us) spinlock_profiling.cc:243] Waited 903 us on lock 0x7b2c001c06e8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:16.758624 (+   627us) log.cc:842] Serialized 52 byte log entry
0924 14:31:17.058824 (+300200us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:17.058872 (+    48us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.058951 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:17.185657 (+126706us) raft_consensus.cc:1671] finished
0924 14:31:17.185679 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:17.185941 (+   262us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":171655,"raft.queue_time_us":220557,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":2128,"spinlock_wait_cycles":2076032,"thread_start_us":261792,"threads_started":1,"wal-append.queue_time_us":404541}
I20260924 14:31:17.491074 27185 rpcz_store.cc:276] 0924 14:31:15.262415 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.843173 (+580758us) service_pool.cc:227] Handling call
0924 14:31:17.181782 (+1338609us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003c82d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:17.181905 (+   123us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2431794048}
I20260924 14:31:17.243590 27191 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
W20260924 14:31:17.335156 31032 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.000s
I20260924 14:31:17.572695 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:17.573055 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:17.574369 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:17.541069 31079 consensus_meta.cc:287] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.886s	user 0.008s	sys 0.002s
W20260924 14:31:17.357019 31258 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.708s	user 0.006s	sys 0.006s
W20260924 14:31:17.358788 31252 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.714s	user 0.001s	sys 0.000s
W20260924 14:31:17.342308 30842 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.897s	user 0.010s	sys 0.001s
I20260924 14:31:17.375814 31171 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.566730 31109 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:17.377429 31218 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:17.487869 27186 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:17.571460 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:17.549479 30739 consensus_meta.cc:287] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.982s	user 0.008s	sys 0.004s
W20260924 14:31:17.316934 31268 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260924 14:31:17.492966 31118 consensus_meta.cc:287] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.212s	user 0.012s	sys 0.000s
I20260924 14:31:17.289031 31285 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:17.317621 31272 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260924 14:31:17.325915 31274 log.cc:924] Time spent T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260924 14:31:17.577081 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:17.556818 31223 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.821s	user 0.009s	sys 0.003s
I20260924 14:31:17.570657 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:17.292994 31254 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:17.324957 30793 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:17.492925 31080 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:17.178025 31004 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:17.362522 31275 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
I20260924 14:31:17.329020 31251 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:17.333317 31071 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:17.330077 31120 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:17.589285 31226 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:17.327724 31194 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:17.603904 30964 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:17.587041 27185 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:17.419902 31176 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:17.194568 30975 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.339012 31158 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:17.621709 30964 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:17.599865 30739 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:17.617586 30963 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:17.625875 30739 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:17.491878 31270 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:17.626618 31270 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:17.617437 31194 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:17.628440 31194 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:17.629439 31194 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:17.492430 31186 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:17.615875 31120 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:31:17.631635 31104 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.832s	user 0.006s	sys 0.006s
I20260924 14:31:17.632066 31104 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.632375 31120 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:17.633404 31120 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:17.634076 31104 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:17.604749 31285 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:17.207607 31141 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:17.589380 31174 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:17.635710 31141 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:17.635979 31174 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:17.636298 31141 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:17.636483 31285 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:17.492653 31261 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.636907 31174 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.638808 31261 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:17.489619 31148 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:17.606794 31223 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:17.640250 31148 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.641307 31148 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:17.641453 31223 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.641954 31176 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.568650 31039 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:17.599061 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:17.643929 31176 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:17.644281 27194 raft_consensus.cc:2354] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8.
I20260924 14:31:17.331554 31196 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.206981 31269 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:17.645671 31269 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.646652 31004 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:17.647696 31155 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.797s	user 0.006s	sys 0.005s
I20260924 14:31:17.648142 31155 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14.
I20260924 14:31:17.494453 30993 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:17.649726 31023 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:17.650223 31023 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.496032 31264 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:17.652643 31265 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.560863 31195 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:17.566972 31206 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:17.566954 31147 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:17.246084 31033 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.354s
I20260924 14:31:17.610158 27191 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260924 14:31:17.588086 31079 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:17.589715 31109 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:17.609022 31080 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:17.590251 31218 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:17.602018 31288 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:17.283352 31111 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:17.681483 31111 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:17.322947 31262 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:17.681924 31111 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:17.683399 31262 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:17.682899 30983 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.660157 31147 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:17.685005 31147 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:17.685644 30983 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:17.614517 30795 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.004s
I20260924 14:31:17.577049 31032 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:17.686570 30795 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:17.687156 31033 consensus_queue.cc:1225] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer 6c22052275034c369b7f7dca086fbd1e log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260924 14:31:17.687868 31032 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:17.245117 31228 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.688179 31218 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.689441 31218 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:17.588778 30842 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:17.623023 31140 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.727s	user 0.009s	sys 0.002s
I20260924 14:31:17.210299 31277 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:17.695788 31277 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:17.696193 31277 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:17.697244 31159 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:17.697768 31159 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:17.678431 31109 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:17.698550 31079 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.691546 31288 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:17.709753 31111 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.735729 31186 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:17.738930 31186 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:17.748451 31270 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:17.750149 31288 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:17.750669 31270 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:17.787489 26251 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.007s	sys 0.007s
W20260924 14:31:17.788549 26251 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.008s	sys 0.007s
I20260924 14:31:17.790467 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1655) took 4498 ms. Trace:
I20260924 14:31:17.791098 26251 rpcz_store.cc:276] 0924 14:31:13.291884 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.901011 (+3609127us) service_pool.cc:227] Handling call
0924 14:31:16.901346 (+   335us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.902499 (+  1153us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:31:17.236263 (+333764us) log.cc:842] Serialized 14 byte log entry
0924 14:31:17.789416 (+553153us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:17.789567 (+   151us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:17.789754 (+   187us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.789972 (+   218us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:17.790324 (+   352us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":333856,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"thread_start_us":844411,"threads_started":2,"wal-append.queue_time_us":455753}
I20260924 14:31:17.795161 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.797363 26251 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:17.798194 26252 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:17.240378 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.799822 26250 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:31:17.801929 26221 leader_election.cc:334] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.554s (SENT)
W20260924 14:31:17.489152 26254 rpcz_store.cc:269] 0924 14:31:14.192361 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.307101 (+1114740us) service_pool.cc:227] Handling call
0924 14:31:17.183828 (+1876727us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b540070fb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:17.183946 (+   118us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":646350080}
W20260924 14:31:17.802011 26255 rpcz_store.cc:269] 0924 14:31:13.272011 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.810768 (+1538757us) service_pool.cc:227] Handling call
0924 14:31:17.353924 (+2543156us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b54000c26d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:17.354120 (+   196us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1899425664}
I20260924 14:31:17.803143 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:17.810253 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.810839 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.811474 26255 raft_consensus.cc:2362] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13. Re-sending same reply.
I20260924 14:31:17.812008 26254 raft_consensus.cc:2362] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14. Re-sending same reply.
W20260924 14:31:17.814646 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.011s	sys 0.007s
W20260924 14:31:17.815184 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.012s	sys 0.007s
I20260924 14:31:17.815376 26326 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:17.816184 26323 raft_consensus.cc:2379] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:17.813446 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:17.817239 26255 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:17.817528 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:17.818326 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.818707 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1043) took 2054 ms. Trace:
I20260924 14:31:17.819140 26332 rpcz_store.cc:276] 0924 14:31:15.764611 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.571254 (+1806643us) service_pool.cc:227] Handling call
0924 14:31:17.818540 (+247286us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:17.819881 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.820206 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:17.490039 26256 rpcz_store.cc:276] 0924 14:31:13.221746 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.776849 (+3555103us) service_pool.cc:227] Handling call
0924 14:31:16.777278 (+   429us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:16.783659 (+  6381us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:16.783705 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:16.813886 (+ 30181us) log.cc:842] Serialized 26 byte log entry
0924 14:31:17.130839 (+316953us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:17.130884 (+    45us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.130969 (+    85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:17.183588 (+ 52619us) raft_consensus.cc:1671] finished
0924 14:31:17.183611 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:17.183749 (+   138us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5772,"raft.queue_time_us":5943,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":118,"thread_start_us":180247,"threads_started":1,"wal-append.queue_time_us":278342}
I20260924 14:31:17.822563 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.822635 26295 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:17.820617 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:17.374482 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 990) took 2192 ms. Trace:
I20260924 14:31:17.824127 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1558) took 1062 ms. Trace:
W20260924 14:31:17.502734 26328 rpcz_store.cc:269] 0924 14:31:15.287820 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.407606 (+119786us) service_pool.cc:227] Handling call
0924 14:31:17.350309 (+1942703us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540012dc54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:17.350511 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3062587776}
I20260924 14:31:17.824637 26332 rpcz_store.cc:276] 0924 14:31:16.761229 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.820178 (+1058949us) service_pool.cc:227] Handling call
0924 14:31:17.824037 (+  3859us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:17.502053 26330 rpcz_store.cc:276] 0924 14:31:11.968768 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:12.890734 (+921966us) service_pool.cc:227] Handling call
0924 14:31:12.891072 (+   338us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:15.422465 (+2531393us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.322676 (+1900211us) spinlock_profiling.cc:243] Waited 2.52 s on lock 0x7b54007800d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:17.322889 (+   213us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5794426752}
I20260924 14:31:17.826032 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:17.827066 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 652677us
I20260924 14:31:17.813482 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:17.827981 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:17.829105 26330 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:17.828867 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:17.829418 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:17.822659 26331 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:17.824198 26325 rpcz_store.cc:276] 0924 14:31:15.182153 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.879105 (+1696952us) service_pool.cc:227] Handling call
0924 14:31:17.374322 (+495217us) spinlock_profiling.cc:243] Waited 69.4 ms on lock 0x7b540012e154. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:17.374423 (+   101us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":159572736}
I20260924 14:31:17.832706 26325 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:17.833395 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 991) took 2649 ms. Trace:
I20260924 14:31:17.833706 26325 rpcz_store.cc:276] 0924 14:31:15.183843 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.831908 (+2648065us) service_pool.cc:227] Handling call
0924 14:31:17.832396 (+   488us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.833127 (+   731us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.833310 (+   183us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:17.821557 26329 raft_consensus.cc:2396] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260924 14:31:17.834915 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:17.602331 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:17.826085 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:17.837337 26397 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.002s	sys 0.026s
W20260924 14:31:17.837989 26397 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.002s	sys 0.026s
I20260924 14:31:17.838901 26402 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:17.839601 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1535) took 1836 ms. Trace:
I20260924 14:31:17.839985 26325 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:17.840385 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1438) took 1739 ms. Trace:
I20260924 14:31:17.589675 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1527) took 2591 ms. Trace:
I20260924 14:31:17.491580 26258 rpcz_store.cc:276] 0924 14:31:15.452909 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.673107 (+1220198us) service_pool.cc:227] Handling call
0924 14:31:17.335949 (+662842us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b540002bf54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:17.336122 (+   173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":848315776}
I20260924 14:31:17.841349 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:17.841697 26403 rpcz_store.cc:276] 0924 14:31:14.998569 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.993317 (+994748us) service_pool.cc:227] Handling call
0924 14:31:15.993721 (+   404us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:15.994076 (+   355us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:15.994109 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:16.036122 (+ 42013us) log.cc:842] Serialized 26 byte log entry
0924 14:31:16.337870 (+301748us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:16.337929 (+    59us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:16.338041 (+   112us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:17.358092 (+1020051us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:17.589370 (+231278us) raft_consensus.cc:1671] finished
0924 14:31:17.589393 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:17.589622 (+   229us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2415,"raft.queue_time_us":11492,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":140,"spinlock_wait_cycles":71209728,"thread_start_us":278523,"threads_started":1,"wal-append.queue_time_us":585175}
I20260924 14:31:17.563691 26398 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:17.844714 26399 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
W20260924 14:31:17.588230 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 356435us
I20260924 14:31:17.601006 27126 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:17.886936 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 285856us
I20260924 14:31:17.592120 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:17.706001 30993 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:17.889508 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 297354us
I20260924 14:31:17.847357 31259 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:17.848721 27188 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:17.850785 30842 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:17.852031 31186 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:17.841012 26402 rpcz_store.cc:276] 0924 14:31:16.100668 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.837611 (+1736943us) service_pool.cc:227] Handling call
0924 14:31:17.838390 (+   779us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.839432 (+  1042us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.840317 (+   885us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:17.827164 26332 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:31:17.820770 26255 raft_consensus.cc:2362] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13. Re-sending same reply.
I20260924 14:31:17.830566 26254 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:17.862038 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:17.849190 31080 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:17.871949 31148 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:17.877265 31100 raft_consensus.cc:2741] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260924 14:31:17.876413 31264 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:17.878010 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1439) took 1776 ms. Trace:
I20260924 14:31:17.882848 31111 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:17.884537 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 72328.1us
I20260924 14:31:17.840169 26397 rpcz_store.cc:276] 0924 14:31:16.003790 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.900245 (+896455us) service_pool.cc:227] Handling call
0924 14:31:16.944284 (+ 44039us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:16.944541 (+   257us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:31:17.156361 (+211820us) log.cc:842] Serialized 14 byte log entry
0924 14:31:17.838884 (+682523us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:17.838998 (+   114us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:17.839160 (+   162us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.839284 (+   124us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:17.839476 (+   192us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":40329,"prepare.queue_time_us":115567,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"thread_start_us":847621,"threads_started":2,"wal-append.queue_time_us":471274}
I20260924 14:31:17.878091 26224 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:17.900871 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1555) took 1142 ms. Trace:
I20260924 14:31:17.871960 31120 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:17.910295 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.910730 31195 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:17.903859 31140 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:17.911882 31140 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:17.911664 31195 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:17.913659 31140 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:17.914523 31195 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:17.899151 31068 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:31:17.915942 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.916122 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:17.916458 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.916792 26255 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:17.917070 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:17.917483 26254 raft_consensus.cc:2362] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11. Re-sending same reply.
I20260924 14:31:17.918157 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:17.918529 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.916254 31068 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:17.915818 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:17.921847 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:17.921885 31068 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:17.923581 30993 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:17.924101 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:17.848716 31141 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:17.926221 31080 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.926497 31141 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.925453 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:17.928938 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:17.909952 26400 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:17.930197 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1540) took 1821 ms. Trace:
I20260924 14:31:17.930591 26400 rpcz_store.cc:276] 0924 14:31:16.108664 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.901610 (+1792946us) service_pool.cc:227] Handling call
0924 14:31:17.909671 (+  8061us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.929840 (+ 20169us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.930093 (+   253us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:17.910524 31120 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:17.901501 31260 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:17.933022 31120 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:17.933539 31260 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:17.918638 26256 raft_consensus.cc:2354] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8.
I20260924 14:31:17.909610 31264 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.935684 26329 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 6 FOLLOWER]: Advancing to term 7
I20260924 14:31:17.936565 31260 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.937772 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1697) took 1692 ms. Trace:
I20260924 14:31:17.939335 26400 rpcz_store.cc:276] 0924 14:31:16.245458 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.936387 (+1690929us) service_pool.cc:227] Handling call
0924 14:31:17.936864 (+   477us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.937372 (+   508us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:17.937405 (+    33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:17.937471 (+    66us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:17.937536 (+    65us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.937594 (+    58us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:17.937703 (+   109us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:17.938772 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:17.938194 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:17.942019 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:17.915818 31148 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:17.943583 26404 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:17.953893 31148 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.946713 31218 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:17.946429 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:17.955976 31218 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:17.956405 27194 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:17.957845 26403 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260924 14:31:17.958545 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1543) took 1101 ms. Trace:
I20260924 14:31:17.958912 26403 rpcz_store.cc:276] 0924 14:31:16.857607 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.957132 (+1099525us) service_pool.cc:227] Handling call
0924 14:31:17.957584 (+   452us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:17.958299 (+   715us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.958500 (+   201us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:17.955149 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:17.960163 26258 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:17.960459 26403 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:17.961066 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1544) took 1073 ms. Trace:
I20260924 14:31:17.961145 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:17.961403 26403 rpcz_store.cc:276] 0924 14:31:16.887696 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.959647 (+1071951us) service_pool.cc:227] Handling call
0924 14:31:17.960171 (+   524us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.960861 (+   690us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.961025 (+   164us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:17.962034 26258 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:17.963419 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:17.963750 31206 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:17.967690 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 89991.4us
I20260924 14:31:17.967949 31206 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:17.907956 26332 rpcz_store.cc:276] 0924 14:31:16.755086 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.825547 (+1070461us) service_pool.cc:227] Handling call
0924 14:31:17.897402 (+ 71855us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":17536}
I20260924 14:31:17.968881 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:17.970817 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 994) took 2785 ms. Trace:
I20260924 14:31:17.971818 31260 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:17.902372 26398 rpcz_store.cc:276] 0924 14:31:16.101013 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.876385 (+1775372us) service_pool.cc:227] Handling call
0924 14:31:17.877063 (+   678us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.877472 (+   409us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:17.877501 (+    29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:17.877544 (+    43us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:17.877638 (+    94us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.877702 (+    64us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:17.877840 (+   138us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:17.971001 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:17.976197 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:17.977439 26254 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:17.978617 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:17.979099 31264 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:17.980289 31264 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:17.941949 31277 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:17.985844 31264 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:17.971218 26332 rpcz_store.cc:276] 0924 14:31:15.185117 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.969342 (+2784225us) service_pool.cc:227] Handling call
0924 14:31:17.969815 (+   473us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.970411 (+   596us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:17.970436 (+    25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:17.970489 (+    53us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:17.970545 (+    56us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:17.970602 (+    57us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:17.970753 (+   151us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:17.995702 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 164396us
I20260924 14:31:17.993777 26250 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:31:17.995630 26255 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:17.998282 26251 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
W20260924 14:31:18.003073 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 67644.8us
I20260924 14:31:18.007475 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:17.999532 31206 raft_consensus.cc:2741] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260924 14:31:18.012315 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:18.018710 31277 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:18.012028 31141 consensus_queue.cc:1046] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260924 14:31:18.020630 31141 raft_consensus.cc:3032] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:18.021024 31141 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:18.027617 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:18.028422 26250 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:18.030328 26258 raft_consensus.cc:2396] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260924 14:31:18.030763 26397 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260924 14:31:18.036170 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:18.037194 31264 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:18.037485 26224 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.037899 31264 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:18.039532 31111 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:18.040819 31264 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:18.041716 31218 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:18.039958 30983 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:18.044351 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.046396 31260 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:18.046489 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:18.047657 30983 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:18.047227 31260 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:18.048229 30983 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:18.048772 30983 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:18.048858 31080 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:18.049553 31080 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:18.050170 31260 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:18.052024 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.053062 31186 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:18.053184 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:18.052276 31080 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:18.041376 31111 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:18.032392 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:18.062287 26329 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:31:18.060702 27194 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:18.065923 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:18.053256 26401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1434) took 3109 ms (client timeout 2031 ms). Trace:
I20260924 14:31:18.067287 31141 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:18.071517 31141 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:18.068876 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:18.074985 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:18.068629 26401 rpcz_store.cc:269] 0924 14:31:14.943827 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:14.943985 (+   158us) service_pool.cc:227] Handling call
0924 14:31:18.052906 (+3108921us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54001c1454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.053046 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3709883776}
I20260924 14:31:18.079936 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:18.081147 31080 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:18.081370 26298 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:18.081806 31080 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:18.083873 31217 raft_consensus.cc:2741] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260924 14:31:18.083994 31186 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:18.083168 31080 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.088850 26254 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:18.089805 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1666) took 2678 ms. Trace:
I20260924 14:31:18.090251 26254 rpcz_store.cc:276] 0924 14:31:15.411308 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.060582 (+2649274us) service_pool.cc:227] Handling call
0924 14:31:18.088510 (+ 27928us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.089472 (+   962us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.089713 (+   241us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.090879 31122 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:18.091624 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:18.063589 31111 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:18.095613 31004 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:18.096277 31004 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:18.097896 31004 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:18.101854 31023 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:18.103513 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 106099us
I20260924 14:31:18.104611 31147 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.105556 31211 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:18.106083 31211 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:18.083070 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.108943 26298 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.109977 31121 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:18.110715 31121 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:18.109741 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:18.111214 27185 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260924 14:31:18.112262 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:18.113664 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:18.114652 27190 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:18.112309 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:18.115911 27185 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:31:18.117842 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 85670.5us
I20260924 14:31:18.121222 31120 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.123482 31175 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:18.123920 31120 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:18.126542 27194 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:18.109767 31238 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 2.242s	user 0.000s	sys 0.004s
I20260924 14:31:18.127761 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:18.121410 31005 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:18.128783 26328 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:18.133162 26255 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 16
I20260924 14:31:18.111217 31121 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:18.148427 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:18.107318 31211 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:18.150172 31211 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:18.151376 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:18.151679 26329 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:18.151540 31121 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:18.152508 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1540) took 2966 ms. Trace:
I20260924 14:31:18.152845 26329 rpcz_store.cc:276] 0924 14:31:15.185892 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.066906 (+2881014us) service_pool.cc:227] Handling call
0924 14:31:18.133007 (+ 66101us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.152201 (+ 19194us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.152419 (+   218us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.154461 26329 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:18.155028 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1544) took 2939 ms. Trace:
I20260924 14:31:18.155694 31147 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:18.155894 31071 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:18.157337 31175 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:18.158058 31175 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.155345 26329 rpcz_store.cc:276] 0924 14:31:15.215692 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.153737 (+2938045us) service_pool.cc:227] Handling call
0924 14:31:18.154182 (+   445us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.154821 (+   639us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.154980 (+   159us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.161398 26255 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
W20260924 14:31:18.165565 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 56803us
I20260924 14:31:18.169754 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:18.171042 27125 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:18.173415 31107 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:18.175514 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 988) took 3088 ms. Trace:
I20260924 14:31:18.175976 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.176152 31175 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:18.176487 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:18.183012 31194 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.006s
I20260924 14:31:18.186043 31060 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:18.165828 31109 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.165773 26256 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260924 14:31:18.096767 31111 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:18.211048 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:18.211171 26256 pending_rounds.cc:76] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 7
I20260924 14:31:18.201535 26330 rpcz_store.cc:276] 0924 14:31:15.088350 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.829882 (+2741532us) service_pool.cc:227] Handling call
0924 14:31:17.898014 (+ 68132us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:17.903537 (+  5523us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:17.903581 (+    44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:17.904493 (+   912us) log.cc:842] Serialized 26 byte log entry
0924 14:31:17.996140 (+ 91647us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:17.996176 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.003605 (+  7429us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:18.173853 (+170248us) raft_consensus.cc:1671] finished
0924 14:31:18.173874 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:18.175215 (+  1341us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b54004100d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.175424 (+   209us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":16649,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":236,"spinlock_wait_cycles":153943936,"thread_start_us":90439,"threads_started":1,"wal-append.queue_time_us":211707}
I20260924 14:31:18.199976 31107 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:18.220451 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:18.224543 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:18.224704 31068 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:18.225595 30964 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:18.216614 26256 pending_rounds.cc:97] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260924 14:31:18.228338 26256 raft_consensus.cc:2866] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:18.222450 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:18.236502 26398 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:18.202833 31041 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:18.238209 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 67279.6us
I20260924 14:31:18.235090 26403 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:18.237116 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:18.243207 26397 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:18.242718 26255 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:18.242681 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:18.239956 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1572) took 1080 ms. Trace:
I20260924 14:31:18.229408 31107 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:18.254317 26256 rpcz_store.cc:276] 0924 14:31:17.159770 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.164849 (+1005079us) service_pool.cc:227] Handling call
0924 14:31:18.165306 (+   457us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.239368 (+ 74062us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.239878 (+   510us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:18.230501 26371 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.668s (SENT)
I20260924 14:31:18.239005 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1454) took 1598 ms. Trace:
W20260924 14:31:18.253628 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 86090.5us
I20260924 14:31:18.253464 27194 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
W20260924 14:31:18.257731 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45292.2us
I20260924 14:31:18.260932 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:18.259673 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.253558 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1681) took 1792 ms. Trace:
I20260924 14:31:18.262892 26255 rpcz_store.cc:276] 0924 14:31:16.460964 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.211537 (+1750573us) service_pool.cc:227] Handling call
0924 14:31:18.233967 (+ 22430us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.253204 (+ 19237us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.253492 (+   288us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:18.263615 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 78929.9us
I20260924 14:31:18.240610 31109 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:18.259235 26324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1036) took 2839 ms (client timeout 1554 ms). Trace:
I20260924 14:31:18.263244 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.266832 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:18.267791 31068 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:18.267917 31228 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
W20260924 14:31:18.268152 26326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1042) took 2503 ms (client timeout 2447 ms). Trace:
W20260924 14:31:18.270747 26324 rpcz_store.cc:269] 0924 14:31:15.348007 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.411071 (+ 63064us) service_pool.cc:227] Handling call
0924 14:31:18.186950 (+2775879us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54005bbdd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.187287 (+   337us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2281487744}
I20260924 14:31:18.272414 26329 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:18.276118 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:18.278074 26369 leader_election.cc:334] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 1.908s (SENT)
I20260924 14:31:18.282431 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1673) took 2476 ms. Trace:
W20260924 14:31:18.274366 26326 rpcz_store.cc:269] 0924 14:31:15.764292 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.661052 (+896760us) service_pool.cc:227] Handling call
0924 14:31:18.267880 (+1606828us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540012eb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.268067 (+   187us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2651134208}
I20260924 14:31:18.283880 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1543) took 3090 ms. Trace:
I20260924 14:31:18.285849 26256 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:18.285949 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:18.275302 31068 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:18.293897 26401 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:18.285146 26328 rpcz_store.cc:276] 0924 14:31:15.189579 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.147682 (+2958103us) service_pool.cc:227] Handling call
0924 14:31:18.230536 (+ 82854us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.274335 (+ 43799us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:18.274367 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:18.274416 (+    49us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:18.279337 (+  4921us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.279406 (+    69us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:18.279524 (+   118us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.283656 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:18.290194 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1573) took 1129 ms. Trace:
I20260924 14:31:18.284067 26251 rpcz_store.cc:276] 0924 14:31:15.802148 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.113616 (+2311468us) service_pool.cc:227] Handling call
0924 14:31:18.114100 (+   484us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.114482 (+   382us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:18.115435 (+   953us) log.cc:842] Serialized 14 byte log entry
0924 14:31:18.278178 (+162743us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:18.278232 (+    54us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:18.278270 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.278332 (+    62us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:18.278444 (+   112us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":246,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":71,"raft.queue_time_us":1630,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":3659,"thread_start_us":162183,"threads_started":1}
I20260924 14:31:18.295415 26256 rpcz_store.cc:276] 0924 14:31:17.160333 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.259542 (+1099209us) service_pool.cc:227] Handling call
0924 14:31:18.280417 (+ 20875us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.289916 (+  9499us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.290131 (+   215us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.291572 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:18.293290 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 845) took 1375 ms. Trace:
W20260924 14:31:18.297305 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 250942us
I20260924 14:31:18.297761 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.260145 26400 rpcz_store.cc:276] 0924 14:31:16.633079 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.952342 (+1319263us) service_pool.cc:227] Handling call
0924 14:31:17.952897 (+   555us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.953359 (+   462us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:31:18.007331 (+ 53972us) log.cc:842] Serialized 14 byte log entry
0924 14:31:18.216860 (+209529us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:18.216995 (+   135us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:18.230019 (+ 13024us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.230097 (+    78us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:18.231268 (+  1171us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":144583,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":11191,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":151,"thread_start_us":221252,"threads_started":2}
I20260924 14:31:18.299741 26324 raft_consensus.cc:2362] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14. Re-sending same reply.
I20260924 14:31:18.302976 26255 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260924 14:31:18.303366 26255 pending_rounds.cc:76] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 7
I20260924 14:31:18.304139 31039 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:18.305225 31039 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:18.296450 27181 rpcz_store.cc:276] 0924 14:31:16.917690 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.917938 (+   248us) service_pool.cc:227] Handling call
0924 14:31:16.918744 (+   806us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:17.006432 (+ 87688us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.292689 (+1286257us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b540070f8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:18.293109 (+   420us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":157021952}
I20260924 14:31:18.309585 26255 pending_rounds.cc:97] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260924 14:31:18.311003 31122 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:18.311657 26323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1004) took 2371 ms (client timeout 1934 ms). Trace:
I20260924 14:31:18.311836 31122 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:18.312230 26323 rpcz_store.cc:269] 0924 14:31:15.940274 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.443576 (+503302us) service_pool.cc:227] Handling call
0924 14:31:18.311183 (+1867607us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002765d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.311583 (+   400us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3131071360}
I20260924 14:31:18.310245 26255 raft_consensus.cc:2866] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:18.306428 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:18.305670 31039 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:18.314492 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:18.309585 26330 raft_consensus.cc:2396] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260924 14:31:18.288255 31194 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:18.313433 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:18.318008 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.318389 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.288957 31228 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:18.332787 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.324016 26323 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
W20260924 14:31:18.334826 26224 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 1.576s (SENT)
I20260924 14:31:18.337878 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:18.320140 31147 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:31:18.345682 30975 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:18.343310 31039 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.347872 31068 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.349859 31068 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:18.351852 26397 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:18.351992 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:18.348603 31039 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:18.354849 26402 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:35444
I20260924 14:31:18.358086 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1703) took 1219 ms. Trace:
I20260924 14:31:18.358036 31122 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:18.358898 27189 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 836) took 3077 ms (client timeout 1576 ms). Trace:
I20260924 14:31:18.360692 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1561) took 3211 ms. Trace:
W20260924 14:31:18.361593 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 104466us
I20260924 14:31:18.361357 31228 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:18.330065 30964 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:18.362394 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1537) took 2311 ms. Trace:
I20260924 14:31:18.362728 26326 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:18.363647 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 150492us
W20260924 14:31:18.363768 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 92458.9us
I20260924 14:31:18.338907 26328 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:18.365109 26402 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:31:18.375882 26402 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
W20260924 14:31:18.356974 31306 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.200s	user 0.000s	sys 0.002s
I20260924 14:31:18.366626 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.362314 30765 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:18.363627 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1351) took 1167 ms. Trace:
I20260924 14:31:18.365684 26404 rpcz_store.cc:276] 0924 14:31:17.139008 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.544808 (+405800us) service_pool.cc:227] Handling call
0924 14:31:18.357831 (+813023us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b540023f854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.358006 (+   175us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":689355520}
I20260924 14:31:18.380076 26323 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:18.374290 26405 rpcz_store.cc:276] 0924 14:31:16.051053 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.916377 (+865324us) service_pool.cc:227] Handling call
0924 14:31:16.916628 (+   251us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.544018 (+627390us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.361876 (+817858us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54001f5ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:18.362199 (+   323us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":522380544}
I20260924 14:31:18.371695 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.376744 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:18.368530 27189 rpcz_store.cc:269] 0924 14:31:15.281004 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.396462 (+115458us) service_pool.cc:227] Handling call
0924 14:31:18.358458 (+2961996us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002c51d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.358669 (+   211us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3238817152}
I20260924 14:31:18.378749 31073 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:18.389535 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.379046 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:18.383594 27182 rpcz_store.cc:276] 0924 14:31:17.196259 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.196935 (+   676us) service_pool.cc:227] Handling call
0924 14:31:17.197321 (+   386us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.496782 (+299461us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.363111 (+866329us) spinlock_profiling.cc:243] Waited 7.01 ms on lock 0x7b54002008d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:18.363399 (+   288us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":16126336}
I20260924 14:31:18.384756 26403 raft_consensus.cc:2379] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:18.390045 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1680) took 1929 ms. Trace:
I20260924 14:31:18.386102 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:18.384728 26371 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.696s (SENT)
I20260924 14:31:18.397810 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.397732 27181 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:18.382810 31148 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:18.397321 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 998) took 3038 ms. Trace:
I20260924 14:31:18.402642 26323 rpcz_store.cc:276] 0924 14:31:15.355742 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.352156 (+2996414us) service_pool.cc:227] Handling call
0924 14:31:18.379798 (+ 27642us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.392514 (+ 12716us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.392976 (+   462us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.401751 26330 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:18.403863 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.392171 30765 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:18.408958 30765 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:18.409353 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1000) took 3052 ms. Trace:
I20260924 14:31:18.382988 31147 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:18.402356 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 49977.1us
I20260924 14:31:18.411230 26330 rpcz_store.cc:276] 0924 14:31:15.356793 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.405342 (+3048549us) service_pool.cc:227] Handling call
0924 14:31:18.408508 (+  3166us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.408872 (+   364us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:18.408906 (+    34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:18.408982 (+    76us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:18.409066 (+    84us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.409130 (+    64us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:18.409261 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.390952 26250 rpcz_store.cc:276] 0924 14:31:16.460574 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.160112 (+1699538us) service_pool.cc:227] Handling call
0924 14:31:18.219568 (+ 59456us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.237092 (+ 17524us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:31:18.264646 (+ 27554us) log.cc:842] Serialized 14 byte log entry
0924 14:31:18.389543 (+124897us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:18.389617 (+    74us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:18.389680 (+    63us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.389776 (+    96us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:18.389970 (+   194us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":23035,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":114,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3012,"thread_start_us":108150,"threads_started":1}
I20260924 14:31:18.425414 26404 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:18.427739 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.368932 26253 rpcz_store.cc:276] 0924 14:31:15.149302 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.957336 (+1808034us) service_pool.cc:227] Handling call
0924 14:31:16.989252 (+ 31916us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:17.145937 (+156685us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:17.145968 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:17.161541 (+ 15573us) log.cc:842] Serialized 26 byte log entry
0924 14:31:17.240974 (+ 79433us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:17.323127 (+ 82153us) log.cc:842] Serialized 14 byte log entry
0924 14:31:17.323684 (+   557us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.359930 (+1036246us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002882d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:18.359998 (+    68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:18.360069 (+    71us) raft_consensus.cc:1671] finished
0924 14:31:18.360216 (+   147us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:18.360467 (+   251us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44092,"prepare.queue_time_us":299655,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4536,"raft.queue_time_us":49861,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":7461,"spinlock_wait_cycles":257825536,"thread_start_us":53216,"threads_started":1,"wal-append.queue_time_us":270558}
W20260924 14:31:18.435743 31309 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.002s
W20260924 14:31:18.440364 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 88469.1us
I20260924 14:31:18.440212 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.453037 26323 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:18.453341 31005 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election won for term 16
W20260924 14:31:18.453496 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 74559.3us
I20260924 14:31:18.453801 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 999) took 3097 ms. Trace:
I20260924 14:31:18.452164 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:18.455278 30972 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:18.455742 31073 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:31:18.459949 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:18.460326 31314 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260924 14:31:18.461120 26255 raft_consensus.cc:2396] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260924 14:31:18.454052 31005 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:18.462921 26255 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260924 14:31:18.462548 31005 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:18.463738 31266 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:18.464174 31266 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:18.465116 31266 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:18.467242 31269 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:18.467682 27123 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:18.467892 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:18.468751 31269 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:18.469153 31269 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:18.469563 31269 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:18.471582 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:18.471709 31174 consensus_queue.cc:1046] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:18.472692 31229 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:18.473107 31229 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:18.473169 31174 raft_consensus.cc:3032] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:18.473572 31229 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:18.476135 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.476954 26405 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:18.477243 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:18.477859 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.473733 31174 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:18.479055 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.479851 31174 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:18.498703 26256 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260924 14:31:18.499360 26256 pending_rounds.cc:76] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 8
I20260924 14:31:18.499652 26256 pending_rounds.cc:97] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260924 14:31:18.486447 26405 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:18.499948 26256 raft_consensus.cc:2866] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:18.479259 31178 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:18.481804 31229 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:18.441339 26371 leader_election.cc:334] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.731s (SENT)
I20260924 14:31:18.454196 26323 rpcz_store.cc:276] 0924 14:31:15.356422 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.403760 (+3047338us) service_pool.cc:227] Handling call
0924 14:31:18.452816 (+ 49056us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.453563 (+   747us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.453748 (+   185us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.441565 26400 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:18.484452 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:18.498319 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 81270.1us
W20260924 14:31:18.502990 26297 leader_election.cc:334] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.729s (SENT)
I20260924 14:31:18.513427 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:18.518560 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.514230 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:18.513216 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:18.523687 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.524272 26403 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 7.
I20260924 14:31:18.524498 26405 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:18.426874 31147 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.530107 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:18.530928 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1346) took 1821 ms. Trace:
I20260924 14:31:18.512660 31107 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:18.536803 26323 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:18.532433 27188 rpcz_store.cc:276] 0924 14:31:16.709326 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.709569 (+   243us) service_pool.cc:227] Handling call
0924 14:31:16.710036 (+   467us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:17.905160 (+1195124us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.519679 (+614519us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540031f954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:18.530861 (+ 11182us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2616964992}
I20260924 14:31:18.538231 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.545601 26253 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:18.547124 26405 raft_consensus.cc:1261] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:31:18.529196 31277 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:18.552083 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1002) took 2617 ms. Trace:
I20260924 14:31:18.557406 26323 rpcz_store.cc:276] 0924 14:31:15.934308 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.529641 (+2595333us) service_pool.cc:227] Handling call
0924 14:31:18.536634 (+  6993us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.546731 (+ 10097us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.552007 (+  5276us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.561133 26403 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:31:18.540102 31174 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:18.569247 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 55935.2us
I20260924 14:31:18.550179 30765 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:18.575093 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:18.580596 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1168) took 5260 ms. Trace:
W20260924 14:31:18.577469 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 93120.3us
I20260924 14:31:18.581430 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:18.525730 31178 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260924 14:31:18.572252 31148 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:18.580385 31211 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260924 14:31:18.581396 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50622.8us
I20260924 14:31:18.581554 31079 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:18.529228 30983 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:18.569435 31080 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:18.582947 31269 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:18.583840 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 80973.7us
I20260924 14:31:18.588478 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:18.589289 26323 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:18.587893 31073 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:18.593233 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:18.552757 31120 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260924 14:31:18.617441 26325 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260924 14:31:18.608068 26297 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.883s (SENT)
I20260924 14:31:18.618358 26297 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:18.606021 31269 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:18.606241 31060 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:18.599927 31079 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:18.606953 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:18.607163 30983 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:18.608047 26258 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260924 14:31:18.609059 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1556) took 1853 ms. Trace:
I20260924 14:31:18.608695 31257 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:18.610180 31107 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:18.609952 31073 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.616233 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1693) took 2609 ms. Trace:
W20260924 14:31:18.637883 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 216777us
I20260924 14:31:18.586412 26249 rpcz_store.cc:276] 0924 14:31:13.271028 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.867881 (+3596853us) service_pool.cc:227] Handling call
0924 14:31:16.868248 (+   367us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:16.899187 (+ 30939us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:16.899216 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:16.899648 (+   432us) log.cc:842] Serialized 26 byte log entry
0924 14:31:17.982792 (+1083144us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b180048f4c0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:31:18.163032 (+180240us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:18.163098 (+    66us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.163209 (+   111us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:18.504031 (+340822us) raft_consensus.cc:1671] finished
0924 14:31:18.504052 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:18.531257 (+ 27205us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":59706,"raft.queue_time_us":35918,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":18991,"spinlock_wait_cycles":2798336,"thread_start_us":95754,"threads_started":1,"wal-append.queue_time_us":1441802}
I20260924 14:31:18.595103 31035 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260924 14:31:18.625464 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 993) took 3435 ms. Trace:
I20260924 14:31:18.640581 26325 rpcz_store.cc:276] 0924 14:31:15.184857 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.841156 (+2656299us) service_pool.cc:227] Handling call
0924 14:31:18.595881 (+754725us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:31:18.596125 (+   244us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.618151 (+ 22026us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.619849 (+  1698us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":42578304}
I20260924 14:31:18.629235 31279 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:18.632869 26323 rpcz_store.cc:276] 0924 14:31:16.755909 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.584583 (+1828674us) service_pool.cc:227] Handling call
0924 14:31:18.589019 (+  4436us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.608794 (+ 19775us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.609001 (+   207us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.643177 27188 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260924 14:31:18.643548 26252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1560) took 3511 ms (client timeout 1911 ms). Trace:
W20260924 14:31:18.643780 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 55557.6us
W20260924 14:31:18.644004 26252 rpcz_store.cc:269] 0924 14:31:15.131862 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.002142 (+870280us) service_pool.cc:227] Handling call
0924 14:31:18.637960 (+2635818us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54007b07d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.643456 (+  5496us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3895786752}
I20260924 14:31:18.645483 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:18.645666 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:18.646828 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:18.629383 31269 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260924 14:31:18.643221 31122 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260924 14:31:18.648330 31269 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:18.648753 31269 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:18.648932 31073 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:18.638257 26406 rpcz_store.cc:276] 0924 14:31:16.006274 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.904087 (+897813us) service_pool.cc:227] Handling call
0924 14:31:17.284658 (+380571us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:17.542661 (+258003us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.615867 (+1073206us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54006e78d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.616040 (+   173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":782084864}
I20260924 14:31:18.649878 31269 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.650465 26257 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:18.650489 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:18.651252 26252 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:18.592170 31121 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.626817 31079 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:18.663476 26404 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
W20260924 14:31:18.665187 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 78152.3us
W20260924 14:31:18.660965 31319 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.157s	user 0.001s	sys 0.001s
I20260924 14:31:18.676988 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:18.670295 31141 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:31:18.662653 30983 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:18.684904 30983 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:18.685437 30983 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:18.662461 31120 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:18.676666 26369 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:18.685829 31279 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:31:18.687563 31184 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:18.690289 26252 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:18.680177 31269 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:18.695850 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:18.678014 31257 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader election won for term 7
I20260924 14:31:18.676666 31211 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:18.679076 31229 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:18.707681 26325 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:18.709208 31257 raft_consensus.cc:686] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:18.710484 31257 consensus_queue.cc:237] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:18.712101 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 1341) took 2176 ms. Trace:
I20260924 14:31:18.712769 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 995) took 3527 ms. Trace:
I20260924 14:31:18.713481 26332 rpcz_store.cc:276] 0924 14:31:15.185374 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.994989 (+2809615us) service_pool.cc:227] Handling call
0924 14:31:18.061363 (+ 66374us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:18.127704 (+ 66341us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:18.127727 (+    23us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:18.128299 (+   572us) log.cc:842] Serialized 26 byte log entry
0924 14:31:18.392776 (+264477us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:18.392831 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.392936 (+   105us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:18.712534 (+319598us) raft_consensus.cc:1671] finished
0924 14:31:18.712555 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:18.712704 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":259001,"threads_started":1,"wal-append.queue_time_us":372873}
I20260924 14:31:18.712581 27192 rpcz_store.cc:276] 0924 14:31:16.535807 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.537941 (+  2134us) service_pool.cc:227] Handling call
0924 14:31:18.711689 (+2173748us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540000cb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.711895 (+   206us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2828801920}
W20260924 14:31:18.717957 27186 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 842) took 1807 ms (client timeout 1843 ms). Trace:
W20260924 14:31:18.718499 27186 rpcz_store.cc:269] 0924 14:31:16.910246 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.910421 (+   175us) service_pool.cc:227] Handling call
0924 14:31:18.717362 (+1806941us) spinlock_profiling.cc:243] Waited 76.9 ms on lock 0x7b540079f9d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.717748 (+   386us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":176887680}
I20260924 14:31:18.719033 31264 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:18.719981 31264 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:18.720985 26249 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:31:18.722076 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1557) took 1965 ms. Trace:
I20260924 14:31:18.722297 30765 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:18.722560 26325 rpcz_store.cc:276] 0924 14:31:16.756907 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.658264 (+1901357us) service_pool.cc:227] Handling call
0924 14:31:18.692121 (+ 33857us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.717323 (+ 25202us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.722008 (+  4685us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.722994 30765 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:18.723662 26323 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:18.724089 31211 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:18.725889 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1169) took 5454 ms. Trace:
I20260924 14:31:18.726266 31211 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:18.726303 26257 rpcz_store.cc:276] 0924 14:31:13.271349 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.868252 (+3596903us) service_pool.cc:227] Handling call
0924 14:31:18.650140 (+1781888us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.651069 (+   929us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.725671 (+ 74602us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54007b07d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.725827 (+   156us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4083970304}
W20260924 14:31:18.720690 26327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1010) took 2180 ms (client timeout 1644 ms). Trace:
I20260924 14:31:18.737699 26332 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:18.736110 26327 rpcz_store.cc:269] 0924 14:31:16.540103 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:16.623779 (+ 83676us) service_pool.cc:227] Handling call
0924 14:31:18.720352 (+2096573us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54007cfbd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:18.720505 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":521540864}
I20260924 14:31:18.706993 31147 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:18.706414 31194 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:18.740526 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.745321 26406 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:18.748894 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:18.699239 31178 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260924 14:31:18.749802 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 80968.8us
I20260924 14:31:18.747828 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.752573 26400 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
W20260924 14:31:18.753888 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 115733us
I20260924 14:31:18.755479 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1560) took 1988 ms. Trace:
I20260924 14:31:18.745169 31121 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:18.757198 26332 rpcz_store.cc:276] 0924 14:31:16.767278 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.714912 (+1947634us) service_pool.cc:227] Handling call
0924 14:31:18.715343 (+   431us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.755170 (+ 39827us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.755402 (+   232us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.762547 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.760353 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:18.755040 31122 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:18.767068 26400 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:18.768095 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:18.769321 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.770228 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1669) took 3316 ms. Trace:
I20260924 14:31:18.770658 26258 rpcz_store.cc:276] 0924 14:31:15.453381 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.064631 (+2611250us) service_pool.cc:227] Handling call
0924 14:31:18.147836 (+ 83205us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:18.634603 (+486767us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.769945 (+135342us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400715f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:18.770157 (+   212us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1057422208}
I20260924 14:31:18.747951 31035 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260924 14:31:18.772341 31080 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:18.772733 31080 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:18.773656 31080 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:18.774930 26298 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:18.769227 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 161197us
W20260924 14:31:18.779103 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 45538.1us
I20260924 14:31:18.788224 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:18.781085 26406 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:18.795966 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.800505 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.802484 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.767632 31122 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:18.779743 31229 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:18.810088 26332 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260924 14:31:18.811794 31080 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:18.813709 31004 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:18.806381 30765 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:18.816874 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:18.817268 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 56190.4us
I20260924 14:31:18.817679 26403 raft_consensus.cc:2379] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:18.818616 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:18.820528 30964 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:18.824216 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1052) took 1899 ms. Trace:
I20260924 14:31:18.824717 26332 rpcz_store.cc:276] 0924 14:31:16.925080 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.765244 (+1840164us) service_pool.cc:227] Handling call
0924 14:31:18.809785 (+ 44541us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:18.822196 (+ 12411us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.824145 (+  1949us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.823000 31121 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260924 14:31:18.842708 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 61192.6us
W20260924 14:31:18.810271 26297 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.789s (SENT) [suppressed 47 similar messages]
I20260924 14:31:18.821034 30964 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:18.820999 31147 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260924 14:31:18.821794 31229 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:18.808007 31264 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:18.854029 26406 raft_consensus.cc:3032] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:18.863222 26406 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:18.855208 27186 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:18.846726 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:18.855957 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51464.7us
I20260924 14:31:18.857144 26402 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.864523 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:18.871250 26402 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:18.843883 31330 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260924 14:31:18.875991 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 80162.9us
I20260924 14:31:18.836447 31107 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260924 14:31:18.891611 26403 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260924 14:31:18.874993 26297 leader_election.cc:334] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.911s (SENT)
I20260924 14:31:18.892442 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:18.877789 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:18.891155 26406 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:18.863590 31229 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:18.868319 31194 raft_consensus.cc:2741] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260924 14:31:18.896564 26332 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260924 14:31:18.895220 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1001) took 3538 ms. Trace:
I20260924 14:31:18.898795 26406 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:18.899173 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:35444 (request call id 1541) took 2740 ms. Trace:
I20260924 14:31:18.903748 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1053) took 1893 ms. Trace:
I20260924 14:31:18.876354 31330 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:18.905658 26402 rpcz_store.cc:276] 0924 14:31:16.171649 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.354559 (+2182910us) service_pool.cc:227] Handling call
0924 14:31:18.872204 (+517645us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.906373 26332 rpcz_store.cc:276] 0924 14:31:17.010356 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.825578 (+1815222us) service_pool.cc:227] Handling call
0924 14:31:18.868157 (+ 42579us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:18.902347 (+ 34190us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.903689 (+  1342us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:18.859205 30975 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260924 14:31:18.906347 26330 rpcz_store.cc:276] 0924 14:31:15.357634 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.423678 (+3066044us) service_pool.cc:227] Handling call
0924 14:31:18.513126 (+ 89448us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.513535 (+   409us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:31:18.663756 (+150221us) log.cc:842] Serialized 14 byte log entry
0924 14:31:18.871428 (+207672us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:18.871519 (+    91us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:18.893692 (+ 22173us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.895005 (+  1313us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:18.895150 (+   145us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":262384,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":3371,"raft.queue_time_us":33120,"thread_start_us":324459,"threads_started":2}
I20260924 14:31:18.912495 26257 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
W20260924 14:31:18.913039 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48647.5us
I20260924 14:31:18.948588 26257 pending_rounds.cc:76] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 8
I20260924 14:31:18.949007 26257 pending_rounds.cc:97] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260924 14:31:18.949272 26257 raft_consensus.cc:2866] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:18.898615 31228 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
W20260924 14:31:18.924161 31325 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260924 14:31:18.915413 31264 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:18.915074 31211 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:18.964840 26400 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:18.910111 31330 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:18.923259 31328 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260924 14:31:18.958380 31324 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.201s	user 0.001s	sys 0.001s
I20260924 14:31:18.897569 31122 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:18.989748 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 116653us
W20260924 14:31:18.963240 31327 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260924 14:31:19.002068 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 44266.6us
I20260924 14:31:18.997087 31329 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:18.996937 31211 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:19.002801 31147 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:19.040237 26332 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260924 14:31:19.028558 31039 consensus_meta.cc:287] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.652s	user 0.013s	sys 0.000s
W20260924 14:31:19.056285 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 178581us
I20260924 14:31:19.056279 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1055) took 2043 ms. Trace:
I20260924 14:31:19.056658 26401 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:19.056421 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1548) took 1507 ms. Trace:
I20260924 14:31:19.011825 31328 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:19.060811 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.066658 26328 raft_consensus.cc:2379] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:19.053903 26331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1047) took 2141 ms (client timeout 1843 ms). Trace:
W20260924 14:31:19.067166 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 192201us
I20260924 14:31:19.062186 26398 rpcz_store.cc:276] 0924 14:31:17.549265 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.974863 (+425598us) service_pool.cc:227] Handling call
0924 14:31:17.975425 (+   562us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.250704 (+275279us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:19.056178 (+805474us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400290c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:19.056360 (+   182us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":571036928}
I20260924 14:31:19.057081 26332 rpcz_store.cc:276] 0924 14:31:17.012882 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.958474 (+1945592us) service_pool.cc:227] Handling call
0924 14:31:19.009244 (+ 50770us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:19.054230 (+ 44986us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:19.056219 (+  1989us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:19.057650 26405 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
W20260924 14:31:19.067552 26331 rpcz_store.cc:269] 0924 14:31:16.912387 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:17.134543 (+222156us) service_pool.cc:227] Handling call
0924 14:31:19.017913 (+1883370us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54007d37d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:19.053827 (+ 35914us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":759407488}
I20260924 14:31:19.000177 31264 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:19.060811 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:19.039966 31259 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:19.124963 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 296011us
I20260924 14:31:19.083748 31335 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:19.133648 27181 raft_consensus.cc:2396] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260924 14:31:19.134377 26331 raft_consensus.cc:1261] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260924 14:31:19.139878 26328 raft_consensus.cc:1261] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:31:19.137408 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:19.144845 26406 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260924 14:31:19.052347 31178 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260924 14:31:19.057134 31211 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:19.038820 31079 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:19.167588 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:19.139016 31259 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:19.173205 26255 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:19.145375 26406 pending_rounds.cc:76] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 7
I20260924 14:31:19.157850 26329 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260924 14:31:19.174660 26406 pending_rounds.cc:97] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260924 14:31:19.175063 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1056) took 1342 ms. Trace:
I20260924 14:31:19.175441 26329 rpcz_store.cc:276] 0924 14:31:17.832375 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:19.074593 (+1242218us) service_pool.cc:227] Handling call
0924 14:31:19.144540 (+ 69947us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:19.174771 (+ 30231us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:19.175008 (+   237us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:19.128902 31147 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:31:19.171219 26399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1691) took 3761 ms (client timeout 2436 ms). Trace:
I20260924 14:31:19.159179 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:19.179133 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:19.155566 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 69961.1us
I20260924 14:31:19.177049 26250 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:19.180264 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.180923 26332 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:19.175354 26406 raft_consensus.cc:2866] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:19.167290 31279 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260924 14:31:19.182863 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.183729 31159 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:19.184496 31328 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:19.185078 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1051) took 2262 ms. Trace:
I20260924 14:31:19.185546 26327 rpcz_store.cc:276] 0924 14:31:16.922934 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.754686 (+1831752us) service_pool.cc:227] Handling call
0924 14:31:18.758894 (+  4208us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:18.775247 (+ 16353us) raft_consensus.cc:1520] Early marking committed up to index 5
0924 14:31:18.775297 (+    50us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:18.822273 (+ 46976us) log.cc:842] Serialized 26 byte log entry
0924 14:31:18.989392 (+167119us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:18.989440 (+    48us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.989540 (+   100us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:19.184842 (+195302us) raft_consensus.cc:1671] finished
0924 14:31:19.184866 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:19.185014 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":53321,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":18018,"thread_start_us":120745,"threads_started":1,"wal-append.queue_time_us":153298}
W20260924 14:31:19.191854 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 57682.9us
W20260924 14:31:19.006122 31121 consensus_queue.cc:1159] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e09fbebe29a49d291d23f9d395fe335" responder_term: 13 status { last_received { term: 12 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:31:19.196501 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:19.197453 31211 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:19.198212 31211 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:19.198704 31211 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:19.199054 31329 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:31:19.198659 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:19.183840 26329 raft_consensus.cc:2354] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14.
I20260924 14:31:19.200960 31264 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:19.201443 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:19.202838 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.203603 31335 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:19.204089 31107 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:19.204540 31329 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:19.204977 30274 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:31:19.205370 31335 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:31:19.205930 30983 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.520s	user 0.013s	sys 0.000s
I20260924 14:31:19.200008 31229 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:19.174669 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.206390 30983 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:19.179359 26399 rpcz_store.cc:269] 0924 14:31:15.410256 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:15.684196 (+273940us) service_pool.cc:227] Handling call
0924 14:31:19.160458 (+3476262us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54001e4bd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:19.171113 (+ 10655us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4715548928}
W20260924 14:31:19.208081 31322 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.523s	user 0.000s	sys 0.002s
I20260924 14:31:19.212359 26328 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 7: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:19.206651 31335 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:19.218449 26252 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260924 14:31:19.181867 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:19.211233 31068 consensus_meta.cc:287] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.861s	user 0.011s	sys 0.005s
I20260924 14:31:19.235357 26255 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:19.214876 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:19.187383 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:19.225901 31334 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.118s	user 0.002s	sys 0.005s
W20260924 14:31:19.226622 31338 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260924 14:31:19.226724 31345 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:19.227054 31174 consensus_meta.cc:287] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.641s	user 0.006s	sys 0.006s
W20260924 14:31:19.228047 31332 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260924 14:31:19.227109 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.229207 31041 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:19.229485 31229 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:19.229568 31336 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260924 14:31:19.291630 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:19.183997 31340 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260924 14:31:19.231762 31341 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:19.185115 31328 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:19.241936 26327 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:19.240578 26400 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:19.302995 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:19.204121 31321 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.632s	user 0.004s	sys 0.000s
I20260924 14:31:19.241712 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1559) took 2477 ms. Trace:
I20260924 14:31:19.242228 26331 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:19.250803 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:19.238636 26298 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:19.305316 26323 rpcz_store.cc:276] 0924 14:31:16.764048 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.643795 (+1879747us) service_pool.cc:227] Handling call
0924 14:31:18.723463 (+ 79668us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.724072 (+   609us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:19.241498 (+517426us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b54007cfbd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:19.241647 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":182251904}
W20260924 14:31:19.230123 31333 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260924 14:31:19.291786 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.318061 26329 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:19.318740 27181 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:19.297556 27182 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:19.251436 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:19.306890 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50456.5us
W20260924 14:31:19.315343 31080 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260924 14:31:19.256104 30964 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:19.304956 31211 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:19.310622 26324 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:19.236032 31329 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:19.316033 31328 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:19.324501 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1687) took 1436 ms. Trace:
I20260924 14:31:19.350761 27181 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8.
I20260924 14:31:19.350657 26251 rpcz_store.cc:276] 0924 14:31:17.887877 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.300946 (+413069us) service_pool.cc:227] Handling call
0924 14:31:18.319988 (+ 19042us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:18.341506 (+ 21518us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:18.341544 (+    38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:18.342157 (+   613us) log.cc:842] Serialized 26 byte log entry
0924 14:31:18.448190 (+106033us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:18.483882 (+ 35692us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.483949 (+    67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:19.203426 (+719477us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:19.324207 (+120781us) raft_consensus.cc:1671] finished
0924 14:31:19.324242 (+    35us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:19.324447 (+   205us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":409,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":6838528,"thread_start_us":96184,"threads_started":1,"wal-append.queue_time_us":217754}
W20260924 14:31:19.352433 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 113882us
W20260924 14:31:19.345317 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 88039.8us
I20260924 14:31:19.327548 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1050) took 2406 ms. Trace:
I20260924 14:31:19.319406 26329 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260924 14:31:19.337862 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:19.347049 31004 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:19.347366 30274 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:31:19.341150 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.341488 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:19.345173 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 53188.5us
I20260924 14:31:19.338668 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:19.339684 27188 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:19.318734 27189 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260924 14:31:19.364703 31041 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:19.355168 26325 rpcz_store.cc:276] 0924 14:31:16.920831 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.733364 (+1812533us) service_pool.cc:227] Handling call
0924 14:31:18.748949 (+ 15585us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:18.752525 (+  3576us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:18.752559 (+    34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:18.764581 (+ 12022us) log.cc:842] Serialized 26 byte log entry
0924 14:31:18.893970 (+129389us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:18.894001 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:18.894067 (+    66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:19.327241 (+433174us) raft_consensus.cc:1671] finished
0924 14:31:19.327273 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:19.327475 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":114908,"threads_started":1,"wal-append.queue_time_us":180134}
I20260924 14:31:19.366474 26323 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 13. Ops: []
I20260924 14:31:19.372730 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1561) took 1215 ms. Trace:
I20260924 14:31:19.373183 26323 rpcz_store.cc:276] 0924 14:31:18.157362 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:19.306514 (+1149152us) service_pool.cc:227] Handling call
0924 14:31:19.366286 (+ 59772us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:19.372335 (+  6049us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:19.372659 (+   324us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:19.300364 31159 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:19.368062 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.368297 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.302671 31341 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:19.379145 26331 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:19.380592 27181 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260924 14:31:19.387262 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1562) took 1229 ms. Trace:
I20260924 14:31:19.387866 26331 rpcz_store.cc:276] 0924 14:31:18.157833 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:19.306781 (+1148948us) service_pool.cc:227] Handling call
0924 14:31:19.378890 (+ 72109us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:19.386977 (+  8087us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:19.387204 (+   227us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:19.385183 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1020) took 1011 ms. Trace:
I20260924 14:31:19.308454 31348 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:19.333140 30983 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:17.099980 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:19.389708 26326 rpcz_store.cc:276] 0924 14:31:18.374038 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:19.340839 (+966801us) service_pool.cc:227] Handling call
0924 14:31:19.385109 (+ 44270us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:19.381044 27181 pending_rounds.cc:76] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 9
I20260924 14:31:19.395141 27181 pending_rounds.cc:97] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260924 14:31:19.395524 27181 raft_consensus.cc:2866] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:19.391510 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1676) took 3402 ms. Trace:
I20260924 14:31:19.391826 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:19.337109 30964 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:19.385481 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.334461 31079 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:19.402290 26328 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:19.405304 26323 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:19.408793 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1565) took 1249 ms. Trace:
I20260924 14:31:19.409242 26328 rpcz_store.cc:276] 0924 14:31:18.158912 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:19.308015 (+1149103us) service_pool.cc:227] Handling call
0924 14:31:19.367078 (+ 59063us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:19.408538 (+ 41460us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:19.408738 (+   200us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:19.411731 26329 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260924 14:31:19.382573 31341 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:19.417647 27188 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:19.414060 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:19.420840 26327 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:19.398782 26254 rpcz_store.cc:276] 0924 14:31:15.988813 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.120023 (+2131210us) service_pool.cc:227] Handling call
0924 14:31:18.213959 (+ 93936us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:18.271024 (+ 57065us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:31:18.294711 (+ 23687us) log.cc:842] Serialized 14 byte log entry
0924 14:31:19.022266 (+727555us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b1800207460. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd557b 00007f09b935888a
0924 14:31:19.391134 (+368868us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:19.391192 (+    58us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:19.391233 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:19.391301 (+    68us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:19.391433 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8787,"prepare.queue_time_us":17366,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":7742,"raft.queue_time_us":22801,"spinlock_wait_cycles":34217216,"thread_start_us":349266,"threads_started":1}
I20260924 14:31:19.256392 31345 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:19.432518 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:19.423471 26405 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:19.419649 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:19.393424 31348 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:19.430975 26323 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:19.434136 26398 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:19.435402 26326 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:31:19.430537 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:19.443286 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:19.332165 31147 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:19.452682 26324 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:19.452741 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:19.443168 31035 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260924 14:31:19.392599 31330 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:19.418427 31107 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260924 14:31:19.472282 31330 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:19.396768 31178 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260924 14:31:19.476694 31277 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:19.477356 31277 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:19.436340 31111 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:19.435823 31348 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:19.487520 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:19.457165 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:19.496657 26328 raft_consensus.cc:2354] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:19.367483 31329 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:19.497552 26332 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:19.499517 31329 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:19.506493 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:19.507835 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:19.508497 27188 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:19.509505 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:19.510142 27191 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:19.513998 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:19.478487 31277 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:19.486128 31353 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260924 14:31:19.534335 27188 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:19.540781 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.541740 26254 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
W20260924 14:31:19.543191 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 395808us
I20260924 14:31:19.545485 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:19.551841 31352 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:19.552551 31352 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:19.554720 31352 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:19.554945 27191 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
W20260924 14:31:19.556154 31354 log.cc:924] Time spent T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260924 14:31:19.571981 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:19.573099 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:19.571663 31329 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:19.575157 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:19.575824 26252 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:19.517951 31068 consensus_queue.cc:1046] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
W20260924 14:31:19.616941 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 197168us
W20260924 14:31:19.628355 26371 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.598s (SENT)
I20260924 14:31:19.574900 31329 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:19.629109 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:19.640390 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:19.632155 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:19.641463 26403 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:19.600598 31330 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:19.653980 26258 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:19.628969 30983 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:19.539532 31346 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.271s
I20260924 14:31:19.658154 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.630052 31352 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
W20260924 14:31:19.657466 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 266653us
W20260924 14:31:19.661785 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 136326us
I20260924 14:31:19.664259 26398 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:19.662107 26255 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:19.667811 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.668399 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.670192 26258 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:19.670354 26252 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:19.681917 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.681614 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:19.686726 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 94699.2us
I20260924 14:31:19.686719 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.688396 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:19.663811 31352 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:19.668004 31330 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:19.693136 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:19.693881 26256 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
W20260924 14:31:19.684803 31357 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260924 14:31:19.706595 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 298359us
W20260924 14:31:19.709661 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 150975us
I20260924 14:31:19.713002 26369 leader_election.cc:398] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14.
I20260924 14:31:19.686791 31345 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:19.723129 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:19.729446 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:19.708899 31355 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260924 14:31:19.679903 30983 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:19.748418 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.749410 26257 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:19.759734 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.762900 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:19.763775 26254 raft_consensus.cc:2362] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11. Re-sending same reply.
I20260924 14:31:19.760783 26256 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:31:19.768473 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.770120 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.772447 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:19.744182 31120 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260924 14:31:19.776445 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 122450us
I20260924 14:31:19.778921 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.779775 26254 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:19.779874 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.698643 31359 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.217s
I20260924 14:31:19.706557 31329 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:19.784803 26257 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
W20260924 14:31:19.786662 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 99762.7us
I20260924 14:31:19.798962 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.779601 31361 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:19.804834 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.808326 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1054) took 2797 ms. Trace:
I20260924 14:31:19.808887 26330 rpcz_store.cc:276] 0924 14:31:17.011085 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.948390 (+1937305us) service_pool.cc:227] Handling call
0924 14:31:19.001510 (+ 53120us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:19.008533 (+  7023us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:19.008573 (+    40us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:19.039261 (+ 30688us) log.cc:842] Serialized 26 byte log entry
0924 14:31:19.242719 (+203458us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:19.242768 (+    49us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:19.242882 (+   114us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:19.808031 (+565149us) raft_consensus.cc:1671] finished
0924 14:31:19.808060 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:19.808247 (+   187us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":96257,"threads_started":1,"wal-append.queue_time_us":325196}
I20260924 14:31:19.811297 26251 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:19.817610 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.819464 26251 raft_consensus.cc:2396] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260924 14:31:19.826134 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 136678us
I20260924 14:31:19.775195 31174 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
W20260924 14:31:19.835716 31365 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260924 14:31:19.835772 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.837508 26254 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:19.837634 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.798403 31107 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:19.838297 26257 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:19.773707 31352 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:19.842375 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 56644.1us
I20260924 14:31:19.850495 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:19.855935 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 56563.7us
W20260924 14:31:19.865613 31378 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260924 14:31:19.870157 31366 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260924 14:31:19.863369 31362 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.139s
I20260924 14:31:19.879678 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:19.853899 31120 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:19.900095 26371 leader_election.cc:334] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.828s (SENT)
I20260924 14:31:19.907881 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:19.909168 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:19.907595 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 76949.2us
I20260924 14:31:19.806280 31330 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:19.896762 31371 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260924 14:31:19.846259 31107 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:19.916162 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 59454.4us
I20260924 14:31:19.924108 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:19.925185 26257 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:19.935681 31120 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:19.880959 31359 consensus_queue.cc:1046] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
W20260924 14:31:19.949991 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 76223us
I20260924 14:31:19.969053 26257 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:19.972409 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:19.973229 26371 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.977s (SENT)
I20260924 14:31:19.988920 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:19.995944 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 136805us
I20260924 14:31:19.968741 31368 raft_consensus.cc:3032] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
W20260924 14:31:20.004757 31384 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260924 14:31:20.010036 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:19.938269 31392 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.116s
I20260924 14:31:20.012701 31107 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:20.012982 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.013695 26254 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
W20260924 14:31:20.014582 31259 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.796s	user 0.013s	sys 0.000s
I20260924 14:31:20.000777 30983 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:20.014503 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.017832 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:20.004161 26257 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:31:19.994933 31148 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:20.022583 31148 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.023111 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.023939 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:20.024501 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.025095 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.025197 26249 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
I20260924 14:31:20.025205 31148 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.025744 26254 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:20.026228 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.026664 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.026947 26249 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:31:20.008188 31159 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.626s	user 0.005s	sys 0.009s
I20260924 14:31:20.029402 31159 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:20.030164 31159 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:20.030550 31159 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.035360 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:20.043633 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 76101.8us
I20260924 14:31:19.998858 31368 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:19.991865 30975 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260924 14:31:20.048218 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:20.047580 27123 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:20.031605 31159 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:20.015065 31259 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.053289 30765 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:20.053755 31370 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:20.054243 30688 consensus_queue.cc:1035] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260924 14:31:20.054497 31368 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.051420 27191 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15. Ops: []
W20260924 14:31:20.056862 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 59428.1us
I20260924 14:31:20.071281 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:20.071362 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:20.079120 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 281827us
I20260924 14:31:20.088563 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:20.093575 31228 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:20.094726 31175 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:20.095264 31395 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:20.102618 31259 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.103315 30765 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:20.104359 30765 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:20.111136 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 63690us
I20260924 14:31:20.112138 26399 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
W20260924 14:31:20.081426 31389 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260924 14:31:20.112138 26400 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:20.117383 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:20.122231 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:20.081259 31370 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.127525 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.128227 26406 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:20.128774 26252 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:20.129673 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:20.130826 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:20.131764 31346 raft_consensus.cc:2741] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260924 14:31:20.132421 31346 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:20.132974 31346 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:20.133805 31004 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.134419 31004 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.116617 30972 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.138012 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:20.138517 30972 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:20.138068 30688 consensus_queue.cc:1046] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260924 14:31:20.140496 30972 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:20.141021 30688 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:20.141423 30688 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.141732 30688 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:20.143422 31361 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260924 14:31:20.125929 31111 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.631s	user 0.006s	sys 0.006s
W20260924 14:31:20.146492 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45461.5us
I20260924 14:31:20.147405 26369 leader_election.cc:398] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8.
I20260924 14:31:20.147728 26369 leader_election.cc:401] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260924 14:31:20.126148 31175 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:20.155208 26251 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260924 14:31:20.114986 31368 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:20.166234 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.164237 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:20.170151 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:20.126327 31370 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:20.176530 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:20.179011 26323 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
W20260924 14:31:20.181732 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 208533us
I20260924 14:31:20.149273 31120 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.183262 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:20.179999 26258 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:20.183948 26406 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:20.158663 31178 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:20.185262 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 54524.3us
I20260924 14:31:20.185717 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:20.192821 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:20.187469 26328 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:31:20.192018 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.196277 26331 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:20.173489 30972 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.176106 31362 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:20.177209 31148 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:20.213284 31035 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:20.194869 31060 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260924 14:31:20.213779 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:20.213716 31035 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:20.217324 26371 leader_election.cc:334] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.048s (SENT)
I20260924 14:31:20.174876 31396 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260924 14:31:20.214531 26398 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:20.219384 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:20.172365 31361 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260924 14:31:20.207482 31259 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader election won for term 8
I20260924 14:31:20.240072 26406 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:20.212231 31148 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.194928 31120 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.197378 31178 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.225924 31175 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:20.259361 27189 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
W20260924 14:31:20.261390 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47748.9us
W20260924 14:31:20.267606 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 118767us
I20260924 14:31:20.278777 31329 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:20.279671 31329 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.283649 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:20.283895 31329 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:20.284610 27191 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:20.211275 30972 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.239704 31408 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:20.286110 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 68832.4us
I20260924 14:31:20.245548 31111 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:20.307477 31111 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.307958 31111 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:20.254736 31148 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:20.258001 31341 consensus_meta.cc:287] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.821s	user 0.013s	sys 0.000s
I20260924 14:31:20.259311 31362 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:20.315830 31341 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.318293 31341 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.220598 31402 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260924 14:31:20.295020 31178 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:20.297092 31348 consensus_meta.cc:287] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.771s	user 0.012s	sys 0.000s
I20260924 14:31:20.297762 31228 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:20.297910 31408 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.323606 31348 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.274873 31395 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260924 14:31:20.325636 31408 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:20.293172 31175 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:20.327234 31348 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:20.304145 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:20.328671 31408 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:20.330166 31408 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:31:20.336849 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.260035 31359 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 7. Current term is 8.
I20260924 14:31:20.340341 31073 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:20.340816 31073 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:20.342001 31073 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.304486 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.347108 26406 raft_consensus.cc:2396] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260924 14:31:20.305925 26405 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:20.214653 31405 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260924 14:31:20.326941 31175 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.300607 30972 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:20.357548 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 190527us
I20260924 14:31:20.361745 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:20.362607 27191 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:20.365154 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 93052.5us
I20260924 14:31:20.373015 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:20.374131 26403 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:20.298935 30688 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.346037 31073 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:20.381091 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 197912us
I20260924 14:31:20.382251 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:20.375274 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.355290 31004 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:20.406953 31406 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260924 14:31:20.415414 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49275.3us
W20260924 14:31:20.414805 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 113181us
I20260924 14:31:20.417522 27191 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:20.424475 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:20.429879 26295 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:20.430470 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 58343.3us
I20260924 14:31:20.381239 31361 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260924 14:31:20.434494 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:20.427053 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:20.444526 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:20.446013 31335 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.452416 31335 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.454818 31335 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:20.458170 31360 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.906s	user 0.000s	sys 0.005s
I20260924 14:31:20.459596 31341 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:20.462056 31122 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:20.464773 31035 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.466244 31035 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:31:20.470197 31356 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.868s	user 0.003s	sys 0.000s
I20260924 14:31:20.472844 31194 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:20.473658 31194 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:20.474316 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.475016 26252 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:20.476188 31361 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.476687 31361 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.478770 31361 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.480913 26400 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:20.481307 26400 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:20.442241 31405 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:20.457700 26369 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:20.481279 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:20.494481 26327 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:20.497145 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1370) took 1501 ms. Trace:
I20260924 14:31:20.504623 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:20.446776 31348 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election won for term 15
W20260924 14:31:20.504567 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 101042us
I20260924 14:31:20.445947 31344 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:20.443056 31415 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:20.482520 31405 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.497712 27186 rpcz_store.cc:276] 0924 14:31:18.995844 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.995985 (+   141us) service_pool.cc:227] Handling call
0924 14:31:18.996302 (+   317us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:19.005803 (+  9501us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:19.005850 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:19.069365 (+ 63515us) log.cc:842] Serialized 26 byte log entry
0924 14:31:19.367194 (+297829us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:19.367239 (+    45us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:19.367311 (+    72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:20.469394 (+1102083us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:20.495586 (+ 26192us) raft_consensus.cc:1671] finished
0924 14:31:20.495619 (+    33us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:20.497053 (+  1434us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":24900,"raft.queue_time_us":10692,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":3392,"spinlock_wait_cycles":29450496,"thread_start_us":201612,"threads_started":1,"wal-append.queue_time_us":395961}
I20260924 14:31:20.483765 31277 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:20.511853 26400 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.520931 26256 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260924 14:31:20.514799 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 57193.8us
I20260924 14:31:20.525782 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:20.501349 31259 raft_consensus.cc:686] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:20.528900 26332 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:20.446550 31329 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260924 14:31:20.534773 26400 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:20.435492 31159 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.513027 31335 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:20.497944 31361 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.542675 27191 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260924 14:31:20.537963 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44035.4us
I20260924 14:31:20.474821 31194 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.481950 31111 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:20.545675 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 162931us
I20260924 14:31:20.530615 31405 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:20.553210 27125 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:20.551901 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 115344us
I20260924 14:31:20.526742 31348 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:20.501626 31370 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:20.525816 31422 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:20.559708 31422 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.560231 31422 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:20.549602 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:20.564270 26400 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260924 14:31:20.564648 26400 pending_rounds.cc:76] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 9
I20260924 14:31:20.557900 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:20.564922 26400 pending_rounds.cc:97] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260924 14:31:20.565214 26400 raft_consensus.cc:2866] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:20.566610 31175 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.567188 31175 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.569276 31175 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:20.544363 31060 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260924 14:31:20.570664 31194 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:20.572889 31159 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.551488 31279 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.574282 31279 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:20.576150 31409 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260924 14:31:20.576555 31279 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.566047 31335 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260924 14:31:20.577104 31228 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:20.577592 31228 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.577926 31228 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:20.577720 31348 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.580153 27188 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:20.580935 31266 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:20.581734 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 69319.5us
I20260924 14:31:20.555925 31361 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.586712 31107 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Stepping down as leader of term 19
I20260924 14:31:20.587081 31361 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.565138 26332 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:20.587720 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:20.620947 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 63196.3us
I20260924 14:31:20.590615 31370 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:20.572889 31259 consensus_queue.cc:237] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:20.599385 31400 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:20.601635 31107 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:20.602427 30688 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:20.602986 31369 raft_consensus.cc:3032] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:20.603762 31407 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:20.604378 31424 consensus_queue.cc:1046] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260924 14:31:20.655104 31346 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:20.655994 31344 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.655776 31346 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.656527 31344 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.657984 31346 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:20.605006 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 55773.5us
I20260924 14:31:20.658524 31344 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:20.587524 31148 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:20.659142 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:20.660557 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:20.660337 31107 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.661795 31107 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:20.622248 26327 raft_consensus.cc:2396] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260924 14:31:20.664318 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:20.664602 26327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1063) took 1222 ms (client timeout 1582 ms). Trace:
I20260924 14:31:20.664098 26076 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 8 to 11, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
W20260924 14:31:20.665023 26327 rpcz_store.cc:269] 0924 14:31:19.442515 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:20.480928 (+1038413us) service_pool.cc:227] Handling call
0924 14:31:20.664543 (+183615us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:20.665433 30983 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:20.626441 31418 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Stepping down as leader of term 15
W20260924 14:31:20.665958 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 59281.8us
I20260924 14:31:20.666218 31418 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:20.645288 31400 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:20.667062 31418 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:20.624792 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:20.667474 31400 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.668293 26406 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:20.668643 31400 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:20.669593 31175 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:20.670089 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:20.670766 27181 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:20.671679 31194 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:31:20.672341 31194 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:20.672389 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:20.627681 26297 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:20.673223 27186 raft_consensus.cc:2354] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17.
W20260924 14:31:20.632534 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 73432.1us
I20260924 14:31:20.621352 26402 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:20.673982 31173 raft_consensus.cc:3032] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:20.674480 31173 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:20.674685 31080 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:20.638345 31345 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:20.675637 31345 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:20.660588 31148 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:20.676796 31415 raft_consensus.cc:3032] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:20.676728 31345 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.640690 31408 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:20.675549 26403 raft_consensus.cc:1226] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:20.677799 31408 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:20.668313 31418 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:20.678848 31408 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.679975 31408 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:20.680227 26295 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:20.653703 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:20.681469 26400 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:20.651396 31369 raft_consensus.cc:729] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:20.683144 31369 consensus_queue.cc:260] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.684322 31369 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:20.653098 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:20.631798 31122 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:20.685729 31122 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.686142 31122 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:20.688232 31068 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:31:20.689813 26251 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:20.690202 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.690467 31111 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:20.691685 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:20.691790 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:20.692444 26256 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:20.693040 31410 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.434s	user 0.000s	sys 0.002s
I20260924 14:31:20.637576 31352 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:20.693956 31370 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.694456 31370 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.695151 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:20.696337 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45170.3us
I20260924 14:31:20.677333 31415 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:20.699903 31415 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.639776 31330 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:20.701025 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:20.701252 31330 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:20.702210 31068 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:20.702065 31330 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.702636 31068 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.703099 31068 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:20.625914 26405 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:20.704668 26405 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260924 14:31:20.685725 26324 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:20.705392 31173 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:20.708314 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 103697us
I20260924 14:31:20.711424 31431 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:20.712651 31431 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:20.713047 31431 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:20.713896 31431 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.715128 31431 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:20.703338 31330 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:20.677964 31345 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:20.719573 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 55365.1us
W20260924 14:31:20.720121 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 93099.9us
I20260924 14:31:20.721261 31120 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:20.723861 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:20.724669 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 44548.4us
I20260924 14:31:20.725782 31405 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.665859 26076 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:20.727792 31359 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.732290 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:20.759720 31405 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.760708 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:20.761592 26403 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:20.762887 26295 leader_election.cc:398] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17.
I20260924 14:31:20.763237 26295 leader_election.cc:401] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260924 14:31:20.763556 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 52974us
I20260924 14:31:20.764007 31080 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:20.764448 31080 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.764811 31080 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:20.765295 31228 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.766342 26221 leader_election.cc:398] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:20.766685 26221 leader_election.cc:401] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:20.768669 31329 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 17
I20260924 14:31:20.770697 31068 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:20.771727 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.772387 26398 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:20.774770 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:20.706595 31173 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:20.775305 27188 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:20.776520 31351 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:20.777225 27189 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260924 14:31:20.777100 31351 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.778337 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:20.779150 31415 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:20.779544 31073 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:31:20.782481 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:20.796319 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.796872 27123 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:20.796993 26371 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.000s (SENT)
I20260924 14:31:20.798013 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:20.797130 26404 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:20.799162 26256 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:20.802286 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:20.802894 26332 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:20.805644 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.753036 31111 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:20.806331 26398 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:20.819434 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.818233 26400 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:20.823127 27189 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:20.822253 26406 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:20.824465 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.825259 26403 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:20.826624 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:20.826814 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44888.1us
I20260924 14:31:20.828208 26399 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260924 14:31:20.829124 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:20.798877 31392 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.000s	sys 0.016s
I20260924 14:31:20.834831 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.835131 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:20.835407 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.835862 26324 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:20.836331 26406 raft_consensus.cc:2379] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:20.781878 31228 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.837478 26332 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
W20260924 14:31:20.840179 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 81977.4us
I20260924 14:31:20.797426 31175 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.796706 31344 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.843796 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:20.797353 31121 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.843490 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:20.845352 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:20.846266 26331 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:20.846182 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:20.847379 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:20.849637 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:20.850518 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:20.851316 26330 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:20.851526 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:20.851233 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 93820.7us
I20260924 14:31:20.819284 31148 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:20.821702 31111 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:20.834976 31392 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.000s	sys 0.017s
I20260924 14:31:20.855043 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:20.855927 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:20.896390 26332 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:20.854439 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:20.858968 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:20.763088 31422 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:20.887621 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.827553 31277 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:20.905516 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:20.906836 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 110030us
I20260924 14:31:20.858016 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:20.887132 26329 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260924 14:31:20.891647 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:20.904786 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 100452us
I20260924 14:31:20.902292 26324 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:20.788173 31359 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:20.905716 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:20.940558 26398 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 7: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:20.837136 31351 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.912724 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:20.916164 26330 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:20.920030 31122 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.923588 31394 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.958518 31394 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:20.959961 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.961431 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.961829 31394 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:20.856962 31175 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:20.962604 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:20.962914 26081 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 140) took 54842 ms (client timeout 15000 ms). Trace:
I20260924 14:31:20.963495 26250 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:20.964424 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:20.905505 31416 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:20.965507 31416 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.965979 31416 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:20.966187 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:20.962668 26297 leader_election.cc:334] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.132s (SENT)
I20260924 14:31:20.967569 27185 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260924 14:31:20.967286 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:20.968729 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:20.969914 26250 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:20.972395 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:20.973430 27181 raft_consensus.cc:2354] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
W20260924 14:31:20.976398 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 87619.4us
I20260924 14:31:20.977758 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:20.979629 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 87114.2us
I20260924 14:31:20.982026 30274 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:20.982535 30274 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:20.983618 26401 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:20.983541 30274 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:20.985271 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:20.986052 26404 raft_consensus.cc:2354] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:20.987169 26403 raft_consensus.cc:2362] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 8. Re-sending same reply.
I20260924 14:31:20.988452 31359 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:20.903308 31120 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:20.989585 31120 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:20.989953 31120 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:20.886701 26076 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 9 to 11, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 11 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:20.893427 31111 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:20.993322 31080 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.994982 26323 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:20.995757 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 59613us
I20260924 14:31:20.996245 31068 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:20.996814 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:20.997594 26323 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:20.969995 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:20.999946 27192 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 13. Ops: []
I20260924 14:31:21.000399 27190 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:21.001861 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1714) took 1768 ms. Trace:
I20260924 14:31:21.002281 26253 rpcz_store.cc:276] 0924 14:31:19.233414 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:19.236509 (+  3095us) service_pool.cc:227] Handling call
0924 14:31:19.236858 (+   349us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:19.242767 (+  5909us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:19.242840 (+    73us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:20.588873 (+1346033us) spinlock_profiling.cc:243] Waited 933 us on lock 0x7b2c001c3658. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:20.589218 (+   345us) log.cc:842] Serialized 26 byte log entry
0924 14:31:20.756871 (+167653us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:20.757992 (+  1121us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:20.787642 (+ 29650us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:21.001531 (+213889us) raft_consensus.cc:1671] finished
0924 14:31:21.001555 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:21.001789 (+   234us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4036,"raft.queue_time_us":1607,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":7476,"spinlock_wait_cycles":2145664,"thread_start_us":165458,"threads_started":1,"wal-append.queue_time_us":316554}
I20260924 14:31:20.855211 31344 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:21.004467 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:21.004823 26327 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:20.925419 31422 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:21.005090 26253 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 13
I20260924 14:31:20.926215 26406 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:20.908425 26331 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:20.928989 27189 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260924 14:31:20.927731 31405 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:20.854758 31228 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:21.007969 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:20.901965 31397 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260924 14:31:21.008582 26331 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:21.004494 26328 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:20.964780 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.979259 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:20.939751 26224 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:20.860908 30983 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:20.886243 31259 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:21.037514 26252 raft_consensus.cc:2379] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:20.854032 31418 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:20.950553 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:21.039422 26400 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:31:21.041606 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 79057.3us
I20260924 14:31:20.945647 26332 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:20.950676 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:20.955621 31429 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 13
I20260924 14:31:20.956102 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:20.956303 27188 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:31:20.956102 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:20.855479 31121 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:21.086005 31121 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:20.957703 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.087270 26249 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:21.088502 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.089273 26249 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:21.090245 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:21.062409 31416 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:20.931516 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.096060 26255 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:21.097082 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:21.098452 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:21.099702 26323 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
W20260924 14:31:21.028573 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 88894.4us
I20260924 14:31:20.964671 31175 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:21.044720 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:21.148191 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:21.149137 26323 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:20.886574 31211 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.167s
I20260924 14:31:20.956832 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.050892 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.054170 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:21.056499 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:21.062736 27190 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:21.065014 31120 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:21.065475 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:21.066017 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:21.068602 26253 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:21.156358 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.155763 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:21.046502 26250 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:21.157895 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:21.158625 26250 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:21.072175 31348 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:21.076229 31329 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17.
I20260924 14:31:21.076623 26221 leader_election.cc:398] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:21.169831 26221 leader_election.cc:401] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260924 14:31:21.170490 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 94107us
I20260924 14:31:21.077373 31423 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:21.037567 31405 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:21.043058 31418 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:21.036623 31351 raft_consensus.cc:2741] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260924 14:31:21.049165 31259 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:21.041728 31039 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:20.937587 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:21.104048 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106216us
I20260924 14:31:21.106928 31174 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:21.109110 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 129935us
I20260924 14:31:21.109407 31073 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
W20260924 14:31:20.963306 26081 rpcz_store.cc:269] 0924 14:30:26.061064 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:26.061619 (+   555us) service_pool.cc:227] Handling call
0924 14:31:20.886968 (+54825349us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:08.015987 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:08.045305 (+ 29318us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:08.045361 (+    56us) write_op.cc:435] Acquired schema lock
0924 14:31:08.045393 (+    32us) tablet.cc:660] Decoding operations
0924 14:31:08.046166 (+   773us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:08.046278 (+   112us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:08.046336 (+    58us) tablet.cc:683] Acquiring locks for 4 operations
0924 14:31:08.046910 (+   574us) tablet.cc:699] Row locks acquired
0924 14:31:08.046945 (+    35us) write_op.cc:260] PREPARE: finished
0924 14:31:08.047135 (+   190us) write_op.cc:270] Start()
0924 14:31:08.047305 (+   170us) write_op.cc:276] Timestamp: P: 1790260268047055 usec, L: 0
0924 14:31:08.064594 (+ 17289us) op_driver.cc:331] REPLICATION: starting
0924 14:31:08.143975 (+ 79381us) log.cc:842] Serialized 1717 byte log entry
0924 14:31:08.912456 (+768481us) op_driver.cc:444] REPLICATION: finished
0924 14:31:09.250046 (+337590us) write_op.cc:301] APPLY: starting
0924 14:31:09.262477 (+ 12431us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:09.563855 (+301378us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:09.585546 (+ 21691us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:10.413697 (+828151us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:10.669994 (+256297us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0924 14:31:10.695626 (+ 25632us) write_op.cc:312] APPLY: finished
0924 14:31:10.696017 (+   391us) log.cc:842] Serialized 48 byte log entry
0924 14:31:10.848828 (+152811us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:10.848940 (+   112us) write_op.cc:454] Released schema lock
0924 14:31:10.849008 (+    68us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:10.878715 (+ 29707us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:10.881686 (+  2971us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":227558,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":286636,"apply.run_cpu_time_us":38959,"apply.run_wall_time_us":1651709,"cfile_cache_hit":14,"cfile_cache_hit_bytes":14644,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6219,"cfile_init":2,"lbm_read_time_us":30172,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":11,"num_ops":4,"prepare.queue_time_us":396839,"prepare.run_cpu_time_us":12061,"prepare.run_wall_time_us":222722,"raft.queue_time_us":410283,"raft.run_cpu_time_us":25658,"raft.run_wall_time_us":262417,"replication_time_us":811624,"thread_start_us":859854,"threads_started":5,"wal-append.queue_time_us":416028,"wal-append.run_cpu_time_us":81964,"wal-append.run_wall_time_us":2802836}]]}
I20260924 14:31:21.116767 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.197796 26325 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:21.115985 26076 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:21.118316 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:21.119094 31080 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:21.121011 31429 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:20.999655 31370 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:21.124450 31344 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:21.125820 27194 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:21.130450 27182 raft_consensus.cc:2396] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260924 14:31:21.137202 26224 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:21.146350 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.045385 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:21.151755 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 95308.3us
I20260924 14:31:21.167886 26254 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:21.150687 26257 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:21.157013 26253 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:21.154983 27189 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:21.156592 27125 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:21.148424 26330 raft_consensus.cc:2362] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13. Re-sending same reply.
I20260924 14:31:21.157934 26331 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:21.153980 27186 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:31:21.046943 31122 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:21.188271 26329 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:21.195695 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:21.196273 26399 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
W20260924 14:31:21.192909 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 349465us
I20260924 14:31:21.271836 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:21.272650 26252 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:21.257115 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:21.205461 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 267928us
I20260924 14:31:21.186503 31415 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:21.274724 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:21.212072 26328 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:21.154810 31121 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:21.215022 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:21.305714 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:21.266227 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:21.154959 30975 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
W20260924 14:31:21.316666 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 179528us
I20260924 14:31:21.317826 26224 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:21.310537 26406 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:21.318019 26332 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
W20260924 14:31:21.320194 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45613.5us
I20260924 14:31:21.322304 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:21.333405 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:21.334474 26253 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
W20260924 14:31:21.337419 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 122475us
I20260924 14:31:21.259219 31400 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260924 14:31:21.319319 31329 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:21.344475 31429 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:21.344743 31329 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:21.345587 26076 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 4, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 4 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:21.346515 31424 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:21.347368 31423 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:31:21.347555 31422 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:31:21.347764 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:21.348755 26328 raft_consensus.cc:2354] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:21.348881 26331 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:21.349068 31422 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:21.349491 31422 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:21.349611 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:21.349879 31422 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:21.350112 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:21.350795 26331 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:21.351395 31080 raft_consensus.cc:2741] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260924 14:31:21.353153 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:21.351648 31367 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.141s
I20260924 14:31:21.354516 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:21.258235 30972 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260924 14:31:21.356096 31122 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:21.347126 31424 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:21.356662 31122 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:21.357653 31122 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:21.358579 31429 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:21.352707 31344 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:21.350289 26328 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:21.355412 31121 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:21.305289 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 97506us
I20260924 14:31:21.360872 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:21.347107 31329 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:21.369272 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:21.374786 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:21.381973 26331 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:21.383965 26331 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:31:21.386480 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:21.332115 31348 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:21.317157 31416 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:21.352137 31369 consensus_meta.cc:287] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.004s
W20260924 14:31:21.393723 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 237203us
I20260924 14:31:21.338572 31415 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:21.395267 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:21.360293 31359 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:31:21.187330 31418 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:21.260695 26255 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:21.308547 31351 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260924 14:31:21.266275 30983 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:21.434358 30972 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:21.378530 26076 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:21.393205 31073 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:21.399883 31259 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:21.413220 31359 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:21.361434 31431 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
W20260924 14:31:21.458020 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 88836us
I20260924 14:31:21.458823 31387 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260924 14:31:21.413748 31330 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.661s	user 0.013s	sys 0.000s
W20260924 14:31:21.463119 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 139736us
I20260924 14:31:21.392755 31348 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:21.465797 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:21.462282 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:21.189898 31405 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:21.204098 31039 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:21.378980 31368 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260924 14:31:21.471120 27123 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:21.513041 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:21.392957 31370 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:21.515169 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:21.204512 31352 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.246s
I20260924 14:31:21.203541 31174 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:21.374176 31111 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:21.434851 31400 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:21.525935 31174 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:21.528298 26323 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:21.529299 27189 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260924 14:31:21.531054 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.531358 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:21.531940 27192 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:21.508970 31392 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:21.500980 27126 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:21.502014 26327 raft_consensus.cc:2396] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260924 14:31:21.502754 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:21.542330 26255 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:21.544414 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:21.515107 31396 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:21.461615 30983 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:21.504114 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:21.471935 31335 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.010s	sys 0.033s
I20260924 14:31:21.507808 27190 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:21.510754 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:21.471998 31359 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:31:21.472173 26369 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.017s	sys 0.028s
I20260924 14:31:21.491003 31387 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:21.588937 27188 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
W20260924 14:31:21.593601 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 49271.9us
W20260924 14:31:21.508288 30274 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.045s
I20260924 14:31:21.516922 31121 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
I20260924 14:31:21.499614 31080 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:21.499562 30972 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:21.525208 31111 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:21.433583 31429 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:21.506134 31367 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:21.520341 31175 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:21.557948 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:21.561434 27182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 863) took 1540 ms (client timeout 2011 ms). Trace:
I20260924 14:31:21.473618 26256 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:21.551641 31120 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:21.503358 26252 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
W20260924 14:31:21.589501 26369 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.017s	sys 0.028s
W20260924 14:31:21.628652 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 701640us
I20260924 14:31:21.630329 26401 raft_consensus.cc:2396] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260924 14:31:21.629997 27192 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:21.597430 31387 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:21.596920 26330 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260924 14:31:21.635694 26330 pending_rounds.cc:76] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 7
I20260924 14:31:21.635959 26330 pending_rounds.cc:97] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260924 14:31:21.636188 26330 raft_consensus.cc:2866] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:21.547294 31392 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:21.552668 31441 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260924 14:31:21.639169 27126 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:21.604961 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 100952us
I20260924 14:31:21.574210 31073 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:21.588464 31335 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.011s	sys 0.033s
I20260924 14:31:21.608404 31039 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:21.602272 30274 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.008s	sys 0.045s
I20260924 14:31:21.596640 31400 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:21.552103 31174 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:21.611289 26295 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:21.521142 31370 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:21.632006 27182 rpcz_store.cc:269] 0924 14:31:20.020692 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:20.020887 (+   195us) service_pool.cc:227] Handling call
0924 14:31:21.561178 (+1540291us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54000ce9d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:21.561360 (+   182us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2486492544}
I20260924 14:31:21.595326 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:21.658605 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 63359.1us
I20260924 14:31:21.633445 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:21.602584 31394 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:21.640216 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:21.590546 31397 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:21.647917 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:21.638281 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50385.8us
I20260924 14:31:21.629673 31175 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:21.644914 31408 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.964s	user 0.013s	sys 0.000s
I20260924 14:31:21.646240 31068 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:21.610126 31346 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260924 14:31:21.642169 31392 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:21.612566 31080 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:21.704155 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.646360 31424 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:21.646593 31367 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260924 14:31:21.712545 26252 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
W20260924 14:31:21.682900 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 71710.8us
I20260924 14:31:21.718619 26295 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:21.681984 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:21.697166 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:21.656891 31174 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:21.693818 31399 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.062s	sys 0.069s
I20260924 14:31:21.668145 31345 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:21.667793 30975 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:21.678449 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:21.676049 31358 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:21.676659 31175 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:21.619475 30972 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:21.656508 26326 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.020s	sys 0.055s
W20260924 14:31:21.755354 26326 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.020s	sys 0.055s
I20260924 14:31:21.680971 31394 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:21.652819 30983 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:21.733707 31174 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:21.765728 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:21.741235 31121 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:21.701061 31111 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260924 14:31:21.749449 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 71121.3us
I20260924 14:31:21.766839 31068 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:21.676120 31039 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
W20260924 14:31:21.734555 31399 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.063s	sys 0.069s
W20260924 14:31:21.731776 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 58586.4us
I20260924 14:31:21.732774 26327 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:21.780750 26369 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:21.746253 31358 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:21.782701 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:21.758013 31408 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:31:21.745405 26328 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:21.608577 26076 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 9, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 9 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:21.746974 31348 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:21.780808 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:21.766505 27189 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:31:21.785224 27125 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:21.797919 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:21.774453 31111 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:21.777616 30983 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:21.775755 31405 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:21.799698 26295 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:21.786988 31358 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:21.812307 26076 catalog_manager.cc:5495] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:21.787554 31400 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:21.787122 31329 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:31:21.832227 31448 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260924 14:31:21.772779 31068 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:21.841952 31068 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:21.807569 31111 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:21.843225 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:21.844059 26328 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:21.844213 31080 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:21.844753 31394 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:21.845846 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:21.846612 27189 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:21.822763 31399 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:21.847966 31399 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:21.848162 31446 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260924 14:31:21.848471 31399 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:21.848771 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:21.849181 31344 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:21.855602 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49433.4us
I20260924 14:31:21.855926 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1748) took 1129 ms. Trace:
I20260924 14:31:21.795917 31345 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260924 14:31:21.860069 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:21.860502 31345 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:21.803771 31277 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:21.856338 26251 rpcz_store.cc:276] 0924 14:31:20.726900 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:20.728604 (+  1704us) service_pool.cc:227] Handling call
0924 14:31:20.728929 (+   325us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:20.729295 (+   366us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:20.729323 (+    28us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:20.729860 (+   537us) log.cc:842] Serialized 52 byte log entry
0924 14:31:21.148151 (+418291us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:21.148198 (+    47us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:21.148268 (+    70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:21.855663 (+707395us) raft_consensus.cc:1671] finished
0924 14:31:21.855683 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:21.855866 (+   183us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":5773,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":29139,"thread_start_us":417912,"threads_started":1,"wal-append.queue_time_us":927942}
I20260924 14:31:21.799283 31121 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:21.874244 31080 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:21.826787 31346 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:21.861099 31405 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:21.827661 31400 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:21.879364 31080 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:21.878417 31345 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:21.880820 31212 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:21.892242 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:21.884111 31370 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:21.880136 31277 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:21.898918 27123 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:21.883072 31346 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:21.885310 31405 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:21.888294 31400 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:21.910650 26253 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.015s	sys 0.019s
W20260924 14:31:21.917558 26253 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.016s	sys 0.020s
I20260924 14:31:21.896905 31212 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:21.893035 31178 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:21.892457 31345 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:21.912518 31346 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:21.928354 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:21.911172 26076 catalog_manager.cc:5495] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 8 to 9, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 9 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:21.930433 31212 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:21.964608 26258 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.012s	sys 0.023s
W20260924 14:31:21.936973 26403 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.011s	sys 0.020s
I20260924 14:31:21.915808 31405 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:21.945355 31370 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:21.936080 31178 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:21.931754 31121 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:21.928009 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.007s	sys 0.028s
W20260924 14:31:21.971486 26403 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.011s	sys 0.020s
I20260924 14:31:21.914992 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:21.992414 31449 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
W20260924 14:31:21.996219 31418 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.003s
W20260924 14:31:21.998487 31422 consensus_meta.cc:287] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.646s	user 0.008s	sys 0.007s
I20260924 14:31:21.965804 31277 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:21.969906 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.013s	sys 0.021s
W20260924 14:31:22.020875 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.013s	sys 0.022s
W20260924 14:31:22.021653 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 703823us
I20260924 14:31:22.022574 26224 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:22.026479 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.026275 31399 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:22.028486 31399 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:22.028182 31111 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.027189 27189 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:22.030555 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:21.972707 31345 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.035804 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:22.010628 31173 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:22.010990 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.008s	sys 0.028s
I20260924 14:31:22.046342 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:22.047183 26255 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:22.014586 26258 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.013s	sys 0.024s
W20260924 14:31:22.055938 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 258093us
I20260924 14:31:22.057688 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:22.061827 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:22.064184 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.065311 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:22.068668 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:22.070377 31348 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:22.071014 31348 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:22.071192 31451 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:22.071938 31451 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:22.072147 31348 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.072800 31451 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.033799 31344 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:22.082080 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.088151 27186 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:31:22.083302 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.083776 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:22.084491 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:22.092585 26250 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:22.085258 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:22.032209 31080 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.090598 27181 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:31:22.040457 31345 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:22.011852 31422 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.011621 31418 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.110056 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:22.110916 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:22.112006 26251 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:22.097568 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1606) took 1166 ms. Trace:
I20260924 14:31:22.113759 26258 rpcz_store.cc:276] 0924 14:31:20.931399 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:20.932001 (+   602us) service_pool.cc:227] Handling call
0924 14:31:20.932298 (+   297us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:20.932629 (+   331us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:20.932657 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:20.933724 (+  1067us) log.cc:842] Serialized 26 byte log entry
0924 14:31:21.329244 (+395520us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:22.080310 (+751066us) log.cc:842] Serialized 14 byte log entry
0924 14:31:22.097159 (+ 16849us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:22.097264 (+   105us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:22.097324 (+    60us) raft_consensus.cc:1671] finished
0924 14:31:22.097341 (+    17us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:22.097498 (+   157us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16155,"prepare.queue_time_us":602940,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1304,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":9541,"thread_start_us":1061291,"threads_started":2,"wal-append.queue_time_us":757941}
I20260924 14:31:22.095427 26254 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:31:22.104280 27189 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260924 14:31:22.086120 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:22.089586 31440 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260924 14:31:22.108083 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:22.110991 26257 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:22.126272 27194 raft_consensus.cc:2379] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:22.091269 31344 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:22.138749 26250 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:22.100883 31453 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:22.129235 31359 consensus_meta.cc:287] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.516s	user 0.014s	sys 0.000s
I20260924 14:31:22.117060 26255 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:22.053351 31173 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.162670 31358 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.149024 26249 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:22.141789 31472 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260924 14:31:22.152717 31453 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:31:22.155913 31178 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.031915 26076 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:22.127707 31111 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.169440 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:22.174057 31173 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:22.140869 31344 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:22.148204 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:22.250965 27189 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:22.146363 31454 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:22.257932 26251 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:22.166551 31424 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:22.269074 26397 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:22.176403 30972 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:22.276721 27125 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:22.275816 31440 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260924 14:31:22.166689 31359 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:22.269969 31472 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:22.295099 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1225) took 1232 ms. Trace:
I20260924 14:31:22.300228 26249 rpcz_store.cc:276] 0924 14:31:21.050805 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.089959 (+ 39154us) service_pool.cc:227] Handling call
0924 14:31:22.282811 (+1192852us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54004fae54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:22.283024 (+   213us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2419055488}
W20260924 14:31:22.297302 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 516524us
I20260924 14:31:22.302897 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:22.305406 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:22.278307 31259 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:22.284433 31424 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:22.289177 31408 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:22.315728 26406 raft_consensus.cc:1226] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:31:22.356485 31491 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:22.360265 31473 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:22.361229 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:22.363952 31476 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:22.371963 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:22.321504 31174 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.543s	user 0.004s	sys 0.012s
I20260924 14:31:22.428813 26332 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:31:22.430456 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:22.378830 31500 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:22.439134 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:22.299875 31445 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:22.394565 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:22.424590 31392 consensus_meta.cc:287] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.708s	user 0.008s	sys 0.004s
I20260924 14:31:22.312631 31431 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:31:22.325007 31387 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.680s	user 0.013s	sys 0.000s
I20260924 14:31:22.426577 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:22.378180 26369 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:22.399698 31121 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:22.457661 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 79573.4us
I20260924 14:31:22.431949 26332 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:22.459250 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:22.282816 30972 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:22.460445 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 67466.5us
I20260924 14:31:22.462188 30972 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.436196 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:22.438081 31457 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260924 14:31:22.463511 26398 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:22.464358 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1616) took 1271 ms. Trace:
I20260924 14:31:22.464473 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:22.465654 26369 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:22.464892 26403 rpcz_store.cc:276] 0924 14:31:21.192867 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.192988 (+   121us) service_pool.cc:227] Handling call
0924 14:31:21.265617 (+ 72629us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:21.309518 (+ 43901us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:21.309565 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:21.330453 (+ 20888us) log.cc:842] Serialized 26 byte log entry
0924 14:31:21.997736 (+667283us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:21.997804 (+    68us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:21.997898 (+    94us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:22.464162 (+466264us) raft_consensus.cc:1671] finished
0924 14:31:22.464181 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:22.464293 (+   112us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":66096,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":16312,"thread_start_us":595914,"threads_started":1,"wal-append.queue_time_us":937720}
I20260924 14:31:22.424981 31473 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:31:22.424376 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 362582us
I20260924 14:31:22.307557 31472 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.448460 26399 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
W20260924 14:31:22.454146 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53760.8us
W20260924 14:31:22.473903 27188 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.004s	sys 0.017s
I20260924 14:31:22.464434 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:22.478271 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:22.481557 26399 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:22.481786 26398 raft_consensus.cc:2379] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:22.482474 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:22.479516 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:22.447840 31418 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:22.478714 27188 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.005s	sys 0.017s
I20260924 14:31:22.452557 31476 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:22.503679 26224 leader_election.cc:398] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:22.539037 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:22.544071 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:22.545362 26406 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 14
I20260924 14:31:22.457226 31422 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.543195 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:22.551174 26403 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:22.554639 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 89044.1us
W20260924 14:31:22.476490 31429 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.019s	sys 0.002s
I20260924 14:31:22.402614 31259 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:22.453542 31500 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:22.530309 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 56686.8us
I20260924 14:31:22.530858 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:22.564911 26398 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:31:22.558879 26328 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:22.438773 31174 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:22.554814 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 425112us
I20260924 14:31:22.426401 31491 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:22.562582 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 83212.3us
I20260924 14:31:22.567916 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:22.542127 31478 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260924 14:31:22.493471 31501 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:22.570992 26221 leader_election.cc:398] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13.
I20260924 14:31:22.571658 31500 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:22.573184 31266 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:22.562047 31408 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:22.576085 31475 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260924 14:31:22.581007 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:22.454028 31392 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:22.586285 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:22.559942 31374 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:22.605001 26398 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
W20260924 14:31:22.585402 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46483.2us
I20260924 14:31:22.476166 31121 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.470965 31387 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:22.559641 31358 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:22.567876 31455 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260924 14:31:22.600170 26406 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:22.453792 31445 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.587064 31266 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:22.565908 31422 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:22.606101 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:22.666658 26328 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:22.612859 31466 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260924 14:31:22.613021 31479 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260924 14:31:22.586951 31501 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:22.613449 26076 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 10 to 12, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 12 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
W20260924 14:31:22.615103 31461 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260924 14:31:22.615751 31392 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.558051 31418 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260924 14:31:22.685444 31374 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:22.686518 31374 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:22.683130 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 91888.8us
W20260924 14:31:22.697273 31346 consensus_meta.cc:287] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.746s	user 0.008s	sys 0.004s
I20260924 14:31:22.697784 31346 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.700085 31346 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.699993 31431 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260924 14:31:22.702280 31483 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260924 14:31:22.702560 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 134728us
I20260924 14:31:22.703617 30983 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:22.703997 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1584) took 1878 ms. Trace:
I20260924 14:31:22.660516 26406 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:22.704523 26326 rpcz_store.cc:276] 0924 14:31:20.825531 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:20.825659 (+   128us) service_pool.cc:227] Handling call
0924 14:31:20.849821 (+ 24162us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:21.755876 (+906055us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:31:21.952608 (+196732us) log.cc:842] Serialized 14 byte log entry
0924 14:31:22.391799 (+439191us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:22.425115 (+ 33316us) log.cc:842] Serialized 26 byte log entry
0924 14:31:22.425280 (+   165us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:22.425326 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:22.425430 (+   104us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:22.703699 (+278269us) raft_consensus.cc:1671] finished
0924 14:31:22.703726 (+    27us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:22.703888 (+   162us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":40146,"prepare.queue_time_us":473788,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":93539,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":30784,"thread_start_us":1455490,"threads_started":3,"wal-append.queue_time_us":508280}
W20260924 14:31:22.706234 31463 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260924 14:31:22.582507 31473 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.659960 27189 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:22.708618 31501 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.665151 27190 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260924 14:31:22.617105 31460 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.340s	user 0.000s	sys 0.001s
W20260924 14:31:22.668435 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 155054us
I20260924 14:31:22.670040 26398 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:31:22.712145 26298 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:22.712932 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:22.713840 26398 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:22.714949 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:22.619280 31490 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:22.626740 31359 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:22.665302 31465 log.cc:924] Time spent T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260924 14:31:22.674674 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:22.665652 31266 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:22.557205 31476 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.567608 31424 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:22.681105 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:22.682348 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:22.668099 30975 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:22.664832 31358 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:22.735538 31405 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.004s
I20260924 14:31:22.735136 31358 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.736088 31405 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.736673 31473 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:22.737478 26399 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:22.737712 31358 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.738605 31472 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.666193 31422 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:22.703984 31487 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260924 14:31:22.669137 31492 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:22.715869 31277 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:22.748234 26076 catalog_manager.cc:5495] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 3 to 6, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:31:22.628319 31464 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.346s	user 0.001s	sys 0.001s
I20260924 14:31:22.752184 31259 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.753726 31344 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.571168 31429 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:22.692706 31368 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260924 14:31:22.716279 31514 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:22.720068 31400 consensus_meta.cc:287] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.789s	user 0.009s	sys 0.003s
I20260924 14:31:22.721647 31121 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:22.723160 31387 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.724270 31500 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:22.724982 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1223) took 1679 ms. Trace:
I20260924 14:31:22.726110 31174 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:22.726361 31212 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.763s	user 0.011s	sys 0.001s
W20260924 14:31:22.726890 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 162182us
I20260924 14:31:22.719300 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:22.719003 26406 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260924 14:31:22.732867 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 58291.9us
I20260924 14:31:22.733454 26402 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:22.708640 31491 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.706830 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:22.745141 26399 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260924 14:31:22.743716 31422 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:22.792616 26326 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:22.796489 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:22.796832 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:22.798118 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.755772 26328 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260924 14:31:22.799690 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.783804 31429 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:22.800730 31212 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:22.775166 31514 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.803046 31212 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.803526 31514 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.767338 31400 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.768682 31476 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.769193 31405 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:22.769956 31175 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.001s	user 0.009s	sys 0.004s
I20260924 14:31:22.816761 31400 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.816977 31175 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.750167 26076 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 7 to 10, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 10 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:22.770891 31445 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.820111 26330 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:22.821636 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 162573us
I20260924 14:31:22.822320 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:22.823081 26252 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:22.772221 31266 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.747277 31507 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:22.776156 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1613) took 1584 ms. Trace:
I20260924 14:31:22.780328 26253 rpcz_store.cc:276] 0924 14:31:21.045223 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.335615 (+290392us) service_pool.cc:227] Handling call
0924 14:31:21.336063 (+   448us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:21.356933 (+ 20870us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:21.356970 (+    37us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:21.357731 (+   761us) log.cc:842] Serialized 52 byte log entry
0924 14:31:21.918127 (+560396us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:22.359495 (+441368us) log.cc:842] Serialized 14 byte log entry
0924 14:31:22.379643 (+ 20148us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:22.379742 (+    99us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:22.724745 (+345003us) raft_consensus.cc:1671] finished
0924 14:31:22.724767 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:22.724923 (+   156us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20353,"prepare.queue_time_us":14562,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":35155,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":136,"thread_start_us":1018728,"threads_started":3,"wal-append.queue_time_us":787264}
I20260924 14:31:22.826930 26400 rpcz_store.cc:276] 0924 14:31:21.191822 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.203755 (+ 11933us) service_pool.cc:227] Handling call
0924 14:31:21.264115 (+ 60360us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:21.264615 (+   500us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:21.264656 (+    41us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:21.331212 (+ 66556us) log.cc:842] Serialized 52 byte log entry
0924 14:31:21.940246 (+609034us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:21.940319 (+    73us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:21.940390 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:22.775928 (+835538us) raft_consensus.cc:1671] finished
0924 14:31:22.775948 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:22.776094 (+   146us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31097,"raft.queue_time_us":684,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":40979,"thread_start_us":480028,"threads_started":1,"wal-append.queue_time_us":771294}
I20260924 14:31:22.830642 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:22.801084 31429 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.765640 31344 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.831830 26400 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:22.804013 26398 raft_consensus.cc:2379] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:22.799803 26402 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:22.745177 31492 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:22.745602 30975 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:22.801117 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.798197 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:22.807633 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.845101 27182 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 10.
I20260924 14:31:22.847227 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:22.810763 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:22.814390 31345 consensus_meta.cc:287] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.705s	user 0.013s	sys 0.000s
I20260924 14:31:22.760914 31277 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.851148 26406 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:22.816203 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:22.853055 26249 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:22.822424 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:22.775277 26403 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
W20260924 14:31:22.803709 31468 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.378s	user 0.003s	sys 0.002s
I20260924 14:31:22.822540 31514 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:22.859680 31514 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:22.823249 31175 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.870157 31277 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.857769 30975 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:22.873066 31266 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:22.873979 30975 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:22.874843 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 52518.5us
I20260924 14:31:22.875538 31405 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:22.875761 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:22.876142 31422 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:22.877933 31405 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:22.878615 31405 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:22.878890 31422 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:22.879733 31515 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:22.879951 26401 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260924 14:31:22.880424 31515 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:22.880697 31501 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:22.881572 26402 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:22.892055 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.893718 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.895711 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:22.896487 27192 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:22.895748 26401 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:22.899044 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:22.862902 31514 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:22.905562 26397 raft_consensus.cc:2396] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260924 14:31:22.906167 26251 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:22.912535 31005 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:22.914108 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:22.907617 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.841476 31492 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:22.842281 31212 consensus_queue.cc:1046] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260924 14:31:22.912335 31515 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.920140 27191 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:22.920341 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:22.919272 27192 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:22.922623 26369 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:22.907285 26252 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:22.930177 26400 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:22.935083 27189 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260924 14:31:22.951207 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:22.949824 31277 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:22.962973 31277 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:22.974077 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:22.950690 31514 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:31:22.976459 31514 raft_consensus.cc:686] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:22.976643 26404 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260924 14:31:22.977130 26404 pending_rounds.cc:76] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 9
I20260924 14:31:22.977336 26404 pending_rounds.cc:97] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260924 14:31:22.977562 26404 raft_consensus.cc:2866] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:22.977645 31514 consensus_queue.cc:237] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:22.978912 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1057) took 1315 ms. Trace:
I20260924 14:31:22.979636 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1061) took 1309 ms. Trace:
I20260924 14:31:22.982522 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1058) took 1314 ms. Trace:
I20260924 14:31:22.983479 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1059) took 1314 ms. Trace:
I20260924 14:31:22.985790 31277 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:22.986263 26295 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:22.987567 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1047) took 2052 ms. Trace:
W20260924 14:31:22.990567 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 278488us
I20260924 14:31:22.991372 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:22.991302 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:22.991707 27123 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:22.998291 26327 rpcz_store.cc:276] 0924 14:31:21.669821 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.781265 (+111444us) service_pool.cc:227] Handling call
0924 14:31:21.796156 (+ 14891us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:21.799614 (+  3458us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:21.799659 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:21.805286 (+  5627us) log.cc:842] Serialized 26 byte log entry
0924 14:31:21.855191 (+ 49905us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b48000c21d8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:31:22.306095 (+450904us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:22.438636 (+132541us) log.cc:842] Serialized 14 byte log entry
0924 14:31:22.979090 (+540454us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:22.979189 (+    99us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:22.979251 (+    62us) raft_consensus.cc:1671] finished
0924 14:31:22.979371 (+   120us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:22.979561 (+   190us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":554833,"prepare.queue_time_us":163699,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":545217,"raft.run_cpu_time_us":2005,"raft.run_wall_time_us":52628,"spinlock_wait_cycles":12942080,"thread_start_us":488833,"threads_started":2,"wal-append.queue_time_us":536504}
I20260924 14:31:22.998634 26331 rpcz_store.cc:276] 0924 14:31:21.669261 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.669381 (+   120us) service_pool.cc:227] Handling call
0924 14:31:21.669802 (+   421us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:21.683082 (+ 13280us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:21.683128 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:21.700002 (+ 16874us) log.cc:842] Serialized 26 byte log entry
0924 14:31:21.986072 (+286070us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:22.474626 (+488554us) log.cc:842] Serialized 14 byte log entry
0924 14:31:22.803361 (+328735us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b24004e1fb8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b627ecf4 00007f09b627cdfb 00007f09b630b559 00007f09b630b1e5 00007f09b630a82d 00007f09b8fb946e 00007f09b8fd0c0c 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551
0924 14:31:22.983037 (+179676us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:22.983116 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:22.983158 (+    42us) raft_consensus.cc:1671] finished
0924 14:31:22.983270 (+   112us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:22.983396 (+   126us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":600164,"prepare.queue_time_us":297813,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3016,"raft.queue_time_us":534535,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":13073,"spinlock_wait_cycles":34730752,"thread_start_us":692744,"threads_started":2,"wal-append.queue_time_us":555085}
I20260924 14:31:22.993221 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:23.002352 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:22.998363 26323 rpcz_store.cc:276] 0924 14:31:21.667797 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.667944 (+   147us) service_pool.cc:227] Handling call
0924 14:31:21.672514 (+  4570us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:21.672971 (+   457us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:21.673011 (+    40us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:21.685669 (+ 12658us) log.cc:842] Serialized 26 byte log entry
0924 14:31:22.117645 (+431976us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:22.472237 (+354592us) log.cc:842] Serialized 14 byte log entry
0924 14:31:22.982232 (+509995us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:22.982302 (+    70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:22.982341 (+    39us) raft_consensus.cc:1671] finished
0924 14:31:22.982353 (+    12us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:22.982459 (+   106us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":525554,"prepare.queue_time_us":319376,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":39,"raft.queue_time_us":511100,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":19228,"thread_start_us":646588,"threads_started":2,"wal-append.queue_time_us":594519}
I20260924 14:31:23.003736 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:23.004204 26398 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:22.992064 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:22.999518 27191 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:22.996397 27126 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:23.007668 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:23.011719 26399 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:23.012220 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:23.013365 27190 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:31:23.015189 26399 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:23.018657 26400 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:23.020454 26398 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:23.006398 26325 rpcz_store.cc:276] 0924 14:31:20.935067 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.282729 (+347662us) service_pool.cc:227] Handling call
0924 14:31:21.339161 (+ 56432us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:21.363739 (+ 24578us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:21.363775 (+    36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:21.590657 (+226882us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b2c001e1fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:21.601710 (+ 11053us) log.cc:842] Serialized 26 byte log entry
0924 14:31:22.041820 (+440110us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:22.554196 (+512376us) log.cc:842] Serialized 14 byte log entry
0924 14:31:22.987043 (+432847us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:22.987122 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:22.987165 (+    43us) raft_consensus.cc:1671] finished
0924 14:31:22.987274 (+   109us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:22.987496 (+   222us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":503370,"prepare.queue_time_us":386544,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":459940,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":31085,"spinlock_wait_cycles":49797376,"thread_start_us":810957,"threads_started":2,"wal-append.queue_time_us":658363}
I20260924 14:31:22.998153 26329 rpcz_store.cc:276] 0924 14:31:21.663388 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.663584 (+   196us) service_pool.cc:227] Handling call
0924 14:31:21.663970 (+   386us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:21.696192 (+ 32222us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:21.696234 (+    42us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:21.734132 (+ 37898us) log.cc:842] Serialized 26 byte log entry
0924 14:31:22.055687 (+321555us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:22.427848 (+372161us) log.cc:842] Serialized 14 byte log entry
0924 14:31:22.978580 (+550732us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:22.978654 (+    74us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:22.978705 (+    51us) raft_consensus.cc:1671] finished
0924 14:31:22.978716 (+    11us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:22.978853 (+   137us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":658986,"prepare.queue_time_us":381240,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":586682,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":11806,"thread_start_us":567426,"threads_started":2,"wal-append.queue_time_us":405325}
I20260924 14:31:23.030885 27191 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:23.033241 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:23.035535 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:23.036593 27189 raft_consensus.cc:3037] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:23.037412 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:23.065153 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:23.066871 26255 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:23.069772 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:23.072330 26258 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:23.073302 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:23.075313 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:23.075739 26295 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:23.076622 26330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1055) took 1415 ms (client timeout 1727 ms). Trace:
W20260924 14:31:23.077178 26330 rpcz_store.cc:269] 0924 14:31:21.661602 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.661736 (+   134us) service_pool.cc:227] Handling call
0924 14:31:23.076434 (+1414698us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400237654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:23.076564 (+   130us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2504468736}
I20260924 14:31:23.073968 26258 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:23.079732 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:23.080588 26255 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:23.084275 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:23.085988 26251 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260924 14:31:23.087152 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:23.087357 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:23.088478 26257 raft_consensus.cc:2354] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:23.089178 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:23.019968 31330 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:23.084802 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:23.029518 31175 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:23.042102 31445 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:23.020052 31329 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:23.112749 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:23.114380 26249 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:23.090085 26256 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:23.104296 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:23.099550 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:23.116016 26255 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:23.079041 30975 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:23.084755 31422 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:23.072383 31266 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:23.038190 31344 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:23.041455 31476 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:23.133590 26257 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:31:23.134793 26251 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:23.138208 26256 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:31:23.108894 27189 raft_consensus.cc:2454] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:23.078747 31399 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260924 14:31:23.098433 31424 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:23.150646 27191 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260924 14:31:23.079653 31451 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260924 14:31:23.159706 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:23.129635 31445 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:23.130210 31121 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260924 14:31:23.213552 26399 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:23.122797 31175 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:23.216365 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 53924.3us
I20260924 14:31:23.139324 31266 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:23.153558 30975 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:23.128567 31329 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:23.206077 31374 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:23.220572 31429 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:23.244042 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 168408us
W20260924 14:31:23.237480 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 77954.4us
I20260924 14:31:23.152761 31422 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:23.214947 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:23.237941 31501 consensus_queue.cc:1046] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260924 14:31:23.238231 31330 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:23.207394 31279 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260924 14:31:23.207692 31345 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260924 14:31:23.157840 31344 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:23.239135 31266 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:23.322402 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 159309us
I20260924 14:31:23.242954 31392 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:23.206038 31424 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:23.240726 31417 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:23.332954 26371 leader_election.cc:398] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:23.345386 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:23.346649 26371 leader_election.cc:401] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:23.285898 31330 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:23.323503 31486 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260924 14:31:23.354482 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:23.356058 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:23.290479 31178 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:23.361101 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:23.329038 31500 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:23.379608 26330 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:31:23.361943 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 127979us
I20260924 14:31:23.255177 31492 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:23.337894 31080 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:31:23.363229 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:23.233665 31526 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.167s
W20260924 14:31:23.427690 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 64561.7us
I20260924 14:31:23.257458 31374 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:23.321363 31405 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:23.376906 31451 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:23.287263 31429 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:23.356952 31417 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:23.258566 31521 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.173s
W20260924 14:31:23.473811 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 213671us
I20260924 14:31:23.475638 26253 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:23.428472 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:23.362159 31330 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:23.362353 31400 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 17
I20260924 14:31:23.430558 31476 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260924 14:31:23.482187 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 53807.8us
I20260924 14:31:23.377931 31277 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:23.364737 31508 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:23.381949 31500 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:23.377831 31531 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:23.494632 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:23.437625 31503 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:23.484110 31455 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260924 14:31:23.475270 26295 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:23.384593 31178 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:23.363695 31445 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:23.552547 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.004s	sys 0.020s
I20260924 14:31:23.484494 31345 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:31:23.470835 31111 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
W20260924 14:31:23.550093 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 74903.1us
W20260924 14:31:23.555676 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.004s	sys 0.020s
I20260924 14:31:23.497814 31476 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:23.561785 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 528649us
I20260924 14:31:23.508536 31277 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:23.509305 31500 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:23.562443 26297 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:23.478164 31495 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260924 14:31:23.527364 31518 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:23.472765 31451 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:23.575874 31530 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260924 14:31:23.528537 31491 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:23.576905 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1743) took 1181 ms. Trace:
I20260924 14:31:23.589902 26404 rpcz_store.cc:276] 0924 14:31:22.395286 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:22.395419 (+   133us) service_pool.cc:227] Handling call
0924 14:31:22.438599 (+ 43180us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:22.978021 (+539422us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:23.576677 (+598656us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b540018ced4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:23.576847 (+   170us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1236461824}
I20260924 14:31:23.364943 30975 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:23.564256 31476 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:23.575884 31529 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260924 14:31:23.567272 31277 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:23.567497 31175 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:23.563688 31335 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:23.577286 31528 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260924 14:31:23.559686 31535 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:23.578320 31525 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260924 14:31:23.577242 31405 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:23.571743 31329 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:23.570711 31345 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:23.546734 31503 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:23.576907 31429 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:23.613335 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1041) took 2702 ms. Trace:
I20260924 14:31:23.607059 31539 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:23.591885 31422 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:23.546927 31531 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:23.616691 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 284035us
W20260924 14:31:23.622356 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 277083us
I20260924 14:31:23.633165 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:23.648743 26256 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:31:23.613166 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 210573us
I20260924 14:31:23.633827 26324 rpcz_store.cc:276] 0924 14:31:20.903727 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.174859 (+271132us) service_pool.cc:227] Handling call
0924 14:31:21.212560 (+ 37701us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:21.351541 (+138981us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:21.351577 (+    36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:21.562569 (+210992us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b2c001e1fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:21.563097 (+   528us) log.cc:842] Serialized 26 byte log entry
0924 14:31:22.041436 (+478339us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:22.398952 (+357516us) log.cc:842] Serialized 14 byte log entry
0924 14:31:22.978004 (+579052us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b18004c68a0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09b6261990 00007f09b6423864 00007f09b6423155 00007f09b93463d0 00007f09b9358576 00007f09b93690d7 00007f09b9369058
0924 14:31:22.978173 (+   169us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:23.605901 (+627728us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540012e654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:23.605970 (+    69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:23.606034 (+    64us) raft_consensus.cc:1671] finished
0924 14:31:23.606053 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:23.606225 (+   172us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":50788,"prepare.queue_time_us":385991,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":8674,"raft.queue_time_us":612362,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":201,"spinlock_wait_cycles":327227648,"thread_start_us":787915,"threads_started":2,"wal-append.queue_time_us":704134}
I20260924 14:31:23.649887 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:23.639478 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1078) took 1589 ms. Trace:
I20260924 14:31:23.625387 31175 raft_consensus.cc:2781] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:23.651188 26332 rpcz_store.cc:276] 0924 14:31:22.049941 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:22.460177 (+410236us) service_pool.cc:227] Handling call
0924 14:31:22.460488 (+   311us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:22.982168 (+521680us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:22.982197 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:22.982653 (+   456us) log.cc:842] Serialized 26 byte log entry
0924 14:31:23.122896 (+140243us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:31:23.122957 (+    61us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:23.123072 (+   115us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:23.639240 (+516168us) raft_consensus.cc:1671] finished
0924 14:31:23.639259 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:23.639401 (+   142us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":521293,"raft.queue_time_us":522365,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"thread_start_us":139764,"threads_started":1,"wal-append.queue_time_us":395373}
I20260924 14:31:23.596108 31491 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:23.626130 31329 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election won for term 15
W20260924 14:31:23.726939 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45257.4us
I20260924 14:31:23.615065 31335 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:23.724823 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 75009.4us
I20260924 14:31:23.622689 31501 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:23.736337 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1491) took 1063 ms. Trace:
I20260924 14:31:23.860744 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1794) took 1035 ms. Trace:
I20260924 14:31:23.865064 26250 rpcz_store.cc:276] 0924 14:31:22.825425 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:22.825567 (+   142us) service_pool.cc:227] Handling call
0924 14:31:22.856454 (+ 30887us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:22.856688 (+   234us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:22.856712 (+    24us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:22.886660 (+ 29948us) log.cc:842] Serialized 26 byte log entry
0924 14:31:23.080181 (+193521us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:23.080214 (+    33us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:23.080280 (+    66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:23.860456 (+780176us) raft_consensus.cc:1671] finished
0924 14:31:23.860486 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:23.860666 (+   180us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":193226,"threads_started":1,"wal-append.queue_time_us":321162}
I20260924 14:31:23.646399 31080 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:23.647655 31544 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:23.636526 31392 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:23.625566 31344 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:23.633270 31424 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:23.633713 31345 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:23.734579 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:23.870504 26401 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.007s	sys 0.019s
I20260924 14:31:23.726673 31526 consensus_queue.cc:1225] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260924 14:31:23.618849 31535 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:23.883839 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 321415us
I20260924 14:31:23.905917 26297 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:23.907470 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:23.680389 31492 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:23.892077 26397 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.000s	sys 0.030s
W20260924 14:31:23.894425 26401 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.007s	sys 0.019s
I20260924 14:31:23.857241 31501 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:23.857147 31405 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:23.864254 26403 rpcz_store.cc:276] 0924 14:31:22.673203 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:22.854350 (+181147us) service_pool.cc:227] Handling call
0924 14:31:22.854727 (+   377us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:22.857136 (+  2409us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:22.857171 (+    35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:22.887996 (+ 30825us) log.cc:842] Serialized 26 byte log entry
0924 14:31:22.996588 (+108592us) raft_consensus.cc:1631] Marking committed up to 3
0924 14:31:22.996635 (+    47us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:22.996710 (+    75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:23.736108 (+739398us) raft_consensus.cc:1671] finished
0924 14:31:23.736131 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:23.736266 (+   135us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2004,"raft.queue_time_us":2648,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":9454,"thread_start_us":74789,"threads_started":1,"wal-append.queue_time_us":249344}
W20260924 14:31:23.903762 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 169249us
I20260924 14:31:23.732744 31422 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:23.931497 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:23.904091 31518 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:23.873533 31175 raft_consensus.cc:686] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:23.728170 31538 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.184s
W20260924 14:31:23.912655 26397 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.000s	sys 0.030s
I20260924 14:31:23.914414 31417 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:23.900475 31374 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:23.732017 31543 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.189s
I20260924 14:31:23.729399 31541 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.191s
I20260924 14:31:23.868435 31329 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:23.917685 31344 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:23.918569 31509 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:23.648154 31539 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:23.929090 31111 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:23.977316 26403 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:31:23.934980 31422 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:23.984859 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:23.922493 31178 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:23.922981 31405 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:23.891099 31544 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:23.970239 31451 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:24.026782 31533 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
I20260924 14:31:23.990110 31509 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:24.003360 31422 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:24.035235 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:24.035917 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:23.985780 31536 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260924 14:31:23.951830 31518 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:24.012564 31546 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.045257 31491 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:24.046051 31561 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260924 14:31:24.049163 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.049863 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:23.970443 31080 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:24.060690 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:24.061627 26255 raft_consensus.cc:2396] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260924 14:31:23.977514 31501 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.040572 31451 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:24.062366 26406 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:31:24.055660 31462 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 1.723s	user 0.009s	sys 0.004s
I20260924 14:31:24.047971 31335 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:24.091372 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.092469 26330 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:24.042443 31503 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:24.042523 31554 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.042704 31495 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:24.070423 26257 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:24.043856 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:24.098060 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:24.098693 26325 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:24.099114 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 58682.9us
I20260924 14:31:24.101022 26323 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:31:24.106590 26325 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
W20260924 14:31:24.095566 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 368569us
I20260924 14:31:24.043582 31538 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260924 14:31:23.980816 31392 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.114279 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:24.008108 31552 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260924 14:31:24.066097 31551 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:24.062388 31531 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:24.096279 27124 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.014s	sys 0.023s
W20260924 14:31:24.121927 27124 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.014s	sys 0.024s
I20260924 14:31:24.110047 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:24.115242 26255 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:31:24.038295 31175 consensus_queue.cc:237] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:24.115964 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.071187 31542 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:24.132340 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:24.133028 26257 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:24.135751 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.137650 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.108652 31335 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.095700 31345 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:24.141026 26221 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:24.051663 31329 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.147320 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.143159 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1760) took 1140 ms. Trace:
I20260924 14:31:24.149320 26397 rpcz_store.cc:276] 0924 14:31:23.003051 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:23.005762 (+  2711us) service_pool.cc:227] Handling call
0924 14:31:23.006045 (+   283us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:23.006239 (+   194us) raft_consensus.cc:1520] Early marking committed up to index 11
0924 14:31:23.947674 (+941435us) log.cc:842] Serialized 14 byte log entry
0924 14:31:24.141983 (+194309us) log.cc:842] Serialized 14 byte log entry
0924 14:31:24.142873 (+   890us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:24.142915 (+    42us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:24.142950 (+    35us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:24.143004 (+    54us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:24.143102 (+    98us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1022368,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":1319,"thread_start_us":1096576,"threads_started":2}
I20260924 14:31:24.152247 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:24.153218 26399 raft_consensus.cc:2396] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260924 14:31:24.157083 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:24.158857 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.060763 31546 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.119138 31451 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260924 14:31:24.150326 31526 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:24.158495 31429 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:24.165180 31514 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:24.170658 31345 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:24.164302 31344 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:24.164989 31495 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:24.155814 31335 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.186810 31526 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:24.208106 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1624) took 2623 ms. Trace:
I20260924 14:31:24.180223 31556 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.158823 31535 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:24.186450 31173 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.210867 26332 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:24.113365 31554 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.227697 27192 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:24.095252 31509 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:24.229453 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:24.234176 31542 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.241133 27190 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260924 14:31:24.241609 27188 raft_consensus.cc:1203] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 13.10->[14.11-16.12]   Dedup: 14.11->[16.12-16.12]
I20260924 14:31:24.269222 27188 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17. Ops: [14.11-16.12]
I20260924 14:31:24.242965 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.243923 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.246532 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:24.246835 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.247629 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.164517 31553 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.095s
I20260924 14:31:24.249363 26405 rpcz_store.cc:276] 0924 14:31:21.573526 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:21.573740 (+   214us) service_pool.cc:227] Handling call
0924 14:31:21.583283 (+  9543us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:21.583859 (+   576us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:21.583900 (+    41us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:21.602237 (+ 18337us) log.cc:842] Serialized 26 byte log entry
0924 14:31:22.056521 (+454284us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:22.056568 (+    47us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:22.056658 (+    90us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:23.577665 (+1521007us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:24.187132 (+609467us) raft_consensus.cc:1671] finished
0924 14:31:24.187164 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:24.196816 (+  9652us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8403840,"thread_start_us":449240,"threads_started":1,"wal-append.queue_time_us":565722}
I20260924 14:31:24.204211 31405 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:24.284523 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.200354 31370 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:24.211509 31514 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:24.228333 31566 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.208377 31345 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:24.214422 31107 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.140164 31551 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.270747 27194 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:24.254617 31526 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:24.283619 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.215636 31539 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:24.285866 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.248780 31544 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:24.328764 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 82297us
I20260924 14:31:24.183957 31568 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
W20260924 14:31:24.329835 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 420477us
W20260924 14:31:24.333134 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 44250.4us
W20260924 14:31:24.330000 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106561us
I20260924 14:31:24.376809 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.374017 26295 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:24.334558 31514 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:24.378984 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.337196 31491 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.377453 26331 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:24.357518 31345 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:24.264400 31173 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.284564 31542 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:24.387470 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.358160 31429 raft_consensus.cc:686] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:24.388476 31563 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260924 14:31:24.388761 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.335099 31518 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:24.335361 31370 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.390197 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:24.414898 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.390925 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.390837 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.267525 31545 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260924 14:31:24.394361 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.359615 31367 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:24.395205 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.387840 31539 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260924 14:31:24.387888 31565 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260924 14:31:24.264526 31556 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:24.403422 26323 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:24.364817 31107 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:24.410622 31400 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.915s	user 0.009s	sys 0.003s
W20260924 14:31:24.413189 31277 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.800s	user 0.010s	sys 0.000s
I20260924 14:31:24.395212 31491 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.446029 31400 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:24.446148 31277 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.447515 31491 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.447688 26400 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:24.415807 27192 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:24.421298 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.394296 31451 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:24.431306 31569 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.431892 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:24.406536 31560 log.cc:924] Time spent T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260924 14:31:24.389567 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.436105 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:24.467231 26331 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:24.414033 31476 consensus_meta.cc:287] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.805s	user 0.007s	sys 0.003s
I20260924 14:31:24.400008 31405 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:24.474097 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.440267 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.422559 31544 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:24.431373 31571 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260924 14:31:24.455672 26295 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:24.460841 31266 consensus_meta.cc:287] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.106s	user 0.012s	sys 0.002s
W20260924 14:31:24.458292 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 1.62176e+06us
I20260924 14:31:24.362010 31344 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.460239 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.469416 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:24.397105 31454 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260924 14:31:24.439854 31546 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:24.454093 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:24.570250 26325 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:24.448382 31277 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.481367 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:24.357688 31566 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:24.456945 26399 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:24.473315 31569 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.498085 31544 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:24.475248 26327 raft_consensus.cc:2379] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:24.567958 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 112380us
I20260924 14:31:24.580574 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:24.581298 26323 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:24.473697 31451 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.603732 26324 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260924 14:31:24.573580 31462 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260924 14:31:24.606060 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1081) took 1275 ms. Trace:
I20260924 14:31:24.610668 26329 rpcz_store.cc:276] 0924 14:31:23.330513 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:23.330647 (+   134us) service_pool.cc:227] Handling call
0924 14:31:23.331021 (+   374us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:23.517959 (+186938us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:23.518011 (+    52us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:23.554361 (+ 36350us) log.cc:842] Serialized 26 byte log entry
0924 14:31:23.932881 (+378520us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:23.933468 (+   587us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:23.933548 (+    80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:24.605749 (+672201us) raft_consensus.cc:1671] finished
0924 14:31:24.605770 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:24.605949 (+   179us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1834,"raft.queue_time_us":149322,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":13411,"thread_start_us":498405,"threads_started":2,"wal-append.queue_time_us":532505}
W20260924 14:31:24.608635 31559 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260924 14:31:24.609323 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:24.540423 31335 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:24.617431 26328 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:24.618763 26327 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:24.620626 26323 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:24.626786 26323 raft_consensus.cc:1226] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:24.629343 26323 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:24.574613 31454 consensus_queue.cc:1225] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260924 14:31:24.623772 26325 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:24.480288 31476 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.638415 26331 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
W20260924 14:31:24.619184 31330 consensus_meta.cc:287] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.116s	user 0.012s	sys 0.002s
I20260924 14:31:24.496742 31405 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:24.560122 31266 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.662123 26406 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260924 14:31:24.660497 26327 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:24.605114 31539 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:24.663748 31330 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.610523 31451 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.673890 31551 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.676219 31121 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:24.705765 31121 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 7 FOLLOWER]: Advancing to term 8
W20260924 14:31:24.672264 31572 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.288s	user 0.000s	sys 0.000s
I20260924 14:31:24.672653 31580 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.671921 31539 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:24.671439 31173 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.574744 31429 consensus_queue.cc:237] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:24.674849 31574 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260924 14:31:24.644461 31573 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.302s
I20260924 14:31:24.723838 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:24.684021 31107 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:24.675034 31579 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:24.704883 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:24.677784 31588 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260924 14:31:24.706499 31451 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:24.707054 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.671890 31584 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:24.678735 31405 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:24.671296 31583 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260924 14:31:24.686964 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.644757 31335 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.691888 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1243) took 1586 ms. Trace:
I20260924 14:31:24.840215 26252 rpcz_store.cc:276] 0924 14:31:23.105518 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:23.105764 (+   246us) service_pool.cc:227] Handling call
0924 14:31:23.106179 (+   415us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:23.381536 (+275357us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:31:23.381800 (+   264us) log.cc:842] Serialized 14 byte log entry
0924 14:31:23.663913 (+282113us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:23.663989 (+    76us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:23.664052 (+    63us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:24.691660 (+1027608us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54004f9f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:24.691706 (+    46us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:24.691829 (+   123us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":497,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":2628,"raft.queue_time_us":191167,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":14838,"spinlock_wait_cycles":632707456,"thread_start_us":281790,"threads_started":1}
I20260924 14:31:24.674398 31569 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:24.712260 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.715266 31330 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:24.715499 31523 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.769s	user 0.003s	sys 0.030s
I20260924 14:31:24.641928 31554 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.718721 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:24.644985 31594 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260924 14:31:24.671607 31570 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260924 14:31:24.691644 31553 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.716960 31538 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.721755 31539 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:24.682928 31542 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:24.725267 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.862488 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:24.863588 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.677732 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1489) took 2005 ms. Trace:
W20260924 14:31:24.695686 31500 consensus_meta.cc:287] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.108s	user 0.015s	sys 0.005s
I20260924 14:31:24.864818 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.865167 31500 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.867161 31500 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:24.699362 31111 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.698s	user 0.005s	sys 0.005s
I20260924 14:31:24.727393 26332 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260924 14:31:24.867811 31111 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:24.702121 31527 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.769s	user 0.000s	sys 0.008s
I20260924 14:31:24.725675 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1244) took 1105 ms. Trace:
W20260924 14:31:24.693296 31422 consensus_meta.cc:287] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.002s
I20260924 14:31:24.864488 26402 rpcz_store.cc:276] 0924 14:31:22.672367 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:22.834566 (+162199us) service_pool.cc:227] Handling call
0924 14:31:22.835035 (+   469us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:22.881987 (+ 46952us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:24.677494 (+1795507us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b54002756d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:24.677671 (+   177us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":106571008}
I20260924 14:31:24.869374 31422 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.870218 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.870412 26330 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:24.717965 31551 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:24.871486 31422 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.694368 31566 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:24.731828 31579 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.745307 31277 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.872550 26404 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:24.873080 31277 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.875123 31277 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.702497 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1651) took 1750 ms. Trace:
I20260924 14:31:24.869148 26249 rpcz_store.cc:276] 0924 14:31:23.620395 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:23.620639 (+   244us) service_pool.cc:227] Handling call
0924 14:31:23.621092 (+   453us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:23.971560 (+350468us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:23.971593 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:23.984310 (+ 12717us) log.cc:842] Serialized 26 byte log entry
0924 14:31:24.164601 (+180291us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:24.164646 (+    45us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:24.164760 (+   114us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:24.725483 (+560723us) raft_consensus.cc:1671] finished
0924 14:31:24.725505 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:24.725623 (+   118us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8857,"raft.queue_time_us":289201,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":5733,"thread_start_us":472051,"threads_started":2,"wal-append.queue_time_us":239777}
I20260924 14:31:24.840354 31584 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:24.685530 31540 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:24.871363 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.748526 27181 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
W20260924 14:31:24.701212 31522 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.752s	user 0.000s	sys 0.011s
I20260924 14:31:24.839946 31573 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:24.877955 31573 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:24.878731 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.879580 26402 raft_consensus.cc:2396] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260924 14:31:24.880759 26403 raft_consensus.cc:1226] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 7. Current term is 8. Ops: []
I20260924 14:31:24.676772 31266 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.845256 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:24.849447 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.887348 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:24.688342 31370 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.853099 27182 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:24.875730 26401 rpcz_store.cc:276] 0924 14:31:22.951996 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:22.952130 (+   134us) service_pool.cc:227] Handling call
0924 14:31:22.962656 (+ 10526us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:22.993093 (+ 30437us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:22.993124 (+    31us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:23.028269 (+ 35145us) log.cc:842] Serialized 52 byte log entry
0924 14:31:23.912205 (+883936us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:23.912282 (+    77us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:23.912384 (+   102us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:24.702261 (+789877us) raft_consensus.cc:1671] finished
0924 14:31:24.702288 (+    27us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:24.702438 (+   150us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":30022,"raft.queue_time_us":57377,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":13116,"thread_start_us":806003,"threads_started":1,"wal-append.queue_time_us":993147}
I20260924 14:31:24.874342 31579 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.854843 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.885753 31538 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.859302 31553 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.893692 31538 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:24.894017 31553 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:24.852421 31335 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:24.852043 31594 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:24.895295 31594 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:24.749379 31515 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.775s	user 0.016s	sys 0.030s
W20260924 14:31:24.896544 31515 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.922s	user 0.016s	sys 0.030s
I20260924 14:31:24.871740 31551 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.839425 31518 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:24.899196 31551 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:24.683796 31556 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:24.685197 31544 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:24.876534 31476 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:24.899808 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.900833 31579 consensus_queue.cc:1046] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:31:24.694947 31418 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:24.883878 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:24.899144 31518 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:24.876919 31540 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:24.901085 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:24.737458 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:24.908056 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:24.916570 31005 raft_consensus.cc:3032] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:31:24.916767 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:24.917236 31005 raft_consensus.cc:729] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:24.918416 31005 consensus_queue.cc:260] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:24.920090 26406 raft_consensus.cc:2396] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260924 14:31:24.920740 31506 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:24.682452 31514 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:24.926533 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:24.929761 26405 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260924 14:31:24.906440 31491 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:24.929898 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1804) took 1995 ms. Trace:
I20260924 14:31:24.942296 26253 rpcz_store.cc:276] 0924 14:31:22.934248 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:22.939573 (+  5325us) service_pool.cc:227] Handling call
0924 14:31:22.940040 (+   467us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:23.484057 (+544017us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:24.929667 (+1445610us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b5400716e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:24.929840 (+   173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1231195520}
I20260924 14:31:24.904512 31418 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:24.956147 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:24.912655 31476 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:25.028064 26398 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:24.913014 31451 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:31:25.031972 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:25.025898 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 99144.2us
I20260924 14:31:25.032994 31405 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:25.033453 31555 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:25.036532 31555 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:25.036312 31405 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:25.037005 31555 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:25.038534 31405 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:25.040127 31418 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:25.039760 31540 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:25.009518 31335 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:24.930936 31277 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:25.033290 31370 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:25.042557 31370 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:25.044735 31370 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:25.029254 31005 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:25.049284 31514 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:24.725734 31580 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:25.028194 31491 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:24.910789 31534 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.119s
I20260924 14:31:25.051935 31580 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:25.052299 31534 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:25.052834 31534 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:25.053851 31534 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:25.055757 31518 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:25.051013 31491 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:24.906756 31345 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:25.059012 31491 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:25.059536 26397 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:25.060065 31491 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:25.060611 31491 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:25.061776 31345 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:25.062732 31491 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:25.029397 31584 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:25.072839 31573 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:25.080581 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:25.081629 26406 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
W20260924 14:31:25.123759 27125 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.046s	sys 0.100s
W20260924 14:31:25.124320 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.046s	sys 0.100s
I20260924 14:31:25.129774 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.130656 26252 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:25.132947 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:25.136440 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 50775.3us
I20260924 14:31:25.137193 26369 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:25.133805 26252 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:25.086700 27126 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:25.067041 31121 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:25.164259 26326 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.004s	sys 0.016s
W20260924 14:31:25.164705 26326 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.004s	sys 0.016s
I20260924 14:31:25.160441 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:25.081676 31526 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.000s
W20260924 14:31:25.171170 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 84580.5us
I20260924 14:31:25.171401 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:25.183602 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:25.184273 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.010s	sys 0.011s
I20260924 14:31:25.172223 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:25.184671 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.010s	sys 0.011s
I20260924 14:31:25.055425 31534 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:25.184713 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:25.174535 31175 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.007s	sys 0.013s
W20260924 14:31:25.187594 31175 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.008s	sys 0.013s
I20260924 14:31:25.187058 26331 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:25.189601 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.191763 26331 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:25.194047 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.195693 26331 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:25.197325 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:25.198966 26331 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:25.089126 31451 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:25.203003 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.000s	sys 0.024s
W20260924 14:31:25.163897 31080 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.043s	sys 0.105s
W20260924 14:31:25.180922 31507 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 2.192s	user 0.006s	sys 0.020s
I20260924 14:31:25.206153 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.213246 26323 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
W20260924 14:31:25.208401 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.000s	sys 0.024s
I20260924 14:31:25.209743 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:25.214931 26330 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:25.205274 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.165691 31491 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:25.217592 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:25.218531 26252 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:25.185068 31107 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:25.180310 31541 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:25.221184 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:25.209326 31080 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.043s	sys 0.105s
I20260924 14:31:25.051056 31555 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:25.222743 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.164398 31518 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260924 14:31:25.212059 31451 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:25.231570 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:25.214855 31507 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 2.225s	user 0.006s	sys 0.020s
W20260924 14:31:25.236665 26251 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.003s	sys 0.010s
I20260924 14:31:25.203372 31526 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:25.242632 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.219878 31584 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:25.320417 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:25.278998 26251 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.003s	sys 0.010s
I20260924 14:31:25.185515 31345 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260924 14:31:25.220057 31111 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:25.226310 31107 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:25.275370 26323 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:25.222916 31537 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:25.268167 26252 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:25.300913 31121 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:25.302487 31370 consensus_queue.cc:1046] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:25.269912 31556 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:25.229614 31491 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:25.237501 31544 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:25.307572 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:25.342036 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.311278 27182 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260924 14:31:25.312670 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.313721 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.314865 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:25.249536 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:25.318676 26324 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.014s	sys 0.019s
W20260924 14:31:25.293499 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 59313.2us
I20260924 14:31:25.323915 26406 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:25.323618 26330 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
W20260924 14:31:25.325954 26325 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.019s	sys 0.035s
W20260924 14:31:25.344990 26324 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.014s	sys 0.019s
I20260924 14:31:25.345443 26224 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:25.346040 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:25.347297 26325 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.019s	sys 0.035s
I20260924 14:31:25.327698 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.328965 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:25.328207 31344 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.001s	sys 0.025s
W20260924 14:31:25.340799 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.010s	sys 0.022s
I20260924 14:31:25.348834 27194 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:31:25.349033 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 99550.3us
W20260924 14:31:25.345286 31445 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.204s	user 0.016s	sys 0.028s
I20260924 14:31:25.349668 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:25.319852 31515 consensus_queue.cc:1046] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
W20260924 14:31:25.335741 31329 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.034s	sys 0.020s
I20260924 14:31:25.354928 31039 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:25.328840 31107 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:25.350466 26331 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
W20260924 14:31:25.371030 31445 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.016s	sys 0.029s
I20260924 14:31:25.342571 31537 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:25.373385 26369 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:25.239910 31594 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:25.350697 31556 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:25.369357 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.011s	sys 0.023s
W20260924 14:31:25.368937 31344 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.003s	sys 0.025s
I20260924 14:31:25.378495 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.386611 26323 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:25.350973 31539 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:25.328583 31111 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:25.361073 31580 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260924 14:31:25.387953 31329 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.038s	sys 0.020s
I20260924 14:31:25.352147 31370 raft_consensus.cc:3032] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:25.325660 31584 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:25.432312 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 1.05575e+06us
W20260924 14:31:25.407516 26296 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.012s	sys 0.020s
I20260924 14:31:25.433454 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:25.374317 31564 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:25.405107 31594 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:25.410115 26369 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:25.397742 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48196.7us
I20260924 14:31:25.414602 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.411394 27194 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:31:25.409861 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:25.423218 31429 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.006s	sys 0.020s
W20260924 14:31:25.399080 26403 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.017s	sys 0.033s
W20260924 14:31:25.452729 31429 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.007s	sys 0.020s
W20260924 14:31:25.452883 26403 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.017s	sys 0.033s
I20260924 14:31:25.435072 26221 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:25.390069 31039 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:25.453830 31039 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:25.455374 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.459528 31539 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:25.460021 31539 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:25.460421 31539 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:25.435830 31515 consensus_queue.cc:1046] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260924 14:31:25.436009 31564 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:25.449887 27182 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:25.463364 31039 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:25.465463 31564 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:25.468040 27194 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:25.468529 31039 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:25.472069 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:25.434109 26296 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.018s	sys 0.022s
I20260924 14:31:25.473951 31564 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:25.474757 31564 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:25.479673 31515 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:25.480402 31515 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:25.481961 27182 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:25.483100 27186 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260924 14:31:25.483044 31515 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:25.486943 31515 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:25.487214 26295 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:25.489265 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.489544 31329 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:25.493108 31429 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:25.408466 31606 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:25.432556 31491 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:25.424285 31580 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:25.407322 31556 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:25.416109 31518 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:25.501965 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:25.503100 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.503522 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.496861 31345 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:25.511354 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.525887 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:25.525679 31539 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:25.526458 26258 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:25.511834 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.527913 31539 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:25.528263 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:25.513490 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.509279 31329 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:25.523379 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.529928 26249 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:25.530143 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.535336 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.536559 26257 raft_consensus.cc:2396] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260924 14:31:25.537384 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.534411 26250 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:25.434520 31370 raft_consensus.cc:729] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:25.542457 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.519434 31580 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:25.509744 31606 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:25.549373 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.550275 26257 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:25.555557 26258 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:25.565052 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.521792 31518 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:25.570865 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1608) took 1190 ms. Trace:
I20260924 14:31:25.531965 31266 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:25.572000 26326 rpcz_store.cc:276] 0924 14:31:24.380006 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.593535 (+213529us) service_pool.cc:227] Handling call
0924 14:31:24.627360 (+ 33825us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:24.627806 (+   446us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:31:25.179635 (+551829us) log.cc:842] Serialized 14 byte log entry
0924 14:31:25.563262 (+383627us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:25.563323 (+    61us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:25.563362 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:25.570676 (+  7314us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:25.570800 (+   124us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12335,"prepare.queue_time_us":48909,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":3470,"thread_start_us":906929,"threads_started":2}
I20260924 14:31:25.574726 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:25.546483 31555 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:25.577005 27182 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:25.549196 31526 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:25.548718 31345 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:25.586838 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.571376 31521 raft_consensus.cc:3032] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:31:25.567042 31606 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:25.560856 31451 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:25.573231 31518 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:25.604811 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:25.580410 31266 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:25.614741 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:25.619984 31605 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.123s	user 0.000s	sys 0.002s
I20260924 14:31:25.620464 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.622126 26326 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:25.623210 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.616973 26324 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:25.623813 26326 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:25.623696 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:25.627128 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:25.628036 26402 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:25.629220 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:25.630596 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:25.631806 26327 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 21: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:25.633065 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:25.634629 26402 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:25.642738 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:25.643057 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:25.592080 31521 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:25.645785 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:25.648267 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:25.615649 31600 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260924 14:31:25.652979 26326 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260924 14:31:25.650449 31556 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:25.654728 31556 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:25.655129 31556 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:25.656563 31509 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:25.607914 31111 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:25.649113 31537 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:25.661051 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:25.599781 31370 consensus_queue.cc:260] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:25.666733 31537 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:25.667277 31537 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:25.669291 31537 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:25.650651 31555 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:25.671989 31555 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:25.674093 31555 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:25.676239 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:25.677418 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:25.679785 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.680966 31451 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 21 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:25.681237 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.681897 31584 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:25.682526 27191 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:25.686309 26332 raft_consensus.cc:2396] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260924 14:31:25.692709 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.648145 31329 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:25.692728 31521 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:25.694988 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1814) took 1726 ms. Trace:
I20260924 14:31:25.695359 26254 rpcz_store.cc:276] 0924 14:31:23.968203 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:23.968946 (+   743us) service_pool.cc:227] Handling call
0924 14:31:23.990439 (+ 21493us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:24.385552 (+395113us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:24.385582 (+    30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:24.385623 (+    41us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:24.385659 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:25.694764 (+1309105us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54000c0dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:25.694809 (+    45us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:25.694935 (+   126us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":908163328}
I20260924 14:31:25.685505 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.697232 27182 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:25.671802 31345 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:25.700625 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.700850 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.710646 26397 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:25.719815 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.695182 31521 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:25.722764 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.725775 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:25.663235 31564 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:25.735587 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:25.731387 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:25.725174 31614 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.081s	user 0.004s	sys 0.004s
I20260924 14:31:25.716210 31507 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:25.643528 31604 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.087s
I20260924 14:31:25.740263 26326 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
W20260924 14:31:25.732120 31585 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 1.139s	user 0.008s	sys 0.013s
I20260924 14:31:25.742410 31507 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:25.747042 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:25.750056 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1660) took 1615 ms. Trace:
I20260924 14:31:25.752324 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:25.760244 26400 rpcz_store.cc:276] 0924 14:31:24.135153 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.135277 (+   124us) service_pool.cc:227] Handling call
0924 14:31:24.135603 (+   326us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:24.448290 (+312687us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:25.749829 (+1301539us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54001c0f54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:25.750000 (+   171us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":716333952}
I20260924 14:31:25.761685 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:25.753764 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:25.763938 26257 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:25.754125 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.754848 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.756696 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.766645 26324 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:25.767743 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.769104 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:25.787262 26405 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260924 14:31:25.788486 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.714258 31107 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:25.791627 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:25.792313 27181 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:25.796257 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44040.4us
I20260924 14:31:25.797391 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:25.798789 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:25.800804 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:25.801594 27182 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:25.803503 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:25.804667 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:25.679817 31266 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:25.732414 31584 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260924 14:31:25.806731 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:25.807657 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1411) took 1561 ms. Trace:
I20260924 14:31:25.814565 27190 rpcz_store.cc:276] 0924 14:31:24.246194 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.293753 (+ 47559us) service_pool.cc:227] Handling call
0924 14:31:24.294139 (+   386us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:24.610211 (+316072us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:24.610241 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:24.610733 (+   492us) log.cc:842] Serialized 26 byte log entry
0924 14:31:24.727018 (+116285us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:24.727073 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:25.796044 (+1068971us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b540031f954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:25.796091 (+    47us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:25.796135 (+    44us) raft_consensus.cc:1671] finished
0924 14:31:25.796153 (+    18us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:25.807591 (+ 11438us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":925,"raft.queue_time_us":85,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":10698,"spinlock_wait_cycles":635363200,"thread_start_us":83041,"threads_started":1,"wal-append.queue_time_us":698011}
I20260924 14:31:25.816687 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.817683 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.811058 31546 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:25.790721 31519 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260924 14:31:25.809558 31344 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:25.808995 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:25.790009 31617 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260924 14:31:25.811062 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.826154 27185 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:25.827838 26250 raft_consensus.cc:2354] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15.
I20260924 14:31:25.835743 27181 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:25.829070 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:25.820679 31107 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:25.850345 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.850617 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1763) took 1916 ms. Trace:
I20260924 14:31:25.850965 26404 rpcz_store.cc:276] 0924 14:31:23.934261 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:23.934501 (+   240us) service_pool.cc:227] Handling call
0924 14:31:23.970357 (+ 35856us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:24.873130 (+902773us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:25.850389 (+977259us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b5400290754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:25.850562 (+   173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2074327424}
I20260924 14:31:25.856258 31594 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260924 14:31:25.856576 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51992.9us
I20260924 14:31:25.857002 31335 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:25.894919 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:25.895993 26250 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:25.897148 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:25.898432 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 98819.5us
I20260924 14:31:25.899715 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1091) took 1522 ms. Trace:
I20260924 14:31:25.900344 26328 rpcz_store.cc:276] 0924 14:31:24.376891 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.377113 (+   222us) service_pool.cc:227] Handling call
0924 14:31:25.853574 (+1476461us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54005bbdd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:25.899631 (+ 46057us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":345864064}
I20260924 14:31:25.901954 26329 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:25.904035 26404 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 10. Current term is 11. Ops: []
W20260924 14:31:25.902312 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 303642us
I20260924 14:31:25.906136 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:25.917587 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:25.919412 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 92333.4us
I20260924 14:31:25.920780 26257 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:25.923820 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.929957 31623 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:25.930590 31623 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:25.931066 31623 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:25.918813 31519 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:25.818710 31556 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:25.956809 31005 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:25.959641 27123 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:25.970037 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:25.977643 27190 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260924 14:31:25.983271 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:25.985047 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:25.983692 27182 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 21.
I20260924 14:31:25.992197 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1774) took 1292 ms. Trace:
I20260924 14:31:26.022135 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:25.996301 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:25.998577 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:25.995852 26403 rpcz_store.cc:276] 0924 14:31:24.700085 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.881636 (+181551us) service_pool.cc:227] Handling call
0924 14:31:24.881994 (+   358us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:24.882421 (+   427us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:24.882462 (+    41us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:24.882957 (+   495us) log.cc:842] Serialized 26 byte log entry
0924 14:31:25.453431 (+570474us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:25.755323 (+301892us) log.cc:842] Serialized 14 byte log entry
0924 14:31:25.991321 (+235998us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:25.991400 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:25.991936 (+   536us) raft_consensus.cc:1671] finished
0924 14:31:25.991965 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:25.992130 (+   165us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10438,"prepare.queue_time_us":41209,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":119,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":21382,"thread_start_us":1096851,"threads_started":3,"wal-append.queue_time_us":710021}
I20260924 14:31:26.114086 26326 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260924 14:31:26.116394 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 94330.1us
I20260924 14:31:26.117908 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:26.120682 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:26.122541 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 124080us
W20260924 14:31:26.122400 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 126158us
W20260924 14:31:26.008796 31524 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 3.063s	user 0.007s	sys 0.047s
I20260924 14:31:25.990083 31549 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:26.019280 31279 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:26.024773 31507 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:26.007798 31606 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:26.164248 31556 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:26.166385 26295 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 2.098s (SENT) [suppressed 27 similar messages]
I20260924 14:31:26.156347 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:26.165059 27192 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:26.122305 31584 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260924 14:31:26.170943 31582 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260924 14:31:26.174031 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 884) took 1938 ms. Trace:
I20260924 14:31:26.174398 27189 rpcz_store.cc:276] 0924 14:31:24.235882 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.237334 (+  1452us) service_pool.cc:227] Handling call
0924 14:31:24.237792 (+   458us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:24.238271 (+   479us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:24.238311 (+    40us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:24.239068 (+   757us) log.cc:842] Serialized 52 byte log entry
0924 14:31:24.330149 (+ 91081us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:24.381491 (+ 51342us) log.cc:842] Serialized 14 byte log entry
0924 14:31:24.390604 (+  9113us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:24.390680 (+    76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:24.916057 (+525377us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b1800395860. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:31:25.989800 (+1073743us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b1800395860. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:31:26.173785 (+183985us) raft_consensus.cc:1671] finished
0924 14:31:26.173809 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:26.173975 (+   166us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7762,"prepare.queue_time_us":51102,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":18793,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":253,"spinlock_wait_cycles":51192960,"thread_start_us":90229,"threads_started":1,"wal-append.queue_time_us":175186}
W20260924 14:31:26.176715 31628 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260924 14:31:26.179176 31507 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:25.993487 31567 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:26.128703 31454 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:31:26.154904 31524 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 3.208s	user 0.007s	sys 0.059s
I20260924 14:31:26.127780 31329 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:31:26.018138 31519 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.136s
I20260924 14:31:26.152858 31547 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:26.163021 31639 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:26.184866 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 287820us
I20260924 14:31:26.185863 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:26.169818 31549 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:26.184394 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:26.188400 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 67796.5us
I20260924 14:31:26.169651 31556 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:26.204530 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:26.179759 31507 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:26.210386 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:26.205542 26402 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:26.211339 31549 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:26.180689 31279 raft_consensus.cc:686] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:26.197118 31518 consensus_meta.cc:287] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.000s
I20260924 14:31:26.183732 31578 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:26.197569 31454 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:26.204725 31329 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 21 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:26.216544 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:26.204476 31580 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.000s
I20260924 14:31:26.214393 31451 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:26.218227 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1679) took 1145 ms. Trace:
I20260924 14:31:26.167136 31606 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:26.209439 31503 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:26.215000 31547 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:26.208715 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:26.236250 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:26.219099 31335 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:26.229959 31580 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:26.222014 31578 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:26.247043 26401 rpcz_store.cc:276] 0924 14:31:25.072536 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.431619 (+359083us) service_pool.cc:227] Handling call
0924 14:31:25.431967 (+   348us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:25.432168 (+   201us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:25.499918 (+ 67750us) log.cc:842] Serialized 14 byte log entry
0924 14:31:25.500462 (+   544us) log.cc:842] Serialized 14 byte log entry
0924 14:31:25.946035 (+445573us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b18001f0a20. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09b8febe10 00007f09b8fdf663 00007f09b8fd502a 00007f09b8fd557b 00007f09b935888a 00007f09b9367f0f 00007f09b9367e88
0924 14:31:26.217790 (+271755us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:26.217848 (+    58us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:26.217922 (+    74us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:26.218012 (+    90us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:26.218162 (+   150us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2140,"prepare.queue_time_us":20577,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":268920,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":22161,"spinlock_wait_cycles":34164864,"thread_start_us":271380,"threads_started":1}
I20260924 14:31:26.231412 31644 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:26.257771 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 49127.8us
I20260924 14:31:26.215302 31368 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.033s
I20260924 14:31:26.210891 31623 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:26.255481 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:26.228150 31543 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:26.293993 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:26.295522 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:26.277140 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:26.298367 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:26.215907 31556 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:26.284490 26402 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:26.272611 26221 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:26.304250 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:26.287263 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:26.258466 31080 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:26.286962 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:26.308516 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:26.259248 31454 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:26.309425 26401 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 8.
I20260924 14:31:26.209720 31639 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:26.221441 31518 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:26.267120 31578 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:26.309159 27188 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
W20260924 14:31:26.319783 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 153420us
W20260924 14:31:26.306694 31601 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:31:26.341324 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1812) took 2373 ms. Trace:
I20260924 14:31:26.349567 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1086) took 1152 ms. Trace:
I20260924 14:31:26.350231 26327 rpcz_store.cc:276] 0924 14:31:25.196784 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.216613 (+ 19829us) service_pool.cc:227] Handling call
0924 14:31:26.349239 (+1132626us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54004100d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:26.349465 (+   226us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":821609728}
I20260924 14:31:26.258576 31279 consensus_queue.cc:237] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:26.353174 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 225997us
I20260924 14:31:26.316720 31555 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:26.313424 31641 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:26.360967 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1811) took 2393 ms. Trace:
I20260924 14:31:26.363066 26256 rpcz_store.cc:276] 0924 14:31:23.967154 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:23.967327 (+   173us) service_pool.cc:227] Handling call
0924 14:31:26.360272 (+2392945us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b58001f4a98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eebce 000000000034aea9 00007f09be21aa30 00007f09be2e07b4 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:31:26.360458 (+   186us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:26.360652 (+   194us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:26.360677 (+    25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:26.360719 (+    42us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:26.360751 (+    32us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:26.360804 (+    53us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:26.360911 (+   107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8113152}
I20260924 14:31:26.325413 31549 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:26.326205 31640 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:26.341961 26251 rpcz_store.cc:276] 0924 14:31:23.967532 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:23.968843 (+  1311us) service_pool.cc:227] Handling call
0924 14:31:23.991031 (+ 22188us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:24.028853 (+ 37822us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:24.028900 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:24.044336 (+ 15436us) log.cc:842] Serialized 26 byte log entry
0924 14:31:24.684510 (+640174us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b18004526e0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:31:25.322269 (+637759us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:25.322326 (+    57us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:26.340828 (+1018502us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b5400195ad4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:26.340899 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:26.340964 (+    65us) raft_consensus.cc:1671] finished
0924 14:31:26.341006 (+    42us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:26.341217 (+   211us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2932,"raft.queue_time_us":793,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3089,"spinlock_wait_cycles":91865216,"thread_start_us":637563,"threads_started":1,"wal-append.queue_time_us":1331620}
I20260924 14:31:26.365563 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:26.371837 31507 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:26.374588 31501 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:26.375388 31580 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:26.375481 31518 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:26.376312 31632 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260924 14:31:26.376881 31521 consensus_meta.cc:287] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.644s	user 0.014s	sys 0.005s
W20260924 14:31:26.377220 31635 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260924 14:31:26.357587 31642 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:26.374629 31503 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.005s
I20260924 14:31:26.375622 31540 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:26.635365 31445 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:26.641144 31445 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:26.380455 31634 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260924 14:31:26.364444 31491 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260924 14:31:26.387039 27191 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 1413) took 2140 ms (client timeout 2098 ms). Trace:
I20260924 14:31:26.364770 31636 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:26.392006 31637 log.cc:924] Time spent T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Append to log took a long time: real 0.179s	user 0.003s	sys 0.000s
I20260924 14:31:26.332942 31454 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
W20260924 14:31:26.403208 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 126859us
I20260924 14:31:26.291815 31644 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:26.447692 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 162867us
I20260924 14:31:26.695463 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:26.520363 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 118372us
I20260924 14:31:26.327991 31329 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:26.403110 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 97667.5us
I20260924 14:31:26.631426 31348 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:26.643936 26253 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.034s	sys 0.030s
W20260924 14:31:26.739200 26253 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.035s	sys 0.030s
W20260924 14:31:26.644672 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 340498us
I20260924 14:31:26.647754 26254 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260924 14:31:26.740751 26254 pending_rounds.cc:76] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 5
I20260924 14:31:26.741055 26254 pending_rounds.cc:97] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260924 14:31:26.741325 26254 raft_consensus.cc:2866] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:26.644270 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1637) took 1064 ms. Trace:
I20260924 14:31:26.402005 31228 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election won for term 16
W20260924 14:31:26.554275 31633 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.373s	user 0.000s	sys 0.001s
W20260924 14:31:26.644176 27191 rpcz_store.cc:269] 0924 14:31:24.246788 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.283283 (+ 36495us) service_pool.cc:227] Handling call
0924 14:31:26.386839 (+2103556us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540000cb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:26.386941 (+   102us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3041387392}
I20260924 14:31:26.742271 26258 rpcz_store.cc:276] 0924 14:31:25.581115 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.581445 (+   330us) service_pool.cc:227] Handling call
0924 14:31:25.611192 (+ 29747us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:25.612252 (+  1060us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:25.612293 (+    41us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:25.628310 (+ 16017us) log.cc:842] Serialized 52 byte log entry
0924 14:31:25.966026 (+337716us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:25.966085 (+    59us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:25.966188 (+   103us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:26.644040 (+677852us) raft_consensus.cc:1671] finished
0924 14:31:26.644060 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:26.644199 (+   139us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":3915,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2962,"thread_start_us":273619,"threads_started":1,"wal-append.queue_time_us":513435}
W20260924 14:31:26.773839 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 78513us
I20260924 14:31:26.356674 31547 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:26.643301 31445 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:26.758042 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 450655us
I20260924 14:31:26.736775 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:26.786307 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 49650us
I20260924 14:31:26.787534 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1772) took 2354 ms. Trace:
I20260924 14:31:26.788956 26399 rpcz_store.cc:276] 0924 14:31:24.433494 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.433658 (+   164us) service_pool.cc:227] Handling call
0924 14:31:24.434089 (+   431us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:24.580926 (+146837us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:26.787276 (+2206350us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b5400123ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:26.787475 (+   199us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":44114176}
I20260924 14:31:26.784636 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:26.789575 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:26.763701 31454 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:26.845891 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52355.4us
W20260924 14:31:26.847282 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 71685.9us
I20260924 14:31:26.771518 31348 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:26.787400 31587 raft_consensus.cc:2741] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260924 14:31:26.769840 31431 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:26.855491 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1678) took 1799 ms. Trace:
I20260924 14:31:26.879889 26406 rpcz_store.cc:276] 0924 14:31:25.056464 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.123850 (+ 67386us) service_pool.cc:227] Handling call
0924 14:31:25.124400 (+   550us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:25.347311 (+222911us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:26.855220 (+1507909us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b540023f854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:26.855411 (+   191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":457956224}
W20260924 14:31:26.861021 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 71531.3us
I20260924 14:31:26.861581 31551 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:26.862957 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1671) took 2228 ms. Trace:
I20260924 14:31:26.882411 26398 rpcz_store.cc:276] 0924 14:31:24.634963 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.635185 (+   222us) service_pool.cc:227] Handling call
0924 14:31:24.635695 (+   510us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:25.033305 (+397610us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:26.862692 (+1829387us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54001e41d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:26.862895 (+   203us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":896733056}
I20260924 14:31:26.883026 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:26.872450 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:26.872675 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 405559us
I20260924 14:31:26.872912 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:26.764256 31080 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260924 14:31:26.871018 27126 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:26.896531 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:26.896673 27126 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:26.860272 31661 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260924 14:31:26.906168 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:26.839633 31518 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:26.913515 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:26.922281 26295 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.012s	sys 0.012s
W20260924 14:31:26.924058 26295 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.012s	sys 0.012s
W20260924 14:31:26.930055 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 57680.6us
I20260924 14:31:26.793149 31636 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.117s
I20260924 14:31:26.895574 31551 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:26.903281 31556 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:26.915325 31080 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:26.938917 26369 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:26.905241 31266 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:26.907071 31587 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:31:26.936650 31516 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.031s	sys 0.036s
I20260924 14:31:26.963042 27186 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 890) took 1501 ms. Trace:
I20260924 14:31:26.909610 31173 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:26.963838 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 889) took 1509 ms. Trace:
I20260924 14:31:26.916378 31503 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260924 14:31:26.917518 31175 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:26.976476 26331 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.007s	sys 0.021s
W20260924 14:31:27.011490 26331 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.007s	sys 0.021s
I20260924 14:31:27.011945 27182 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260924 14:31:26.943636 31518 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:26.933163 31279 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.014s	sys 0.009s
I20260924 14:31:26.983788 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:26.987635 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.024269 27188 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:26.988632 27186 rpcz_store.cc:276] 0924 14:31:25.461435 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.461580 (+   145us) service_pool.cc:227] Handling call
0924 14:31:25.482754 (+ 21174us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:25.483853 (+  1099us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:26.962826 (+1478973us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b540000d550. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:26.962979 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":47875968}
I20260924 14:31:26.906409 31604 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260924 14:31:26.998818 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:27.000599 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:27.002660 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:26.995489 27194 rpcz_store.cc:276] 0924 14:31:25.454927 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.455096 (+   169us) service_pool.cc:227] Handling call
0924 14:31:26.963608 (+1508512us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b540056d954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:26.963755 (+   147us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":26623744}
I20260924 14:31:26.901297 31431 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:27.004333 26406 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260924 14:31:27.005930 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:27.007644 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:27.000268 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:27.028083 26399 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:27.003006 31173 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:26.946530 31491 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:26.997232 31639 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:26.933743 31345 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:26.995394 31580 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:27.012217 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1093) took 1403 ms. Trace:
W20260924 14:31:26.988718 31516 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.033s	sys 0.037s
I20260924 14:31:26.946002 31623 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:27.021633 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:27.003996 31266 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:27.022994 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:27.025369 26398 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
W20260924 14:31:27.023648 31279 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.016s	sys 0.013s
I20260924 14:31:27.030431 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:27.030963 26401 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260924 14:31:26.995077 31422 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:26.990378 31666 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260924 14:31:26.931936 31424 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.013s	sys 0.016s
I20260924 14:31:27.011550 31445 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:27.048013 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:27.048401 26328 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:27.022642 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:27.048555 26331 rpcz_store.cc:276] 0924 14:31:25.609219 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.610773 (+  1554us) service_pool.cc:227] Handling call
0924 14:31:25.611147 (+   374us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:25.611572 (+   425us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:25.611604 (+    32us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:25.612305 (+   701us) log.cc:842] Serialized 52 byte log entry
0924 14:31:25.987655 (+375350us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:26.314829 (+327174us) log.cc:842] Serialized 14 byte log entry
0924 14:31:27.011875 (+697046us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:27.011975 (+   100us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:27.012028 (+    53us) raft_consensus.cc:1671] finished
0924 14:31:27.012039 (+    11us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:27.012169 (+   130us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6982,"prepare.queue_time_us":378309,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":23582,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6180,"spinlock_wait_cycles":40581376,"thread_start_us":1347490,"threads_started":3,"wal-append.queue_time_us":600246}
I20260924 14:31:26.954497 31551 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:27.047675 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.071760 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:27.083427 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:27.084081 26331 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 14
I20260924 14:31:27.025043 31228 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:27.014173 31335 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:27.078162 26399 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:26.998068 31005 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260924 14:31:27.040783 31173 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:27.051636 27188 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:26.995146 31454 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:27.095849 31228 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:27.096525 31652 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
W20260924 14:31:27.097241 31660 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260924 14:31:27.071661 31580 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:27.051744 31453 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260924 14:31:27.091079 31666 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:27.083666 31424 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.014s	sys 0.017s
W20260924 14:31:27.103394 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 206947us
I20260924 14:31:27.136251 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:27.137566 31495 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:27.137671 31666 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:27.032424 31518 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:27.139971 31279 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:27.140405 31279 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:27.141351 31518 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:27.141214 31279 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:27.084172 31422 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:27.142433 31422 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:27.086234 31348 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:27.137774 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:27.106566 31431 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:27.050251 31491 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:27.138070 31587 consensus_queue.cc:1046] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260924 14:31:27.023332 31606 consensus_queue.cc:1046] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260924 14:31:27.174461 26328 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:27.112016 26331 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:27.147645 26327 raft_consensus.cc:1226] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:27.148547 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:27.089198 31445 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:27.155467 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 392840us
I20260924 14:31:27.168706 31666 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:27.177229 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:27.181262 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:27.185811 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:27.128032 31644 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:27.143850 31453 consensus_queue.cc:1225] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260924 14:31:27.194612 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:27.209074 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:27.188887 31578 consensus_meta.cc:287] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.856s	user 0.006s	sys 0.006s
I20260924 14:31:27.214176 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1107) took 1423 ms. Trace:
I20260924 14:31:27.178148 31604 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:27.215818 26327 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260924 14:31:27.225422 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 135955us
I20260924 14:31:27.210026 26400 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:27.176838 31507 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:27.210819 26331 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:27.217240 26324 rpcz_store.cc:276] 0924 14:31:25.791218 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.848039 (+ 56821us) service_pool.cc:227] Handling call
0924 14:31:25.848460 (+   421us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:25.948736 (+100276us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:25.948781 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:25.980626 (+ 31845us) log.cc:842] Serialized 26 byte log entry
0924 14:31:26.263908 (+283282us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:26.263948 (+    40us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:26.264039 (+    91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:27.213910 (+949871us) raft_consensus.cc:1671] finished
0924 14:31:27.213940 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:27.214104 (+   164us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":40712,"raft.queue_time_us":51953,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":23342,"thread_start_us":246031,"threads_started":1,"wal-append.queue_time_us":646705}
I20260924 14:31:27.198194 31666 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:27.233762 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:27.254369 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 207935us
I20260924 14:31:27.255704 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:27.272215 26400 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:27.272320 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:27.224750 31604 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:27.258116 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:27.258155 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:27.217432 31644 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:27.275209 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:27.253844 31348 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:27.281783 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:27.223050 31578 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:27.322260 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.195336 31266 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:27.324610 31665 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:27.325048 27194 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260924 14:31:27.345844 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 64152.7us
I20260924 14:31:27.327703 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:27.357936 26253 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:27.289089 31644 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:27.332310 31672 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260924 14:31:27.257509 31681 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:27.336125 31674 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260924 14:31:27.338265 31445 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:27.338531 26256 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:27.284165 31604 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:27.233515 31507 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:27.279523 31639 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:27.356462 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.325625 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:27.380587 26251 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 7.
I20260924 14:31:27.380543 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.364017 27188 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:27.363227 31644 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:27.364697 31666 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:27.377668 27194 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:27.397475 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:27.367182 31580 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:27.401880 26401 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:27.376219 26253 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:27.400964 31491 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:27.408005 31491 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:31:27.377847 31681 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:27.410115 31495 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:27.389940 31424 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:27.365578 31577 consensus_queue.cc:1046] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260924 14:31:27.410737 31424 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:27.411207 31424 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:27.390586 31639 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:27.415023 26257 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:27.416438 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:27.417317 26251 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:27.420095 26330 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 21 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260924 14:31:27.435372 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:27.434222 31639 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:27.410743 31681 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:27.441216 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:27.454902 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:27.455189 26401 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
W20260924 14:31:27.459205 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 223090us
I20260924 14:31:27.460742 26251 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 7.
W20260924 14:31:27.442247 31501 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.038s	user 0.013s	sys 0.000s
I20260924 14:31:27.419945 31580 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:27.487748 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.480711 31587 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:27.442018 31639 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:27.443224 31507 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:27.509433 31587 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:27.435078 31666 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260924 14:31:27.508275 31664 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:27.516168 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 235274us
I20260924 14:31:27.515892 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.446244 31551 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:27.532436 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.520383 31587 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:27.512518 31684 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:27.545750 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.548707 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.519397 31495 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:27.482117 31536 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260924 14:31:27.550766 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:27.556380 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:27.515877 31228 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:27.530759 31552 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:27.569455 26255 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:27.512399 31580 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:27.571651 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:27.572710 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:27.531162 31695 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:27.516770 31639 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:27.538323 31666 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:27.534646 31664 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:27.574221 31228 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:27.581333 31701 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:27.581456 31578 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:27.565987 31665 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:27.552331 31684 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:27.587499 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:27.596730 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:27.569229 31518 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:27.583909 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:27.994091 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 941) took 1008 ms. Trace:
I20260924 14:31:28.038112 27182 rpcz_store.cc:276] 0924 14:31:26.985654 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:26.985853 (+   199us) service_pool.cc:227] Handling call
0924 14:31:26.986285 (+   432us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:27.012384 (+ 26099us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:27.993764 (+981380us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b54000028d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:27.994028 (+   264us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":58408192}
I20260924 14:31:27.992988 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:27.993450 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:27.579991 31451 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.324s	user 0.009s	sys 0.005s
W20260924 14:31:27.882843 31422 consensus_meta.cc:287] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.740s	user 0.008s	sys 0.002s
I20260924 14:31:28.092530 26253 raft_consensus.cc:2396] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260924 14:31:27.878875 31173 consensus_meta.cc:287] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.767s	user 0.009s	sys 0.000s
I20260924 14:31:27.571354 31507 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:28.090876 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 507021us
W20260924 14:31:27.746554 31690 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
W20260924 14:31:27.651368 31540 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.011s	user 0.008s	sys 0.005s
I20260924 14:31:28.112366 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:27.879092 31580 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:27.877974 31639 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:27.732764 31692 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
W20260924 14:31:28.113847 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 299579us
I20260924 14:31:28.121047 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:28.126718 26251 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.005s	sys 0.005s
W20260924 14:31:28.143919 26251 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.005s	sys 0.006s
I20260924 14:31:27.871335 31642 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:28.129202 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.008s	sys 0.016s
I20260924 14:31:27.869415 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00498335d0b847f299d347155be5c437"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
W20260924 14:31:27.733086 31694 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260924 14:31:27.916167 31703 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:28.118940 31451 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:27.566533 31495 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:28.162681 27189 raft_consensus.cc:595] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
W20260924 14:31:28.163947 27125 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.011s	sys 0.015s
W20260924 14:31:28.234225 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.011s	sys 0.016s
W20260924 14:31:28.146454 31345 consensus_meta.cc:287] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.095s	user 0.012s	sys 0.001s
I20260924 14:31:28.117503 31507 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:27.579697 31552 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:28.278559 26328 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:28.149550 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:28.281304 26331 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:28.034209 31666 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:28.034374 31424 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:28.163645 31521 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:28.163096 31639 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:28.380206 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1519) took 1359 ms. Trace:
I20260924 14:31:27.586288 31695 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:28.194608 31580 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:28.387882 26398 rpcz_store.cc:276] 0924 14:31:27.089236 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.089551 (+   315us) service_pool.cc:227] Handling call
0924 14:31:27.089880 (+   329us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:27.092877 (+  2997us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:27.092908 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:27.108317 (+ 15409us) log.cc:842] Serialized 26 byte log entry
0924 14:31:27.346392 (+238075us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:27.347292 (+   900us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:27.347388 (+    96us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:28.379946 (+1032558us) raft_consensus.cc:1671] finished
0924 14:31:28.379974 (+    28us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:28.380134 (+   160us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":19758,"raft.queue_time_us":17437,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":1747,"thread_start_us":214935,"threads_started":1,"wal-append.queue_time_us":400490}
I20260924 14:31:28.128273 31422 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:28.428215 26372 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.046s	sys 0.109s
W20260924 14:31:28.428658 26372 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.047s	sys 0.109s
I20260924 14:31:28.356356 31345 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:28.312431 31711 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:28.448448 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.006s	sys 0.014s
I20260924 14:31:28.130718 31173 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:28.460132 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.006s	sys 0.014s
I20260924 14:31:28.379523 31551 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:28.213469 31642 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:28.376232 31540 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260924 14:31:28.454146 31700 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:28.489917 31700 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:28.492332 31700 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:28.451785 31580 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:28.495577 31580 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:28.389680 31521 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:28.627120 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.044s	user 0.013s	sys 0.016s
W20260924 14:31:28.744412 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.013s	sys 0.017s
W20260924 14:31:28.745012 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 1.16245e+06us
W20260924 14:31:28.628120 26222 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.005s	sys 0.006s
W20260924 14:31:28.747751 26222 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.006s	sys 0.006s
W20260924 14:31:28.148818 26250 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.000s	sys 0.024s
W20260924 14:31:28.628743 26223 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.006s	sys 0.013s
W20260924 14:31:28.748879 26250 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.223s	user 0.000s	sys 0.024s
I20260924 14:31:28.144066 26253 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
W20260924 14:31:28.751494 26253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1268) took 1229 ms (client timeout 1601 ms). Trace:
W20260924 14:31:28.752007 26253 rpcz_store.cc:269] 0924 14:31:27.522241 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:28.120731 (+598490us) service_pool.cc:227] Handling call
0924 14:31:28.751150 (+630419us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:28.744452 31424 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:28.754679 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:28.220821 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1259) took 2471 ms. Trace:
W20260924 14:31:28.653836 26251 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.000s	sys 0.012s
W20260924 14:31:28.755774 26251 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.001s	sys 0.012s
I20260924 14:31:28.755744 26254 rpcz_store.cc:276] 0924 14:31:25.749842 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.851225 (+101383us) service_pool.cc:227] Handling call
0924 14:31:25.894885 (+ 43660us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:26.770729 (+875844us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:28.220480 (+1449751us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b540070fb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:28.220737 (+   257us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1708438912}
I20260924 14:31:28.757707 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1262) took 1296 ms. Trace:
I20260924 14:31:28.663386 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1518) took 1649 ms. Trace:
I20260924 14:31:28.758610 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:28.641943 31730 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:28.758785 26404 rpcz_store.cc:276] 0924 14:31:27.013461 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.013619 (+   158us) service_pool.cc:227] Handling call
0924 14:31:27.041294 (+ 27675us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:27.050495 (+  9201us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:27.291699 (+241204us) log.cc:842] Serialized 14 byte log entry
0924 14:31:27.540091 (+248392us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:27.566873 (+ 26782us) log.cc:842] Serialized 26 byte log entry
0924 14:31:27.567009 (+   136us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:27.567040 (+    31us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:27.567123 (+    83us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:28.662506 (+1095383us) raft_consensus.cc:1671] finished
0924 14:31:28.662584 (+    78us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:28.663044 (+   460us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":257723,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":21880,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":44005,"thread_start_us":386029,"threads_started":2,"wal-append.queue_time_us":1152941}
I20260924 14:31:28.665704 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1103) took 1201 ms. Trace:
I20260924 14:31:28.690407 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:28.763834 27182 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:28.693384 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:28.426573 31111 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.025s	sys 0.032s
W20260924 14:31:28.766047 31111 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.027s	sys 0.034s
I20260924 14:31:28.691532 27186 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "70ed8dcd463347619717298c81a42998"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:28.741362 31495 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:28.741778 31422 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:28.767165 27186 raft_consensus.cc:595] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:28.767745 27186 raft_consensus.cc:603] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:28.768071 31422 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:28.688264 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:28.748876 26223 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.006s	sys 0.013s
I20260924 14:31:28.760073 31730 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:28.758195 26251 rpcz_store.cc:276] 0924 14:31:27.462699 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.463675 (+   976us) service_pool.cc:227] Handling call
0924 14:31:27.464044 (+   369us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:27.464245 (+   201us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:28.144381 (+680136us) log.cc:842] Serialized 14 byte log entry
0924 14:31:28.756932 (+612551us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:28.757051 (+   119us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:28.757179 (+   128us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:28.757300 (+   121us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:28.757581 (+   281us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6970,"prepare.queue_time_us":70200,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":43,"thread_start_us":1241587,"threads_started":2,"wal-append.queue_time_us":506607}
I20260924 14:31:28.772758 31730 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:28.772842 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:28.773885 31730 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:28.769047 31422 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:28.761905 26324 rpcz_store.cc:276] 0924 14:31:27.463980 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.464112 (+   132us) service_pool.cc:227] Handling call
0924 14:31:27.464407 (+   295us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:27.464669 (+   262us) raft_consensus.cc:1520] Early marking committed up to index 6
0924 14:31:27.464729 (+    60us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:27.465144 (+   415us) log.cc:842] Serialized 26 byte log entry
0924 14:31:27.613249 (+148105us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:27.613299 (+    50us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:27.613403 (+   104us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:28.665145 (+1051742us) raft_consensus.cc:1671] finished
0924 14:31:28.665182 (+    37us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:28.665479 (+   297us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":147761,"threads_started":1,"wal-append.queue_time_us":905568}
I20260924 14:31:28.776902 31422 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:28.741863 31507 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:28.712661 31173 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:28.781914 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1274) took 1201 ms. Trace:
I20260924 14:31:28.782565 26254 rpcz_store.cc:276] 0924 14:31:27.580954 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:28.758094 (+1177140us) service_pool.cc:227] Handling call
0924 14:31:28.781773 (+ 23679us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:28.773890 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1271) took 1242 ms. Trace:
W20260924 14:31:28.783525 26253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1273) took 1213 ms (client timeout 1563 ms). Trace:
I20260924 14:31:28.774811 31730 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:28.788211 26251 rpcz_store.cc:276] 0924 14:31:27.532022 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:28.772439 (+1240417us) service_pool.cc:227] Handling call
0924 14:31:28.773805 (+  1366us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:28.788427 26253 rpcz_store.cc:269] 0924 14:31:27.569807 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:28.753882 (+1184075us) service_pool.cc:227] Handling call
0924 14:31:28.783447 (+ 29565us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:28.800572 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:28.801832 26253 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:28.802342 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1269) took 1279 ms. Trace:
I20260924 14:31:28.804246 26253 rpcz_store.cc:276] 0924 14:31:27.522538 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:28.800118 (+1277580us) service_pool.cc:227] Handling call
0924 14:31:28.802271 (+  2153us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:28.788818 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:28.787379 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:28.810570 31228 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:28.814043 31335 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:28.817821 31111 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:28.827996 31733 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:28.502550 31551 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:28.528515 31666 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:28.379194 31665 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:28.504823 31684 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:28.502090 31695 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:28.235430 31703 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:28.995918 31726 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:28.996573 31703 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:28.532027 31707 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260924 14:31:28.474336 31711 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:28.996827 31726 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:28.999787 31711 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:28.483866 31710 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260924 14:31:29.002185 31716 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:28.496259 31713 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:29.004343 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1649) took 1608 ms. Trace:
I20260924 14:31:29.004490 31713 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.004946 26256 rpcz_store.cc:276] 0924 14:31:27.395668 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.396232 (+   564us) service_pool.cc:227] Handling call
0924 14:31:27.418495 (+ 22263us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:27.418760 (+   265us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:27.418794 (+    34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:27.436702 (+ 17908us) log.cc:842] Serialized 26 byte log entry
0924 14:31:27.710410 (+273708us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:27.710469 (+    59us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:27.710579 (+   110us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:29.003718 (+1293139us) raft_consensus.cc:1671] finished
0924 14:31:29.003808 (+    90us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:29.004078 (+   270us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":201829,"threads_started":1,"wal-append.queue_time_us":664958}
I20260924 14:31:29.007189 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:28.512478 31718 raft_consensus.cc:3032] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:29.008349 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1277) took 1426 ms. Trace:
I20260924 14:31:29.008773 31552 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.008847 26256 rpcz_store.cc:276] 0924 14:31:27.581862 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.006771 (+1424909us) service_pool.cc:227] Handling call
0924 14:31:29.008287 (+  1516us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:29.010722 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.011679 26256 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:29.012203 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1264) took 1497 ms. Trace:
I20260924 14:31:29.012584 26256 rpcz_store.cc:276] 0924 14:31:27.515375 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.010237 (+1494862us) service_pool.cc:227] Handling call
0924 14:31:29.012154 (+  1917us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:29.013475 31671 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:29.014463 31671 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:28.966663 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 809235us
I20260924 14:31:28.979843 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1098) took 1973 ms. Trace:
I20260924 14:31:28.980674 31335 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:28.985733 31642 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:28.986599 31111 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.013590 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:28.806013 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:28.915573 31730 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:28.916141 31424 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.096825 26254 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:29.028855 31726 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:29.100185 26254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1278) took 1516 ms (client timeout 1734 ms). Trace:
W20260924 14:31:29.101240 26254 rpcz_store.cc:269] 0924 14:31:27.582161 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:28.788285 (+1206124us) service_pool.cc:227] Handling call
0924 14:31:29.098064 (+309779us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:29.101886 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:29.105093 26371 leader_election.cc:334] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.551s (SENT)
I20260924 14:31:29.106005 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:29.107719 31642 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260924 14:31:29.110198 31335 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:29.111899 31623 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:29.111819 31424 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.113027 31623 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.114677 31424 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.115005 26253 raft_consensus.cc:2362] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12. Re-sending same reply.
I20260924 14:31:29.113518 26256 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:29.115612 26256 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:29.115573 31733 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.123942 31734 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:29.102644 31725 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:29.125000 31734 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.125474 31725 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.127877 31735 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:29.128417 31551 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.128793 31735 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.017287 26326 rpcz_store.cc:276] 0924 14:31:27.006448 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.085099 (+ 78651us) service_pool.cc:227] Handling call
0924 14:31:27.100870 (+ 15771us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:27.110598 (+  9728us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:27.110627 (+    29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:27.143685 (+ 33058us) log.cc:842] Serialized 52 byte log entry
0924 14:31:27.432122 (+288437us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:27.583513 (+151391us) log.cc:842] Serialized 14 byte log entry
0924 14:31:28.028054 (+444541us) log.cc:842] Serialized 14 byte log entry
0924 14:31:28.967392 (+939338us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:28.979265 (+ 11873us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:28.979337 (+    72us) raft_consensus.cc:1671] finished
0924 14:31:28.979365 (+    28us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:28.979676 (+   311us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1568142,"prepare.queue_time_us":1082417,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":97,"raft.queue_time_us":1254450,"raft.run_cpu_time_us":557,"raft.run_wall_time_us":17415,"thread_start_us":622798,"threads_started":3,"wal-append.queue_time_us":401319}
I20260924 14:31:29.008597 31718 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:29.140615 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.143167 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.118494 31733 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:29.103866 31713 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.146453 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1104) took 1676 ms. Trace:
W20260924 14:31:29.148746 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 176435us
W20260924 14:31:29.150895 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 134519us
I20260924 14:31:29.136098 31623 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:29.153177 26331 consensus_meta.cc:287] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.870s	user 0.012s	sys 0.016s
W20260924 14:31:29.151937 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 59760.5us
I20260924 14:31:29.156565 31736 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:29.156911 26326 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bcac7743e33046019152a570ae473ca3"
dest_uuid: "f59f3234edf84aab852fa989fb754d31"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:32930
I20260924 14:31:29.157440 31551 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.157696 26326 raft_consensus.cc:595] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:29.158351 26326 raft_consensus.cc:603] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:29.157512 31736 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.161407 31728 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:29.161994 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1105) took 1692 ms. Trace:
I20260924 14:31:29.162305 31728 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.160437 31695 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.162499 26326 rpcz_store.cc:276] 0924 14:31:27.470113 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.160650 (+1690537us) service_pool.cc:227] Handling call
0924 14:31:29.161173 (+   523us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:29.161591 (+   418us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:29.161616 (+    25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:29.161660 (+    44us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:29.161744 (+    84us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.161815 (+    71us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:29.161920 (+   105us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:29.163698 31695 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.162714 31736 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.169004 31695 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:29.169163 31111 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:29.153388 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 330233us
I20260924 14:31:29.152060 31666 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.171082 31417 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:29.171977 31666 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.173045 31417 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:29.173406 31417 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:29.174286 31666 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:29.175177 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 388047us
I20260924 14:31:29.179073 26254 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:29.180019 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1263) took 1717 ms. Trace:
I20260924 14:31:29.180390 26254 rpcz_store.cc:276] 0924 14:31:27.514928 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.133967 (+1619039us) service_pool.cc:227] Handling call
0924 14:31:29.178747 (+ 44780us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:29.179721 (+   974us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.179940 (+   219us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:29.184033 26326 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260924 14:31:29.186017 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1106) took 1713 ms. Trace:
I20260924 14:31:29.187662 26326 rpcz_store.cc:276] 0924 14:31:27.472198 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.163984 (+1691786us) service_pool.cc:227] Handling call
0924 14:31:29.164403 (+   419us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:29.185636 (+ 21233us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.185921 (+   285us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:29.147842 26324 rpcz_store.cc:276] 0924 14:31:27.467714 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:28.776552 (+1308838us) service_pool.cc:227] Handling call
0924 14:31:28.777324 (+   772us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:28.819321 (+ 41997us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:28.822851 (+  3530us) log.cc:842] Serialized 14 byte log entry
0924 14:31:29.099132 (+276281us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:29.114540 (+ 15408us) log.cc:842] Serialized 52 byte log entry
0924 14:31:29.114683 (+   143us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:29.114715 (+    32us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.114804 (+    89us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:29.143826 (+ 29022us) raft_consensus.cc:1671] finished
0924 14:31:29.143852 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:29.143996 (+   144us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":145455,"prepare.queue_time_us":147826,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":8412,"raft.queue_time_us":222749,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":37060,"thread_start_us":205162,"threads_started":1,"wal-append.queue_time_us":317347}
I20260924 14:31:29.160445 26256 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.189051 26331 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:29.159217 31734 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:29.189577 26369 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.026s	sys 0.028s
I20260924 14:31:29.165218 31671 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.175087 31417 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.208352 31518 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.208671 31417 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:29.209571 26324 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:29.209287 31518 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:29.210299 31638 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 2.565s	user 0.004s	sys 0.030s
I20260924 14:31:29.201778 31509 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:29.202013 26369 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.082s	user 0.026s	sys 0.029s
I20260924 14:31:29.216375 31732 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:29.215119 31728 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.215991 31509 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:29.200321 26371 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.563s (SENT)
I20260924 14:31:29.196874 31735 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.221669 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:29.223006 31587 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.659s	user 0.008s	sys 0.013s
I20260924 14:31:29.195358 31725 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.224048 31509 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.223661 31718 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.225481 31718 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:29.217108 31732 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.231417 26256 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:29.232033 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:29.232381 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:29.232811 26406 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:29.233549 31518 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.234239 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:29.198498 31664 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:29.234752 26403 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260924 14:31:29.235302 26331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1107) took 1567 ms (client timeout 1937 ms). Trace:
I20260924 14:31:29.234722 26400 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:29.235316 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:29.236088 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 65319.6us
I20260924 14:31:29.236172 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1646) took 3557 ms. Trace:
I20260924 14:31:29.236752 26405 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:29.237077 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:31:29.202405 31266 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.211s	user 0.039s	sys 0.044s
W20260924 14:31:29.238130 31266 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.347s	user 0.039s	sys 0.045s
I20260924 14:31:29.238459 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:29.239135 31266 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:29.239569 26397 raft_consensus.cc:2379] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 9: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:29.223673 31587 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.241916 31266 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:29.220808 31666 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:29.243281 31266 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.243885 31587 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.236693 26249 rpcz_store.cc:276] 0924 14:31:25.678905 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.698199 (+ 19294us) service_pool.cc:227] Handling call
0924 14:31:25.698579 (+   380us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:25.718110 (+ 19531us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:25.718155 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:25.718712 (+   557us) log.cc:842] Serialized 26 byte log entry
0924 14:31:26.156307 (+437595us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:26.156350 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:26.156421 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:29.209429 (+3053008us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:29.235691 (+ 26262us) raft_consensus.cc:1671] finished
0924 14:31:29.235710 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:29.235978 (+   268us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":19970,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":7497,"spinlock_wait_cycles":16974080,"thread_start_us":415027,"threads_started":1,"wal-append.queue_time_us":626083}
I20260924 14:31:29.246708 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1108) took 1591 ms. Trace:
I20260924 14:31:29.248133 31569 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:29.249155 31569 consensus_queue.cc:1225] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260924 14:31:29.250275 31654 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.250691 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1655) took 1800 ms. Trace:
I20260924 14:31:29.250357 31671 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:29.251219 31654 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.251695 31518 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:29.252441 31518 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.253131 31518 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.253713 31654 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.254650 31636 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:29.251010 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 67647.7us
I20260924 14:31:29.255326 31636 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:29.256136 26371 leader_election.cc:334] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.590s (SENT)
I20260924 14:31:29.247187 26326 rpcz_store.cc:276] 0924 14:31:27.655328 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.217866 (+1562538us) service_pool.cc:227] Handling call
0924 14:31:29.245964 (+ 28098us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:29.246314 (+   350us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:29.246340 (+    26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:29.246406 (+    66us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:29.246442 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.246505 (+    63us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:29.246628 (+   123us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:29.257575 31636 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:29.258244 31644 consensus_meta.cc:287] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.852s	user 0.000s	sys 0.020s
I20260924 14:31:29.258692 31644 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.261013 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:29.261463 26221 connection.cc:232] RPC call timeout handler was delayed by -1.6934838679999302s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
I20260924 14:31:29.230783 31732 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:29.263373 31587 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:29.263758 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:29.264074 31645 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:29.264993 31734 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:29.265552 31734 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:29.265959 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:29.266032 31734 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:29.217618 31716 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:29.268322 26254 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260924 14:31:29.267220 26253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1276) took 1685 ms (client timeout 1806 ms). Trace:
W20260924 14:31:29.278059 26253 rpcz_store.cc:269] 0924 14:31:27.581574 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:28.976949 (+1395375us) service_pool.cc:227] Handling call
0924 14:31:29.267003 (+290054us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5400382cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:29.267149 (+   146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":23704960}
W20260924 14:31:29.235780 26331 rpcz_store.cc:269] 0924 14:31:27.650630 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:28.149195 (+498565us) service_pool.cc:227] Handling call
0924 14:31:29.202363 (+1053168us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:29.242926 31666 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:29.257241 26371 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.680s (SENT)
I20260924 14:31:29.251644 31671 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:29.272286 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1071) took 4891 ms. Trace:
W20260924 14:31:29.273280 26332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1619) took 4101 ms (client timeout 1590 ms). Trace:
I20260924 14:31:29.274140 31730 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:29.274515 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1078) took 4871 ms. Trace:
I20260924 14:31:29.264423 31587 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.255394 31518 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:29.287601 26332 rpcz_store.cc:269] 0924 14:31:25.171837 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.171986 (+   149us) service_pool.cc:227] Handling call
0924 14:31:29.272849 (+4100863us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54007cecd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:29.273072 (+   223us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1146554112}
I20260924 14:31:29.287266 26329 rpcz_store.cc:276] 0924 14:31:24.380780 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.614512 (+233732us) service_pool.cc:227] Handling call
0924 14:31:25.899139 (+1284627us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:25.902364 (+  3225us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.271877 (+3369513us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54005bbdd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:29.272099 (+   222us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2953298176}
I20260924 14:31:29.295087 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:29.299616 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:29.300364 31587 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.301041 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:29.301378 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44202.2us
I20260924 14:31:29.301921 27190 raft_consensus.cc:2379] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:29.299963 31671 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.298064 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:29.296198 31623 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260924 14:31:29.306702 31623 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:29.307139 31558 raft_consensus.cc:3032] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:29.307639 31558 raft_consensus.cc:729] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:29.308753 31558 consensus_queue.cc:260] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.310563 31730 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.313146 31730 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.313596 31552 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:29.314303 31552 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.315276 31666 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:29.316596 31552 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:29.316886 31604 consensus_meta.cc:287] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.927s	user 0.008s	sys 0.013s
I20260924 14:31:29.290136 26325 rpcz_store.cc:276] 0924 14:31:24.403515 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:24.631780 (+228265us) service_pool.cc:227] Handling call
0924 14:31:24.632117 (+   337us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:24.707207 (+ 75090us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:24.707524 (+   317us) log.cc:842] Serialized 14 byte log entry
0924 14:31:25.381067 (+673543us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:25.381584 (+   517us) log.cc:842] Serialized 26 byte log entry
0924 14:31:25.381738 (+   154us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:25.381782 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.273894 (+3892112us) spinlock_profiling.cc:243] Waited 74.7 ms on lock 0x7b5400497d54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:29.273958 (+    64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:29.274007 (+    49us) raft_consensus.cc:1671] finished
0924 14:31:29.274149 (+   142us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:29.274321 (+   172us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":29912,"prepare.queue_time_us":3326,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":28141,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":11140,"spinlock_wait_cycles":171698944,"thread_start_us":639873,"threads_started":1,"wal-append.queue_time_us":787128,"wal-append.run_cpu_time_us":2255,"wal-append.run_wall_time_us":1033733}
I20260924 14:31:29.317626 31175 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:29.319475 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:29.320091 27185 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 931) took 3929 ms (client timeout 2151 ms). Trace:
I20260924 14:31:29.320425 31665 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:29.251231 26250 rpcz_store.cc:276] 0924 14:31:27.450267 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.455735 (+  5468us) service_pool.cc:227] Handling call
0924 14:31:27.480056 (+ 24321us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:27.480331 (+   275us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:31:28.749934 (+1269603us) log.cc:842] Serialized 14 byte log entry
0924 14:31:29.192774 (+442840us) log.cc:842] Serialized 14 byte log entry
0924 14:31:29.199981 (+  7207us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:29.218520 (+ 18539us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:29.250374 (+ 31854us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.250479 (+   105us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:29.250630 (+   151us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":39959,"prepare.queue_time_us":98125,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":33500,"raft.queue_time_us":74276,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":52318,"thread_start_us":1494344,"threads_started":3,"wal-append.queue_time_us":352735}
I20260924 14:31:29.304728 31716 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260924 14:31:29.327375 27185 rpcz_store.cc:269] 0924 14:31:25.390371 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.390636 (+   265us) service_pool.cc:227] Handling call
0924 14:31:29.319676 (+3929040us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54006c5054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:29.319898 (+   222us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":865803648}
I20260924 14:31:29.345049 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:29.307175 31623 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:29.408300 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:29.409296 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:29.338132 31733 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260924 14:31:29.362957 26256 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260924 14:31:29.418463 26256 pending_rounds.cc:76] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 9
I20260924 14:31:29.345577 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:29.416983 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:29.417477 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:29.348013 31665 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.402276 26403 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:29.422008 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:29.348124 31604 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.422875 26397 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:29.424593 31604 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:29.418922 26256 pending_rounds.cc:97] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260924 14:31:29.425328 26256 raft_consensus.cc:2866] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:29.426923 31644 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:29.428196 27182 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:31:29.428316 31556 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260924 14:31:29.396932 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:29.426756 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1266) took 1904 ms. Trace:
I20260924 14:31:29.430621 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:29.430656 26256 rpcz_store.cc:276] 0924 14:31:27.521598 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.112751 (+1591153us) service_pool.cc:227] Handling call
0924 14:31:29.113281 (+   530us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:29.425961 (+312680us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.426173 (+   212us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:29.351564 31518 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:29.431944 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.348016 31666 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:29.433018 31666 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.433430 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:29.434055 26256 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 9
I20260924 14:31:29.348297 31175 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.436553 31665 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:29.439927 26258 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [15.9-17.10]
I20260924 14:31:29.440367 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:29.441375 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 175567us
I20260924 14:31:29.442318 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:29.443317 31716 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:29.413596 26405 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:29.351179 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:29.414789 31623 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:29.447409 26371 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.765s (SENT)
I20260924 14:31:29.349560 31734 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.448058 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:29.449429 26404 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
W20260924 14:31:29.350538 31659 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 2.382s	user 0.005s	sys 0.019s
W20260924 14:31:29.452606 31738 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260924 14:31:29.453765 31039 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.455380 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:29.449014 26401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1520) took 2299 ms (client timeout 2008 ms). Trace:
I20260924 14:31:29.458817 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:29.461251 26401 rpcz_store.cc:269] 0924 14:31:27.145162 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.149600 (+  4438us) service_pool.cc:227] Handling call
0924 14:31:29.443928 (+2294328us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54004f0e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:29.444107 (+   179us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":485360384}
I20260924 14:31:29.458027 31175 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.476233 31552 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:29.477396 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 56164us
I20260924 14:31:29.476375 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.478587 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.479594 26326 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:29.479682 31721 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:29.480713 31552 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.483294 31552 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.482360 31174 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.478503 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.493273 31174 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.495749 31174 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:29.474859 31665 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:29.499253 31039 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.484606 31623 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:29.504498 27126 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:29.505959 31704 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.629s	user 0.017s	sys 0.006s
I20260924 14:31:29.501780 31039 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.512331 31266 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:29.512890 31266 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:29.513309 31266 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:29.513505 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.514554 26250 raft_consensus.cc:2396] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260924 14:31:29.516695 31541 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.517338 31541 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:29.524958 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 66085.3us
I20260924 14:31:29.522285 26251 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:29.520951 31039 raft_consensus.cc:979] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:31:29.527980 26250 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:29.510141 31666 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.535511 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:29.541528 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 190589us
I20260924 14:31:29.546082 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:29.548349 31577 raft_consensus.cc:1067] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:31:29.536561 31508 consensus_queue.cc:1046] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260924 14:31:29.512346 31597 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
W20260924 14:31:29.556102 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 111728us
I20260924 14:31:29.557374 31531 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.562678 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.557981 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1856) took 2213 ms. Trace:
I20260924 14:31:29.563661 26252 rpcz_store.cc:276] 0924 14:31:27.343988 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.344113 (+   125us) service_pool.cc:227] Handling call
0924 14:31:27.344388 (+   275us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:27.415843 (+ 71455us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:27.415889 (+    46us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:27.415947 (+    58us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:27.415990 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.552035 (+2136045us) spinlock_profiling.cc:243] Waited 71.2 ms on lock 0x7b54007828d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:29.555147 (+  3112us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:29.557793 (+  2646us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":163729792}
I20260924 14:31:29.553378 31561 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.569367 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.571626 31178 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:29.568322 31566 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:29.572664 27123 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:29.572942 31566 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:29.573597 31566 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:29.574363 31654 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:29.568653 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:29.568918 26371 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.791s (SENT)
I20260924 14:31:29.574852 31597 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260924 14:31:29.577342 31579 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260924 14:31:29.579608 31717 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.070s	user 0.007s	sys 0.018s
I20260924 14:31:29.570459 31175 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:29.592532 31175 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.563416 31174 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260924 14:31:29.587347 31541 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:29.596155 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:29.587881 31561 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.577351 31665 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:29.589920 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.558094 31531 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.595232 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:29.587219 31654 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:29.604183 31579 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260924 14:31:29.593564 31734 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:29.630522 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 77640.9us
I20260924 14:31:29.631842 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.617151 31531 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.601684 27126 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:29.637267 31561 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:29.623557 26298 connection.cc:232] RPC call timeout handler was delayed by -1.7344290390001333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
W20260924 14:31:29.654762 26298 connection.cc:232] RPC call timeout handler was delayed by -1.4338290390005568s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
I20260924 14:31:29.643548 31642 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:31:29.644057 31178 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:29.644502 27126 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:29.645402 31175 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:29.646155 26256 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:31:29.646727 31509 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:29.659729 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.607414 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.648958 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1854) took 2350 ms. Trace:
I20260924 14:31:29.660367 26252 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
W20260924 14:31:29.650527 31667 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 2.323s	user 0.022s	sys 0.018s
I20260924 14:31:29.652793 31501 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:29.653365 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1628) took 2643 ms. Trace:
I20260924 14:31:29.601212 31721 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:29.660651 26255 rpcz_store.cc:276] 0924 14:31:27.298579 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.322869 (+ 24290us) service_pool.cc:227] Handling call
0924 14:31:27.323238 (+   369us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:27.574812 (+251574us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.648505 (+2073693us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54000c0dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:29.648791 (+   286us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":561632512}
I20260924 14:31:29.641578 31541 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:29.693321 26250 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:29.693557 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.694229 26324 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:29.694332 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.658640 31178 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:29.695026 26250 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
W20260924 14:31:29.689687 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 45272.7us
I20260924 14:31:29.684973 26327 rpcz_store.cc:276] 0924 14:31:27.009450 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.178452 (+169002us) service_pool.cc:227] Handling call
0924 14:31:27.194172 (+ 15720us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:27.224965 (+ 30793us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.652923 (+2427958us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b540012eb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:29.653207 (+   284us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":48767744}
I20260924 14:31:29.695230 31178 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:29.692572 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.704063 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.704269 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:29.704581 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.705981 26253 raft_consensus.cc:2379] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 9: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:29.684043 31501 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:29.615583 31665 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.712057 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.711293 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.713361 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.714097 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.714202 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:29.714919 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.715948 31370 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:29.696599 31726 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:29.718844 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.720033 31503 raft_consensus.cc:3032] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:31:29.722950 26250 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:29.726101 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.727273 31726 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:29.730446 26399 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00498335d0b847f299d347155be5c437"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:35444
I20260924 14:31:29.731076 31726 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:29.731249 26399 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:31:29.732563 26399 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:29.732477 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.732519 26325 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260924 14:31:29.734112 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.734390 31501 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.734897 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.735360 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.735893 26256 raft_consensus.cc:2396] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260924 14:31:29.736048 31749 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260924 14:31:29.727530 31503 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:29.740187 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:29.735576 31501 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:29.744746 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.746155 27123 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:29.756209 26249 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:29.757862 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.763705 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:29.749945 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:29.762181 26253 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:29.766657 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.774335 26398 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260924 14:31:29.771260 26249 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:29.767385 26250 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
W20260924 14:31:29.776715 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 210226us
I20260924 14:31:29.774602 26252 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:29.779829 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.781047 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.790759 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.766925 31658 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:29.792461 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:29.775322 31551 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:29.795483 31454 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.389s	user 0.044s	sys 0.091s
I20260924 14:31:29.796633 31654 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:29.802285 31454 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.470s	user 0.045s	sys 0.091s
I20260924 14:31:29.804332 31568 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:29.804816 31568 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:29.804759 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.805357 31568 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:29.805688 26324 raft_consensus.cc:2379] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:29.806989 31754 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260924 14:31:29.812913 31665 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:29.798380 26371 leader_election.cc:334] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.806s (SENT)
I20260924 14:31:29.800706 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:29.744540 31541 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.801086 27125 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:29.814880 26255 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:29.818244 31551 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:29.821496 31266 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.823765 27191 raft_consensus.cc:1261] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:29.817947 26332 raft_consensus.cc:2396] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260924 14:31:29.827090 26252 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:29.831753 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.834259 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.835012 26327 raft_consensus.cc:2396] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260924 14:31:29.849074 26400 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:29.836738 31623 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.851385 31623 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.853384 31623 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:29.856338 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1100) took 2849 ms. Trace:
I20260924 14:31:29.856796 26323 rpcz_store.cc:276] 0924 14:31:27.006935 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.087802 (+ 80867us) service_pool.cc:227] Handling call
0924 14:31:27.088165 (+   363us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:27.088436 (+   271us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:27.088472 (+    36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:29.662147 (+2573675us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b2c001e1fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:29.662743 (+   596us) log.cc:842] Serialized 26 byte log entry
0924 14:31:29.789776 (+127033us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:29.839844 (+ 50068us) log.cc:842] Serialized 14 byte log entry
0924 14:31:29.840097 (+   253us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.845870 (+  5773us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:29.845925 (+    55us) raft_consensus.cc:1671] finished
0924 14:31:29.855935 (+ 10010us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:29.856141 (+   206us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":13461,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":206,"spinlock_wait_cycles":12893440,"thread_start_us":88565,"threads_started":1,"wal-append.queue_time_us":82715}
I20260924 14:31:29.860239 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1625) took 4406 ms. Trace:
W20260924 14:31:29.863907 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 70403.3us
I20260924 14:31:29.843991 31669 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:29.860102 31329 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:29.916231 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 60909.7us
I20260924 14:31:29.920483 26326 raft_consensus.cc:2396] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260924 14:31:29.924566 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 58088.7us
I20260924 14:31:29.925151 26399 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.917429 26330 rpcz_store.cc:276] 0924 14:31:25.453392 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.459568 (+  6176us) service_pool.cc:227] Handling call
0924 14:31:26.351659 (+892091us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:27.420515 (+1068856us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.456435 (+2035920us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54004100d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:29.859751 (+403316us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54004100d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:29.860012 (+   261us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4506775168}
I20260924 14:31:29.927594 27125 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:29.936483 26406 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:29.938725 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:29.931883 31677 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:31:29.943866 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.940790 31562 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:29.921257 31579 consensus_queue.cc:1046] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260924 14:31:29.978756 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.957306 26399 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.979555 26331 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:29.955700 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:29.955602 31562 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:29.984926 31562 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:29.986079 31562 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:29.986207 31765 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260924 14:31:29.988751 31693 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 2.450s	user 0.005s	sys 0.025s
I20260924 14:31:29.915565 31566 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.961428 31431 raft_consensus.cc:3032] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Stepping down as leader of term 7
I20260924 14:31:29.962193 31178 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.963315 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:29.943117 31597 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:29.971567 26405 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:29.972520 31266 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:29.976501 26251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49594 (request call id 1275) took 2395 ms (client timeout 1791 ms). Trace:
I20260924 14:31:29.921312 31669 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:29.977417 31726 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:29.978027 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 908) took 2624 ms. Trace:
I20260924 14:31:29.990196 26249 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:29.960289 31686 consensus_queue.cc:1046] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:29.998754 26327 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260924 14:31:30.022550 26251 rpcz_store.cc:269] 0924 14:31:27.581286 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:28.803404 (+1222118us) service_pool.cc:227] Handling call
0924 14:31:29.976230 (+1172826us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b540015e0d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:29.976383 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":578725632}
W20260924 14:31:30.031311 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 109774us
I20260924 14:31:30.027035 27188 rpcz_store.cc:276] 0924 14:31:27.353551 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.353999 (+   448us) service_pool.cc:227] Handling call
0924 14:31:27.354357 (+   358us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:27.381889 (+ 27532us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.977421 (+2595532us) spinlock_profiling.cc:243] Waited 9.42 ms on lock 0x7b540000f854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:29.977800 (+   379us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":21656832}
I20260924 14:31:29.949895 31329 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.054488 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.041553 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:30.018359 31541 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:29.998690 31431 raft_consensus.cc:729] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:30.059193 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 122455us
I20260924 14:31:30.025003 31677 raft_consensus.cc:686] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:30.064276 26251 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260924 14:31:30.064707 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:30.061559 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 84447.5us
I20260924 14:31:30.066794 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:30.067427 26327 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:30.027566 31658 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260924 14:31:30.067022 31039 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:30.072533 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.087528 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:30.092118 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:30.092869 26255 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:30.093793 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:30.094367 26255 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:30.094417 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 943) took 2610 ms. Trace:
I20260924 14:31:30.102612 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.105463 26251 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260924 14:31:30.105971 26251 pending_rounds.cc:76] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 7
I20260924 14:31:30.111476 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:30.111547 26251 pending_rounds.cc:97] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260924 14:31:30.112653 26251 raft_consensus.cc:2866] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:30.111214 26331 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:30.089742 31597 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:31:30.116551 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:30.122556 26402 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:30.124279 31521 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:30.094878 27194 rpcz_store.cc:276] 0924 14:31:27.484987 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.485161 (+   174us) service_pool.cc:227] Handling call
0924 14:31:27.517713 (+ 32552us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:27.518095 (+   382us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:27.518128 (+    33us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:29.824851 (+2306723us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b2c001c06e8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:29.825601 (+   750us) log.cc:842] Serialized 52 byte log entry
0924 14:31:30.056051 (+230450us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:31:30.056106 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:30.065588 (+  9482us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:30.093934 (+ 28346us) raft_consensus.cc:1671] finished
0924 14:31:30.093954 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:30.094235 (+   281us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":54111,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9614,"spinlock_wait_cycles":3062656,"thread_start_us":192568,"threads_started":1,"wal-append.queue_time_us":246745}
I20260924 14:31:30.144819 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:30.100615 31665 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260924 14:31:30.147974 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.138165 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1651) took 2633 ms. Trace:
I20260924 14:31:30.151242 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:30.153141 26327 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 7. Current term is 8. Ops: []
I20260924 14:31:30.150393 26399 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:30.150327 26257 rpcz_store.cc:276] 0924 14:31:27.398674 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.399142 (+   468us) service_pool.cc:227] Handling call
0924 14:31:27.406807 (+  7665us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:27.420691 (+ 13884us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.987414 (+2566723us) spinlock_profiling.cc:243] Waited 7.31 ms on lock 0x7b54004fb350. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:30.031908 (+ 44494us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":16818432}
I20260924 14:31:30.104934 31039 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:30.112826 31568 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.160347 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.158975 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.161607 26401 raft_consensus.cc:2379] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:30.162179 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1627) took 3150 ms. Trace:
I20260924 14:31:30.150142 31178 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:30.168170 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:30.169194 31721 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:30.169916 31721 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:30.144548 31597 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260924 14:31:30.172804 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:30.173164 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:30.173517 26257 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:30.173965 26249 raft_consensus.cc:2396] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260924 14:31:30.175483 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:30.175984 31766 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260924 14:31:30.176071 31107 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:30.176164 26399 raft_consensus.cc:2362] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17. Re-sending same reply.
I20260924 14:31:30.176723 31107 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:30.177897 31107 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:30.179596 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1432) took 4683 ms. Trace:
I20260924 14:31:30.180037 27192 rpcz_store.cc:276] 0924 14:31:25.461015 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:25.461181 (+   166us) service_pool.cc:227] Handling call
0924 14:31:25.461458 (+   277us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:26.169403 (+707945us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:30.106261 (+3936858us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b5400776854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:30.144023 (+ 37762us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1617560320}
I20260924 14:31:30.180470 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1864) took 1406 ms. Trace:
I20260924 14:31:30.181018 26256 rpcz_store.cc:276] 0924 14:31:28.774082 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.736581 (+962499us) service_pool.cc:227] Handling call
0924 14:31:29.779500 (+ 42919us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:29.794589 (+ 15089us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:29.794621 (+    32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:29.794665 (+    44us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:29.794699 (+    34us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:29.794753 (+    54us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:30.180241 (+385488us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b5400288f50. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:30.180383 (+   142us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":97798400}
I20260924 14:31:30.159745 31566 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:30.162573 26328 rpcz_store.cc:276] 0924 14:31:27.008906 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.175567 (+166661us) service_pool.cc:227] Handling call
0924 14:31:27.175955 (+   388us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:28.279923 (+1103968us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:30.131640 (+1851717us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400557dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:30.158612 (+ 26972us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2535025920}
I20260924 14:31:30.187518 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:30.190478 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:30.234683 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 130426us
I20260924 14:31:30.242784 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:30.188969 31370 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.050s
I20260924 14:31:30.158977 31677 consensus_queue.cc:237] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:30.159572 31431 consensus_queue.cc:260] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:30.248319 31370 consensus_queue.cc:1225] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer 6c22052275034c369b7f7dca086fbd1e log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260924 14:31:30.267688 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:30.268358 26330 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:30.266672 26402 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:30.270346 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.272574 31431 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 7 FOLLOWER]: Advancing to term 8
I20260924 14:31:30.277462 31579 raft_consensus.cc:3032] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:30.277935 31579 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:30.278537 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.279650 31642 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:30.280834 31721 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:30.281210 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:30.281831 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 341563us
I20260924 14:31:30.282104 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.281232 31721 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:30.282840 31721 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:30.283144 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:30.284824 31330 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:30.285184 31330 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:30.285573 31330 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:30.281850 26403 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:30.300207 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.301477 26402 raft_consensus.cc:2354] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9.
I20260924 14:31:30.303566 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:30.304395 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.304585 31568 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:30.305238 26401 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:30.280860 31642 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Stepping down as leader of term 20
I20260924 14:31:30.306668 31642 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:30.302829 31501 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:30.307863 31642 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.308467 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.309381 31665 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:30.309475 31642 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:30.324033 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:30.313656 26405 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:30.325796 26400 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:30.278960 31579 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:30.332679 31579 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:30.342485 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.323980 26255 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:30.348657 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.348403 26403 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
W20260924 14:31:30.355695 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 84981us
I20260924 14:31:30.348497 31501 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:30.347990 31495 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:31:30.393216 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50808.5us
I20260924 14:31:30.368798 31665 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:30.399013 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 230991us
W20260924 14:31:30.404031 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61616.5us
I20260924 14:31:30.401883 27126 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:30.406338 31658 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:30.409075 31658 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:30.410290 31658 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.411012 26253 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 8.
W20260924 14:31:30.423069 31709 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 2.553s	user 0.020s	sys 0.008s
I20260924 14:31:30.401592 31174 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260924 14:31:30.431749 31551 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:30.439505 31783 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:30.441380 31665 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260924 14:31:30.442973 31665 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:30.443466 31665 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:30.392711 31501 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:30.444630 31501 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:30.444576 31665 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:30.446389 31665 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:30.450728 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:30.451624 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.452454 27186 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:30.452556 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.452836 31551 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:30.451701 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.453475 31551 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:30.454149 27188 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
W20260924 14:31:30.457839 31779 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260924 14:31:30.449694 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.459860 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 904) took 3222 ms. Trace:
I20260924 14:31:30.460439 27191 rpcz_store.cc:276] 0924 14:31:27.237915 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.274449 (+ 36534us) service_pool.cc:227] Handling call
0924 14:31:27.274760 (+   311us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:29.828502 (+2553742us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:30.459470 (+630968us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b5400001c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:30.459781 (+   311us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5860144384}
I20260924 14:31:30.460770 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:30.402783 31776 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:30.462527 31579 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:30.473503 27126 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:30.475535 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:30.483188 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 218255us
I20260924 14:31:30.486873 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:30.470037 31783 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:30.490172 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:30.496271 31368 raft_consensus.cc:3032] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:30.497574 31642 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:30.480507 26224 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.504325 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:30.506176 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:30.478415 31579 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:30.511512 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:30.512816 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:30.494225 31677 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:30.514428 26224 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.516382 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:30.519421 31680 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 3.053s	user 0.035s	sys 0.047s
I20260924 14:31:30.519512 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:30.520421 26323 raft_consensus.cc:1226] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:30.521399 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:30.473215 31721 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.529896 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:30.501480 31368 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:30.507184 31642 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:30.532001 31495 raft_consensus.cc:2741] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260924 14:31:30.546842 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1267) took 3025 ms. Trace:
I20260924 14:31:30.547326 26254 rpcz_store.cc:276] 0924 14:31:27.521914 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.181314 (+1659400us) service_pool.cc:227] Handling call
0924 14:31:29.267888 (+ 86574us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:29.277543 (+  9655us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:30.532223 (+1254680us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b5400382cd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:30.546781 (+ 14558us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":195022592}
I20260924 14:31:30.547408 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.526558 31521 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:30.555565 26398 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:30.511112 31551 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:30.531756 31680 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 3.194s	user 0.037s	sys 0.047s
W20260924 14:31:30.558533 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 47148.3us
I20260924 14:31:30.531764 31431 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:30.560272 31551 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:30.560554 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44297.5us
I20260924 14:31:30.479924 31776 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:30.533747 31677 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:30.564720 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:30.552987 31642 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:30.569276 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.557708 31521 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:30.558023 31495 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election won for term 17
W20260924 14:31:30.575094 31788 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260924 14:31:30.576495 31721 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:30.624305 31799 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260924 14:31:30.559278 31330 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:30.578202 31521 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:30.577425 31802 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:30.631932 31802 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:30.632306 31802 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:30.633061 31776 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:30.634658 31789 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260924 14:31:30.635767 31721 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:30.570268 31798 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260924 14:31:30.653146 31803 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:30.653503 31677 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.561718 31579 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.568838 31797 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:30.671950 31330 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:30.647670 31431 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:30.671967 31642 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.672925 31431 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:30.674369 31642 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:30.646432 31495 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:30.675060 31431 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:30.577342 31368 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:30.674959 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 110336us
I20260924 14:31:30.694061 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.659178 31721 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260924 14:31:30.704668 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 135440us
I20260924 14:31:30.701407 31721 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:30.682878 31107 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:30.670476 31797 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:30.679584 31804 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:30.709940 26369 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.680238 31329 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:30.713934 31521 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:30.718047 31726 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:30.732007 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.741974 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:30.746304 31669 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:30.733414 31431 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:30.747743 31431 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:30.748209 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.749513 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.749840 31431 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:30.750470 31783 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:30.751012 31783 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:30.751518 31783 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:30.752271 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.750468 27185 raft_consensus.cc:2396] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260924 14:31:30.754129 31495 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:30.751706 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.708868 31721 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:30.761302 31547 consensus_queue.cc:1046] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:31:30.761938 31698 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:30.782151 27185 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260924 14:31:30.762703 31558 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:30.785048 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:30.785177 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.787161 31698 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.785241 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.763639 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:30.767814 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 237923us
I20260924 14:31:30.768021 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1855) took 3425 ms. Trace:
I20260924 14:31:30.769646 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:30.778707 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:30.783646 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 73865.2us
I20260924 14:31:30.763537 31370 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:30.717854 31802 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.732860 31797 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:30.790201 26258 rpcz_store.cc:276] 0924 14:31:27.342542 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:27.342696 (+   154us) service_pool.cc:227] Handling call
0924 14:31:27.343109 (+   413us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:29.440408 (+2097299us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:30.767730 (+1327322us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b540015ead4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:30.767946 (+   216us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4821066496}
W20260924 14:31:30.794090 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 101590us
I20260924 14:31:30.795969 26257 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260924 14:31:30.797508 26371 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:30.800077 26224 leader_election.cc:398] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9.
I20260924 14:31:30.800427 26224 leader_election.cc:401] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:30.802835 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:30.806726 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:30.807819 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.806775 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:30.807090 31521 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:30.803493 31370 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.805148 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.805588 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:30.809513 26402 raft_consensus.cc:2362] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15. Re-sending same reply.
I20260924 14:31:30.739900 31501 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:30.810316 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.821414 27185 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:30.827298 26369 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:30.811474 31642 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:30.834760 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.809234 31810 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:30.840353 31454 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:30.842022 31803 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:30.842367 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.842547 31454 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:30.844655 31579 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:30.844911 31454 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:30.843062 31803 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:30.845815 31803 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:30.846700 31803 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.848197 31803 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:30.841427 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:30.826601 31809 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 8. Current term is 9.
I20260924 14:31:30.834681 31808 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:30.836750 31642 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:30.830188 31726 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260924 14:31:30.866326 31726 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:30.866665 31726 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:30.861485 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.841137 31810 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:30.863385 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 330618us
I20260924 14:31:30.864276 31370 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:30.864362 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.874226 26399 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:30.876073 31802 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:30.876612 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1116) took 1101 ms. Trace:
I20260924 14:31:30.877005 26326 rpcz_store.cc:276] 0924 14:31:29.780649 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.780803 (+   154us) service_pool.cc:227] Handling call
0924 14:31:30.876421 (+1095618us) spinlock_profiling.cc:243] Waited 86.3 ms on lock 0x7b54005d8c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:30.876553 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":198552448}
I20260924 14:31:30.881102 26404 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:30.884629 31368 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:30.867420 31726 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:30.888797 31726 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:30.887295 27182 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:30.887816 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.864966 31805 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:30.854663 31798 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:30.868371 27188 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:30.862636 31808 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:30.870851 31698 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:30.872843 31039 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:30.896533 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:30.858165 31454 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:30.905588 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1124) took 1070 ms. Trace:
I20260924 14:31:30.870082 31642 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:30.951141 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 46406.9us
I20260924 14:31:30.946975 26324 rpcz_store.cc:276] 0924 14:31:29.832180 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.863787 (+ 31607us) service_pool.cc:227] Handling call
0924 14:31:29.918842 (+ 55055us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:29.931977 (+ 13135us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:29.932011 (+    34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:29.991500 (+ 59489us) log.cc:842] Serialized 26 byte log entry
0924 14:31:30.065863 (+ 74363us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:30.080099 (+ 14236us) log.cc:842] Serialized 14 byte log entry
0924 14:31:30.331484 (+251385us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:30.888907 (+557423us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b18003d7b00. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09bd1f6bcf 00007f09b64267b6 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d
0924 14:31:30.902611 (+ 13704us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:30.902685 (+    74us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:30.902729 (+    44us) raft_consensus.cc:1671] finished
0924 14:31:30.902741 (+    12us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:30.903041 (+   300us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1167866,"prepare.queue_time_us":275367,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":822778,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":15482,"spinlock_wait_cycles":12281344,"thread_start_us":22072,"threads_started":1,"wal-append.queue_time_us":240302}
I20260924 14:31:30.953501 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:30.954406 26403 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 8. Current term is 9. Ops: []
I20260924 14:31:30.949018 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:30.955090 26401 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:30.956612 26258 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:30.957273 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:30.957614 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1121) took 1126 ms. Trace:
I20260924 14:31:30.958511 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:30.893222 31810 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:30.960949 31665 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.514s	user 0.015s	sys 0.000s
I20260924 14:31:30.968518 31454 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:30.947140 31798 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:30.981879 31720 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:30.958210 26325 rpcz_store.cc:276] 0924 14:31:29.831344 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.832820 (+  1476us) service_pool.cc:227] Handling call
0924 14:31:29.833764 (+   944us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:29.834025 (+   261us) raft_consensus.cc:1520] Early marking committed up to index 11
0924 14:31:29.990524 (+156499us) log.cc:842] Serialized 14 byte log entry
0924 14:31:30.457083 (+466559us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:30.957274 (+500191us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:30.957338 (+    64us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:30.957390 (+    52us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:30.957449 (+    59us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:30.957556 (+   107us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":851159,"prepare.queue_time_us":16258,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":845610,"spinlock_wait_cycles":15145216,"thread_start_us":176091,"threads_started":2,"wal-append.queue_time_us":272492}
I20260924 14:31:30.909283 31808 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:30.992100 26255 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:30.968323 31671 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:30.993620 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.000653 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:31.001096 26399 raft_consensus.cc:2379] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 11: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:31.000844 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:30.978550 26257 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:31:30.983548 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:30.995265 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:30.984040 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 680786us
I20260924 14:31:30.908514 31797 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.009887 31677 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:31.010179 31797 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:30.987833 31519 consensus_queue.cc:1046] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260924 14:31:31.011205 31677 consensus_queue.cc:1046] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:31:30.986891 31503 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:31.012349 31178 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Stepping down as leader of term 8
I20260924 14:31:30.974421 31809 consensus_queue.cc:1046] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260924 14:31:31.012668 31503 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:31.012820 31178 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:31.013517 31783 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.014436 27125 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:30.984440 31451 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:31.045217 26405 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:31.048921 26398 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:31.051183 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.029179 31521 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:31.052078 26403 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:31.058450 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.011256 31519 consensus_queue.cc:1225] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer 6c22052275034c369b7f7dca086fbd1e log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260924 14:31:31.072710 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 167685us
I20260924 14:31:31.057879 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.013789 31178 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:31.041133 31330 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.081664 27190 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:31.014649 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:31.044133 31451 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:31.101825 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.111995 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.113193 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:31.074585 31501 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:31.117933 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.119007 31178 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:31.119876 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:31.074925 31521 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:31.119896 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:31.126425 26325 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:31.121191 26328 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:31.121227 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 118213us
I20260924 14:31:31.128306 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:31.132296 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:31.015916 31720 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:31.108167 31451 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:31.150025 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:30.983536 31392 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.137s
I20260924 14:31:31.170269 27186 raft_consensus.cc:2396] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260924 14:31:31.171887 26403 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:31.160177 27191 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:31.175182 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.186494 31573 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.175565 31645 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260924 14:31:31.188086 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:31.134159 31521 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:31.197883 27185 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260924 14:31:31.199523 31346 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:31.200027 31733 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:31.200500 31733 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:31.200905 31733 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:31.201001 31266 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:31.201428 31266 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:31.202062 31279 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:31.202270 31266 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:31.154979 31735 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:31.154721 31335 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:31.210443 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.188058 31121 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260924 14:31:31.109411 31454 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:31.225795 26255 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260924 14:31:31.173712 26295 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.193:38943 timed out after 1.703s (SENT) [suppressed 25 similar messages]
I20260924 14:31:31.236061 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.133572 31330 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.239732 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:31.130963 31721 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:31.243870 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:31.187261 31573 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:31.251142 31573 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:31.222198 31735 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:31.218236 31335 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:31.230129 31783 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:31.266927 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.265633 26401 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:31.217327 31804 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:31.284091 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.285307 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:31.286060 26405 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:31.243549 31696 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:31.292089 26397 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:31.293768 26401 raft_consensus.cc:2354] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17.
W20260924 14:31:31.287071 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 253539us
I20260924 14:31:31.287922 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:31.286998 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.265817 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:31.292433 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 52837.5us
I20260924 14:31:31.298024 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.298609 27126 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:31.298960 27186 raft_consensus.cc:2396] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260924 14:31:31.269289 31266 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:31.277206 31735 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:31.331254 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.331367 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.283586 31335 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:31.270375 31541 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:31.308326 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:31.300828 31547 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:31.367221 27185 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:31.370370 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:31.291982 31696 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:31.381690 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.306932 31501 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:31.321767 31562 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260924 14:31:31.388118 31501 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.397953 31451 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.398622 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.399307 26324 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:31.356345 31431 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.346832 31392 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:31.347100 31798 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:31.301492 31804 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:31.470731 26298 leader_election.cc:398] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17.
I20260924 14:31:31.471153 26298 leader_election.cc:401] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:31.478004 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.478660 26326 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:31.422325 26405 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:31.340701 31645 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:31.370100 31547 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:31.385499 31696 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:31.381453 31541 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:31.445454 27192 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:31.399904 31562 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:31.483474 31445 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.483641 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.471874 31720 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:31.484720 26331 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260924 14:31:31.484954 31516 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.485258 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.485510 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:31.485760 31329 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 17
I20260924 14:31:31.487705 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.488166 31346 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:31.457396 31121 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.475834 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:31.488729 31367 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.223s
I20260924 14:31:31.442826 31671 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.496284 31721 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:31.496423 31818 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260924 14:31:31.497435 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:31.499048 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 325360us
I20260924 14:31:31.502743 26326 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:31.510573 31645 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:31.523718 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.533183 27185 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:31.483750 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:31.464069 31392 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:31.546386 26325 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:31.508153 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.554924 26401 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:31.555289 26326 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:31.504706 31501 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:31.449978 31431 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.510764 31547 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:31.578490 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:31.563140 31808 consensus_meta.cc:287] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.000s
I20260924 14:31:31.537367 31562 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:31.568177 31346 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
W20260924 14:31:31.607905 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 313261us
I20260924 14:31:31.508870 31558 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.056s
I20260924 14:31:31.578639 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.562845 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.509521 31798 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:31.580219 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.581614 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.545406 31445 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.584139 26258 raft_consensus.cc:3032] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:31.563745 31121 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:31.592451 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.595245 31431 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:31.595963 31541 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.604003 31392 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:31.606492 26328 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:31:31.571168 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.616219 31721 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.616452 27185 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:31:31.621889 26405 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:31.557994 31671 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:31.579216 31330 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:31.630419 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.631145 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:31.612458 31346 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:31.553681 31516 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:31.622419 26258 raft_consensus.cc:729] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:31.643996 31721 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:31.644641 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.645025 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.645213 31804 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:31.645736 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:31.633991 31803 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.785s	user 0.008s	sys 0.005s
I20260924 14:31:31.652197 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:31.655694 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:31.648227 31178 consensus_meta.cc:287] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.000s
W20260924 14:31:31.638496 31814 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.773s	user 0.004s	sys 0.002s
I20260924 14:31:31.669888 26401 raft_consensus.cc:2362] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15. Re-sending same reply.
I20260924 14:31:31.675046 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:31.683394 26258 consensus_queue.cc:260] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:31.684818 26258 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:31.685716 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.686807 26401 raft_consensus.cc:2396] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260924 14:31:31.687819 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.687999 31731 raft_consensus.cc:3032] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:31.688444 31731 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:31.689082 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.689304 31731 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:31.690431 31731 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:31.690515 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.691730 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.692248 31579 raft_consensus.cc:3032] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:31.692689 31579 raft_consensus.cc:729] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:31.693087 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.694015 31547 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:31.695303 27186 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:31.693531 31579 consensus_queue.cc:260] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:31.700636 31579 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:31.702236 26397 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:31.704828 31547 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:31.724517 26401 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:31.724712 26258 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:31.725603 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.610076 31808 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:31.736400 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 387543us
I20260924 14:31:31.730722 31805 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.670509 31818 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:31.636848 31798 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.744935 26326 raft_consensus.cc:3032] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:31.740680 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:31.758093 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.758332 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:31.754987 26326 raft_consensus.cc:729] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:31:31.741588 31811 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260924 14:31:31.733140 31623 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:31.764016 26258 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:31.776316 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.725023 31451 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:31.771889 26255 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260924 14:31:31.775714 26397 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:31.762460 31810 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.764s	user 0.009s	sys 0.003s
I20260924 14:31:31.782945 26405 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 7.
I20260924 14:31:31.782384 26326 consensus_queue.cc:260] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:31.763082 31454 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:31.731184 31367 consensus_queue.cc:1046] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260924 14:31:31.824250 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:31.782260 31797 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:31.799897 31503 consensus_meta.cc:287] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.787s	user 0.014s	sys 0.000s
I20260924 14:31:31.840021 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.782963 31818 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:31.847229 26258 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:31.855545 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:31.852150 26326 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:31.751773 31805 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.857916 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:31.814050 31810 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.798180 31346 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:31.912110 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:31.916458 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:31.862913 26258 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:31.851562 31454 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:31.863323 31516 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:31.864310 31335 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.000s
I20260924 14:31:31.864140 31678 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260924 14:31:31.852133 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:31.924909 26255 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:31.925304 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:31.865430 26324 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:31.866006 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:31.878979 31783 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.877532 26326 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260924 14:31:31.883579 31767 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.884871 31808 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:31.886616 27181 raft_consensus.cc:2396] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260924 14:31:31.889498 31645 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:31.891016 31521 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.669s	user 0.007s	sys 0.003s
W20260924 14:31:31.892484 31279 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.690s	user 0.006s	sys 0.007s
I20260924 14:31:31.893810 31798 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:31.961076 27189 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:31.962004 31005 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.962497 31005 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:31.895856 31368 consensus_meta.cc:287] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.011s	user 0.012s	sys 0.003s
W20260924 14:31:31.896029 31820 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260924 14:31:31.964794 27189 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 957) took 2614 ms (client timeout 2032 ms). Trace:
I20260924 14:31:31.897224 31121 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:31.967509 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1786) took 1107 ms. Trace:
I20260924 14:31:31.968144 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1785) took 1135 ms. Trace:
W20260924 14:31:31.897993 31642 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.930s	user 0.008s	sys 0.008s
I20260924 14:31:31.968080 26398 rpcz_store.cc:276] 0924 14:31:30.860321 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:30.860493 (+   172us) service_pool.cc:227] Handling call
0924 14:31:30.864747 (+  4254us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:31.057090 (+192343us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:31.967183 (+910093us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400335754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:31.967415 (+   232us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":423111424}
I20260924 14:31:31.969065 31642 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.898404 31688 raft_consensus.cc:3032] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:31.970155 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:31.970463 31688 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:31.971228 26404 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:31.952756 31767 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:31.971428 31688 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:31.955056 31803 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:31.972776 31688 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:31.972975 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1787) took 1017 ms. Trace:
I20260924 14:31:31.973104 31803 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:31.973614 26404 rpcz_store.cc:276] 0924 14:31:30.956094 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:30.956212 (+   118us) service_pool.cc:227] Handling call
0924 14:31:30.971620 (+ 15408us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:31.971833 (+1000213us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:31.972659 (+   826us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54001e4bd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:31.972903 (+   244us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2298253184}
W20260924 14:31:31.965351 27189 rpcz_store.cc:269] 0924 14:31:29.350235 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:29.356707 (+  6472us) service_pool.cc:227] Handling call
0924 14:31:31.964510 (+2607803us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b540000c154. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:31.964696 (+   186us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5854916992}
I20260924 14:31:31.975788 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:31.976434 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:31.898665 31720 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:31.977288 27189 raft_consensus.cc:2362] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15. Re-sending same reply.
W20260924 14:31:31.898445 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 43547.4us
W20260924 14:31:31.903951 31823 log.cc:924] Time spent T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260924 14:31:31.976465 26327 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:31.856475 31818 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:31.883155 31329 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17.
I20260924 14:31:31.921051 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:31.923638 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:31.925890 26250 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:31.928755 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:31.988027 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 59342us
I20260924 14:31:31.931006 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:31.928604 26326 pending_rounds.cc:76] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 8
I20260924 14:31:31.927572 31335 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:31.921849 31454 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:31.954595 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:32.017506 26251 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:31.955476 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:31.942895 31783 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:31.855172 31797 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:31.917173 26252 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:31.989518 26326 pending_rounds.cc:97] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260924 14:31:32.021423 26326 raft_consensus.cc:2866] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:31.986469 26253 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 8.
I20260924 14:31:31.990975 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:31.968680 26400 rpcz_store.cc:276] 0924 14:31:30.836431 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:30.837084 (+   653us) service_pool.cc:227] Handling call
0924 14:31:30.853603 (+ 16519us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:30.952548 (+ 98945us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:30.952577 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:30.957571 (+  4994us) log.cc:842] Serialized 26 byte log entry
0924 14:31:31.210249 (+252678us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:31.210289 (+    40us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:31.267356 (+ 57067us) spinlock_profiling.cc:243] Waited 56.9 ms on lock 0x7b5400217854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:31.267421 (+    65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:31.967852 (+700431us) raft_consensus.cc:1671] finished
0924 14:31:31.967875 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:31.968088 (+   213us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":130883456,"thread_start_us":230057,"threads_started":1,"wal-append.queue_time_us":687380}
I20260924 14:31:32.019137 26258 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:32.028111 26323 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:31.963209 31562 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:32.030301 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:31.988903 31721 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:32.034039 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:31.983814 31521 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:32.036667 26398 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:32.044241 26326 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:32.045728 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:32.046665 26400 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260924 14:31:32.047166 26400 pending_rounds.cc:76] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 11
I20260924 14:31:32.047428 26400 pending_rounds.cc:97] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260924 14:31:32.044912 27194 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:32.033898 31808 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:32.056185 31808 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:32.051163 31721 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:32.059144 31721 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:32.060107 31805 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:32.060276 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1788) took 1072 ms. Trace:
I20260924 14:31:32.061161 31721 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:32.060761 26399 rpcz_store.cc:276] 0924 14:31:30.988102 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.111645 (+123543us) service_pool.cc:227] Handling call
0924 14:31:31.135317 (+ 23672us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:31.154576 (+ 19259us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:31.154616 (+    40us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:31.171664 (+ 17048us) log.cc:842] Serialized 26 byte log entry
0924 14:31:31.370556 (+198892us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:31.370607 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:31.370673 (+    66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:32.060052 (+689379us) raft_consensus.cc:1671] finished
0924 14:31:32.060075 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:32.060216 (+   141us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":183942,"threads_started":1,"wal-append.queue_time_us":469624}
I20260924 14:31:32.064374 26399 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260924 14:31:32.065613 26399 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:32.066668 26399 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:32.041522 31431 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:32.067767 31829 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:32.068307 31829 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:32.070380 31829 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:32.047739 26400 raft_consensus.cc:2866] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:32.074683 31808 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:32.076081 31562 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:32.077786 26399 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260924 14:31:32.078917 31266 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.744s	user 0.008s	sys 0.002s
I20260924 14:31:32.079485 27188 raft_consensus.cc:595] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:32.079864 26224 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.021s	sys 0.062s
W20260924 14:31:32.080211 26224 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.021s	sys 0.062s
I20260924 14:31:32.081248 31521 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:32.081679 27191 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 964) took 1606 ms (client timeout 2048 ms). Trace:
W20260924 14:31:32.082228 27191 rpcz_store.cc:269] 0924 14:31:30.475004 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:30.475194 (+   190us) service_pool.cc:227] Handling call
0924 14:31:32.081449 (+1606255us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b5400260a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:32.081587 (+   138us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1134063872}
W20260924 14:31:32.083545 26332 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.016s	sys 0.042s
W20260924 14:31:32.083894 26332 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.016s	sys 0.042s
I20260924 14:31:32.089493 31803 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:32.092293 31642 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:32.092921 31818 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:32.094759 31335 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:32.079903 27188 raft_consensus.cc:603] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:32.096364 31810 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:32.096675 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:32.098067 31733 consensus_meta.cc:287] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.897s	user 0.010s	sys 0.003s
I20260924 14:31:32.098578 31733 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:32.040958 31819 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260924 14:31:32.026630 31798 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:32.102273 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:32.106436 26327 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
W20260924 14:31:32.107702 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 395307us
I20260924 14:31:32.075964 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:32.123863 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:32.127553 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:32.129027 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:32.026724 31696 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:32.129971 26326 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
W20260924 14:31:32.100522 26295 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.016s	sys 0.027s
I20260924 14:31:32.135527 26251 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:31:32.140630 26295 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.016s	sys 0.027s
W20260924 14:31:32.141042 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 572410us
W20260924 14:31:32.142807 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.057s	sys 0.063s
W20260924 14:31:32.143258 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.057s	sys 0.063s
I20260924 14:31:32.147214 31720 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:32.148005 31720 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:32.150921 31720 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:32.154290 26331 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260924 14:31:32.155606 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 671865us
I20260924 14:31:32.156461 26298 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:32.157864 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:32.119251 31810 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:32.162041 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:32.163193 26251 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:32.162194 26250 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:32.169485 27182 raft_consensus.cc:1226] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:32.170675 31731 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:32.137233 31547 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:32.173861 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:32.144567 31818 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:32.168915 27125 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:32.175570 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:32.194718 31451 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:32.195291 31451 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:32.195693 31451 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:32.195974 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:32.052415 31830 consensus_queue.cc:1035] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.124s
I20260924 14:31:32.204151 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:32.206940 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:32.125851 31798 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:32.205030 27191 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:32.206080 26330 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:32.218542 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1790) took 1229 ms. Trace:
I20260924 14:31:32.219182 26403 rpcz_store.cc:276] 0924 14:31:30.988808 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.186630 (+197822us) service_pool.cc:227] Handling call
0924 14:31:31.239059 (+ 52429us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:31.263995 (+ 24936us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:31.293667 (+ 29672us) log.cc:842] Serialized 14 byte log entry
0924 14:31:31.635775 (+342108us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:31.635826 (+    51us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:31.646214 (+ 10388us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:31.646281 (+    67us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:32.218318 (+572037us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b54002fbcd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:32.218466 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":27896,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":4445,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":102,"spinlock_wait_cycles":119720832,"thread_start_us":296497,"threads_started":1,"wal-append.queue_time_us":662832}
I20260924 14:31:32.224792 31671 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:32.231017 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:32.232161 26405 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:32.233180 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:32.233402 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:32.234702 26405 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:32.236430 27188 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:32.238391 31346 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:32.238960 31781 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:32.239748 31781 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:32.246758 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:32.247756 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:32.248224 26403 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:32.256167 31781 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:32.258550 27186 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:32.259507 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:32.259975 27185 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:32.262113 31445 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:32.263319 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:32.262581 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e81d1761568f40418d9259b74501d677"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:32.264496 27188 raft_consensus.cc:595] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:32.265627 26401 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260924 14:31:32.268498 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 969) took 1025 ms. Trace:
I20260924 14:31:32.269011 27182 rpcz_store.cc:276] 0924 14:31:31.242945 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.243241 (+   296us) service_pool.cc:227] Handling call
0924 14:31:31.268610 (+ 25369us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:32.169965 (+901355us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:32.268208 (+ 98243us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b5400071a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:32.268423 (+   215us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2071321984}
I20260924 14:31:32.270557 27182 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:32.273070 31678 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:32.235034 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:32.275573 26326 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 17. Ops: []
I20260924 14:31:32.275915 27181 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:32.277444 27181 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:32.278975 31516 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260924 14:31:32.285053 27185 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:32.285534 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:32.286016 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:32.287525 31678 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:32.296655 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:32.271363 27182 raft_consensus.cc:595] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:32.285545 26326 raft_consensus.cc:1226] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [14.11-17.13]
I20260924 14:31:32.303581 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260924 14:31:32.307134 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:31:32.308027 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260924 14:31:32.309018 26076 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 141) took 43009 ms (client timeout 15000 ms). Trace:
I20260924 14:31:32.309306 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:32.318010 26405 raft_consensus.cc:2354] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14.
I20260924 14:31:32.324177 26077 catalog_manager.cc:5495] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 14, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 14 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:32.328337 26403 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:32.333171 26295 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:32.209277 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1547) took 1367 ms. Trace:
I20260924 14:31:32.328512 26077 catalog_manager.cc:5495] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 10, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 10 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:32.337019 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:32.338060 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:32.171263 31828 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260924 14:31:32.338683 27189 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:32.335601 26402 rpcz_store.cc:276] 0924 14:31:30.841644 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:30.853313 (+ 11669us) service_pool.cc:227] Handling call
0924 14:31:30.862937 (+  9624us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:30.863249 (+   312us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:30.863288 (+    39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:30.868091 (+  4803us) log.cc:842] Serialized 26 byte log entry
0924 14:31:31.389548 (+521457us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:31.389590 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:31.389661 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:32.208999 (+819338us) raft_consensus.cc:1671] finished
0924 14:31:32.209030 (+    31us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:32.209205 (+   175us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":419931,"threads_started":1,"wal-append.queue_time_us":887369}
I20260924 14:31:32.359432 26400 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260924 14:31:32.359505 26077 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 13, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 13 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:32.365823 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:32.367605 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:32.212241 31688 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:32.215060 31731 consensus_queue.cc:1046] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
I20260924 14:31:32.368398 27191 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:32.211752 31818 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:32.371047 31688 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:32.169596 31810 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:32.372581 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:32.360572 31721 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:32.201591 31547 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:32.378890 31562 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260924 14:31:32.391098 26077 catalog_manager.cc:5495] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 13, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 13 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:32.371053 31519 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:32.334482 31623 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.047s
I20260924 14:31:32.393682 31721 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:32.337929 31121 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.045s
I20260924 14:31:32.381785 31688 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:32.413018 26077 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 10, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 10 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:32.414654 31767 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:32.381760 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:32.365190 31445 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260924 14:31:32.419862 26077 catalog_manager.cc:5495] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:32.425669 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:32.411979 31735 consensus_meta.cc:287] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.055s	user 0.008s	sys 0.005s
I20260924 14:31:32.435600 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:32.438318 26251 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:31:32.435513 31519 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:32.376124 31279 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260924 14:31:32.400892 31797 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:32.438349 27181 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260924 14:31:32.439918 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:32.439960 26257 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.013s	sys 0.012s
I20260924 14:31:32.405889 31783 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:32.446998 26257 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.013s	sys 0.012s
I20260924 14:31:32.449836 26326 raft_consensus.cc:2354] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9.
I20260924 14:31:32.421926 31721 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:32.443674 26077 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 11, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 11 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
W20260924 14:31:32.454125 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 73402.1us
I20260924 14:31:32.455137 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:32.457108 31266 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:32.457557 31266 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:32.457921 31266 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:32.458240 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:32.459806 31699 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:32.460691 31445 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:32.463570 31677 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:32.464136 31677 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:32.465401 27189 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:32.466485 31677 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:32.467950 31039 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:32.469718 31699 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:32.470166 31699 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:32.473513 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:32.474538 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 43904.6us
I20260924 14:31:32.475980 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:32.459153 26402 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:31:32.376879 31832 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260924 14:31:32.427664 31733 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:32.480151 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:32.487766 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:32.484970 26324 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.017s	sys 0.008s
I20260924 14:31:32.376267 31558 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
W20260924 14:31:32.493013 31573 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.057s	sys 0.085s
I20260924 14:31:32.486207 26077 catalog_manager.cc:5495] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 12, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 12 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
W20260924 14:31:32.491079 26324 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.018s	sys 0.009s
W20260924 14:31:32.483349 26254 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.011s	sys 0.023s
I20260924 14:31:32.488546 26251 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:32.486696 26326 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
W20260924 14:31:32.501472 27192 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 960) took 2051 ms (client timeout 2049 ms). Trace:
W20260924 14:31:32.501864 26254 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.011s	sys 0.024s
W20260924 14:31:32.501875 27192 rpcz_store.cc:269] 0924 14:31:30.450166 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:30.450353 (+   187us) service_pool.cc:227] Handling call
0924 14:31:32.501300 (+2050947us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b540000da54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:32.501398 (+    98us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1351724416}
I20260924 14:31:32.507171 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:32.508538 26077 catalog_manager.cc:5495] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:32.514631 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:32.520809 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:32.519709 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1751) took 1209 ms. Trace:
I20260924 14:31:32.524385 26400 rpcz_store.cc:276] 0924 14:31:31.298502 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:32.073934 (+775432us) service_pool.cc:227] Handling call
0924 14:31:32.074231 (+   297us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:32.360042 (+285811us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:32.506600 (+146558us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54006e78d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:32.508028 (+  1428us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":655022976}
I20260924 14:31:32.532263 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1550) took 1442 ms. Trace:
W20260924 14:31:32.493721 31573 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.057s	sys 0.085s
I20260924 14:31:32.532893 26397 rpcz_store.cc:276] 0924 14:31:31.090329 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.703196 (+612867us) service_pool.cc:227] Handling call
0924 14:31:31.710018 (+  6822us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:31.803884 (+ 93866us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:32.531919 (+728035us) spinlock_profiling.cc:243] Waited 65.5 ms on lock 0x7b54003d59d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:32.532183 (+   264us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":150648064}
I20260924 14:31:32.535250 26401 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260924 14:31:32.523643 27190 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 918) took 1745 ms (client timeout 1785 ms). Trace:
W20260924 14:31:32.548511 27190 rpcz_store.cc:269] 0924 14:31:30.778317 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:30.778440 (+   123us) service_pool.cc:227] Handling call
0924 14:31:32.523432 (+1744992us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b540070f8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:32.523582 (+   150us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":668035456}
I20260924 14:31:32.445503 31735 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:32.497792 31279 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:32.588297 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:32.602536 26331 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
W20260924 14:31:32.589677 26258 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.037s	sys 0.048s
W20260924 14:31:32.595885 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 102969us
I20260924 14:31:32.596443 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:32.595561 26077 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:31:32.595749 26256 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.008s	sys 0.025s
W20260924 14:31:32.605268 26256 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.009s	sys 0.025s
W20260924 14:31:32.605536 26258 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.037s	sys 0.048s
I20260924 14:31:32.500629 31696 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:32.508448 31039 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260924 14:31:32.508116 31804 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.014s	sys 0.062s
W20260924 14:31:32.613719 26252 consensus_meta.cc:287] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.593s	user 0.012s	sys 0.000s
I20260924 14:31:32.614679 26252 raft_consensus.cc:2396] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260924 14:31:32.471242 31699 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:32.616986 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:32.523316 31720 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:32.618539 26326 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:32.620163 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:32.623922 26329 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:32.588204 31329 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:32.591562 31783 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:32.593653 31808 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:32.628484 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:32.625432 26077 catalog_manager.cc:5495] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
W20260924 14:31:32.617974 31804 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.014s	sys 0.062s
I20260924 14:31:32.602154 31279 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:32.639951 26326 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:32.666188 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.006s	sys 0.032s
W20260924 14:31:32.666725 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.006s	sys 0.032s
I20260924 14:31:32.666482 26077 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
W20260924 14:31:32.637945 26222 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.014s	sys 0.023s
W20260924 14:31:32.668185 26222 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.014s	sys 0.024s
W20260924 14:31:32.668617 26332 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.011s	sys 0.020s
W20260924 14:31:32.669040 26332 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.011s	sys 0.020s
I20260924 14:31:32.498860 31519 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:32.623222 31720 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:32.489544 31818 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:32.667084 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 646654us
I20260924 14:31:32.683108 26077 catalog_manager.cc:5495] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 12. New cstate: current_term: 12 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:32.690553 26331 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:32.685114 26325 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
W20260924 14:31:32.641211 31392 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.037s	sys 0.031s
I20260924 14:31:32.626358 31798 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:32.669374 31279 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:32.693977 31547 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:32.641826 31039 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260924 14:31:32.670593 31830 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:32.625303 31329 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:32.704731 26326 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:32.711303 31329 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:32.713187 31329 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:32.719991 31279 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:32.726661 31279 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:32.707285 26077 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 14, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 14 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:32.671079 31645 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:31:32.698961 31392 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.037s	sys 0.032s
I20260924 14:31:32.689675 31720 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:31:32.746713 26253 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.007s	sys 0.022s
W20260924 14:31:32.747301 26253 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.007s	sys 0.022s
W20260924 14:31:32.743147 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52463.1us
I20260924 14:31:32.749341 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:32.768667 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1132) took 1023 ms. Trace:
W20260924 14:31:32.316210 26076 rpcz_store.cc:269] 0924 14:30:49.299407 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:49.299709 (+   302us) service_pool.cc:227] Handling call
0924 14:31:32.308769 (+43009060us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:22.937738 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:22.938688 (+   950us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:22.939062 (+   374us) write_op.cc:435] Acquired schema lock
0924 14:31:22.939093 (+    31us) tablet.cc:660] Decoding operations
0924 14:31:22.940844 (+  1751us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:22.941020 (+   176us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:22.941084 (+    64us) tablet.cc:683] Acquiring locks for 13 operations
0924 14:31:22.943177 (+  2093us) tablet.cc:699] Row locks acquired
0924 14:31:22.943229 (+    52us) write_op.cc:260] PREPARE: finished
0924 14:31:22.943541 (+   312us) write_op.cc:270] Start()
0924 14:31:22.943749 (+   208us) write_op.cc:276] Timestamp: P: 1790260282943425 usec, L: 0
0924 14:31:22.943800 (+    51us) op_driver.cc:331] REPLICATION: starting
0924 14:31:22.944946 (+  1146us) log.cc:842] Serialized 5236 byte log entry
0924 14:31:25.741390 (+2796444us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b64085fe 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb
0924 14:31:27.080242 (+1338852us) op_driver.cc:444] REPLICATION: finished
0924 14:31:27.531303 (+451061us) write_op.cc:301] APPLY: starting
0924 14:31:27.563916 (+ 32613us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:28.526198 (+962282us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:28.526248 (+    50us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:28.574869 (+ 48621us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:28.575313 (+   444us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0924 14:31:28.575399 (+    86us) write_op.cc:312] APPLY: finished
0924 14:31:28.576254 (+   855us) log.cc:842] Serialized 120 byte log entry
0924 14:31:28.576513 (+   259us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:28.576710 (+   197us) write_op.cc:454] Released schema lock
0924 14:31:28.576801 (+    91us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:28.577162 (+   361us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:28.577202 (+    40us) write_op.cc:454] Released schema lock
0924 14:31:30.761591 (+2184389us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b18000bc3e0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426764 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
Metrics: {"mutex_wait_us":1391136,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":389580,"apply.run_cpu_time_us":24447,"apply.run_wall_time_us":1062934,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58519,"num_ops":13,"prepare.queue_time_us":74622,"prepare.run_cpu_time_us":72667,"prepare.run_wall_time_us":3266871,"raft.queue_time_us":216418,"raft.run_cpu_time_us":28174,"raft.run_wall_time_us":299632,"replication_time_us":4136336,"spinlock_wait_cycles":50349312,"thread_start_us":3708223,"threads_started":4,"wal-append.queue_time_us":5452969,"wal-append.run_cpu_time_us":38998,"wal-append.run_wall_time_us":2886591}]]}
I20260924 14:31:32.777801 26330 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260924 14:31:32.769174 26325 rpcz_store.cc:276] 0924 14:31:31.744921 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.745053 (+   132us) service_pool.cc:227] Handling call
0924 14:31:32.096510 (+351457us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:31:32.768504 (+671994us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54002774d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:32.768612 (+   108us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":913175680}
I20260924 14:31:32.644274 31808 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:32.676116 31783 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:32.777406 26399 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.022s	sys 0.020s
W20260924 14:31:32.792672 26399 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.022s	sys 0.020s
W20260924 14:31:32.794627 26328 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.023s	sys 0.036s
W20260924 14:31:32.795042 26328 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.023s	sys 0.037s
I20260924 14:31:32.793615 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1755) took 1292 ms. Trace:
I20260924 14:31:32.744059 31810 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:32.795517 26399 rpcz_store.cc:276] 0924 14:31:31.501650 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:32.079382 (+577732us) service_pool.cc:227] Handling call
0924 14:31:32.086472 (+  7090us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:32.086680 (+   208us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:32.086931 (+   251us) log.cc:842] Serialized 14 byte log entry
0924 14:31:32.793256 (+706325us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:32.793310 (+    54us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:32.793356 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:32.793423 (+    67us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:32.793560 (+   137us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":6264,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":94,"thread_start_us":705924,"threads_started":1}
W20260924 14:31:32.801324 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52106.9us
I20260924 14:31:32.815711 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:32.794265 31541 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.161s	user 0.025s	sys 0.020s
W20260924 14:31:32.826532 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.012s	sys 0.029s
W20260924 14:31:32.831112 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.012s	sys 0.029s
I20260924 14:31:32.822036 26077 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:32.744602 31830 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:32.830071 31541 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.026s	sys 0.021s
I20260924 14:31:32.857353 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:32.858685 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:32.852677 26323 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.015s	sys 0.029s
I20260924 14:31:32.852353 26077 catalog_manager.cc:5495] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:31:32.861045 26323 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.015s	sys 0.030s
I20260924 14:31:32.820115 31005 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:32.863904 31844 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:32.860486 27125 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:32.864719 31844 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:32.864192 26329 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:32.865532 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:32.866150 26257 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:32.867311 31696 raft_consensus.cc:979] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 6c22052275034c369b7f7dca086fbd1e to start an election
I20260924 14:31:32.867810 31696 raft_consensus.cc:1067] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Signalling peer 6c22052275034c369b7f7dca086fbd1e to start an election
W20260924 14:31:32.870596 26329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1138) took 1573 ms (client timeout 1800 ms). Trace:
W20260924 14:31:32.871016 26329 rpcz_store.cc:269] 0924 14:31:31.296733 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.296891 (+   158us) service_pool.cc:227] Handling call
0924 14:31:32.870430 (+1573539us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54007d37d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:32.870539 (+   109us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2967779200}
W20260924 14:31:32.864993 31822 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 1.230s	user 0.000s	sys 0.007s
I20260924 14:31:32.876641 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:32.831415 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 620483us
I20260924 14:31:32.818150 31811 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:32.879755 31846 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:32.893904 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1654) took 1414 ms. Trace:
I20260924 14:31:32.908808 31783 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:32.859175 31451 consensus_meta.cc:287] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.000s
I20260924 14:31:32.933480 31733 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:32.936352 31699 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:32.940840 31454 consensus_meta.cc:287] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.916s	user 0.012s	sys 0.001s
I20260924 14:31:32.947693 26257 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:32.948485 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1477) took 1466 ms. Trace:
I20260924 14:31:32.949932 26077 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
W20260924 14:31:32.951809 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 75294.6us
W20260924 14:31:32.953896 26403 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.015s	sys 0.019s
I20260924 14:31:32.949724 26332 rpcz_store.cc:276] 0924 14:31:31.479227 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.480521 (+  1294us) service_pool.cc:227] Handling call
0924 14:31:31.488198 (+  7677us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:32.084369 (+596171us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:32.084402 (+    33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:32.085224 (+   822us) log.cc:842] Serialized 26 byte log entry
0924 14:31:32.669414 (+584190us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:32.669453 (+    39us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:32.669571 (+   118us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:32.893586 (+224015us) raft_consensus.cc:1671] finished
0924 14:31:32.893612 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:32.893832 (+   220us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1334,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":1179327,"threads_started":2,"wal-append.queue_time_us":771649}
I20260924 14:31:32.865008 31279 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:31:32.960240 26252 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e81d1761568f40418d9259b74501d677"
dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:55802
W20260924 14:31:32.963922 26403 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.015s	sys 0.019s
I20260924 14:31:32.961284 27181 rpcz_store.cc:276] 0924 14:31:31.482516 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:32.278434 (+795918us) service_pool.cc:227] Handling call
0924 14:31:32.278798 (+   364us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:32.446249 (+167451us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:32.941242 (+494993us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54000028d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:32.948389 (+  7147us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":329747328}
I20260924 14:31:32.965209 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:32.968957 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:32.872084 31735 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:32.970777 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:32.975865 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:32.972990 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:32.983793 26077 catalog_manager.cc:5495] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:32.915130 31266 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:32.966985 26295 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.015s	sys 0.020s
W20260924 14:31:32.986389 26295 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.015s	sys 0.020s
I20260924 14:31:32.956373 31733 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:32.957257 31843 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260924 14:31:32.991727 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 620824us
I20260924 14:31:32.995496 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:32.995996 26295 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:32.997598 26404 raft_consensus.cc:1226] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:32.974588 31831 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:32.982736 31431 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:32.974829 31811 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:31:32.994159 31623 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.002s	sys 0.027s
I20260924 14:31:32.957258 31451 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:33.006032 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1732) took 2726 ms. Trace:
I20260924 14:31:32.987337 31279 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:31:32.968490 31454 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:32.971185 31846 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260924 14:31:33.039592 26077 catalog_manager.cc:5495] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 8, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 8 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:33.043370 31451 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:33.014111 31623 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.002s	sys 0.028s
I20260924 14:31:33.013530 26406 rpcz_store.cc:276] 0924 14:31:30.271045 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:30.274776 (+  3731us) service_pool.cc:227] Handling call
0924 14:31:30.902314 (+627538us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b5800143d98. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2a42b0 00007f09be28f7f9 00007f09be28f2a2 00007f09b7975684
0924 14:31:30.902494 (+   180us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:30.909539 (+  7045us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:30.909577 (+    38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:30.968630 (+ 59053us) log.cc:842] Serialized 26 byte log entry
0924 14:31:31.338476 (+369846us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:31.338524 (+    48us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:31.338601 (+    77us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:32.864386 (+1525785us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:32.997250 (+132864us) raft_consensus.cc:1671] finished
0924 14:31:32.997275 (+    25us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:32.997412 (+   137us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":4329,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":34378,"spinlock_wait_cycles":51910144,"thread_start_us":308797,"threads_started":1,"wal-append.queue_time_us":641683}
W20260924 14:31:33.055056 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 442257us
I20260924 14:31:33.013759 31843 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:33.059592 26249 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260924 14:31:33.063019 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 472490us
I20260924 14:31:33.037562 31431 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:33.047698 31830 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:33.056691 31688 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.621s	user 0.004s	sys 0.008s
W20260924 14:31:33.075232 27194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 912) took 2343 ms (client timeout 2174 ms). Trace:
I20260924 14:31:33.050921 31279 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:33.076993 27194 rpcz_store.cc:269] 0924 14:31:30.731803 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:30.731958 (+   155us) service_pool.cc:227] Handling call
0924 14:31:33.075034 (+2343076us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540000d054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:33.075156 (+   122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2810929408}
W20260924 14:31:33.041352 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45933.7us
I20260924 14:31:33.072744 26077 catalog_manager.cc:5495] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 6 to 11, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 11 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:33.058866 31808 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:33.083986 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:33.084879 26401 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:33.088387 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.091369 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.093730 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.095940 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.097635 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1128) took 1429 ms. Trace:
W20260924 14:31:33.098140 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 102298us
I20260924 14:31:33.098263 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.100064 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1139) took 1125 ms. Trace:
W20260924 14:31:33.097711 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 110986us
I20260924 14:31:33.099103 26295 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.105005 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:33.105764 26406 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 21.
I20260924 14:31:33.102888 31731 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:33.100584 26324 rpcz_store.cc:276] 0924 14:31:31.974992 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.975117 (+   125us) service_pool.cc:227] Handling call
0924 14:31:31.975513 (+   396us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:31.975919 (+   406us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:31.975947 (+    28us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:31.976552 (+   605us) log.cc:842] Serialized 52 byte log entry
0924 14:31:32.503564 (+527012us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:33.011385 (+507821us) log.cc:842] Serialized 14 byte log entry
0924 14:31:33.099676 (+ 88291us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:33.099775 (+    99us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:33.099830 (+    55us) raft_consensus.cc:1671] finished
0924 14:31:33.099846 (+    16us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:33.099997 (+   151us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":174217,"prepare.queue_time_us":264118,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":7986,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4019,"thread_start_us":908117,"threads_started":2,"wal-append.queue_time_us":768889}
I20260924 14:31:33.108024 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:33.108775 26399 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:33.109777 31857 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:33.111642 31573 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:33.112941 31573 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:33.117197 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:33.109836 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.098104 26328 rpcz_store.cc:276] 0924 14:31:31.709004 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.730381 (+ 21377us) service_pool.cc:227] Handling call
0924 14:31:31.730712 (+   331us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:31.730928 (+   216us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:32.130570 (+399642us) log.cc:842] Serialized 14 byte log entry
0924 14:31:32.795946 (+665376us) log.cc:842] Serialized 14 byte log entry
0924 14:31:33.097015 (+301069us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b180037cfc0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09b6261990 00007f09b6423864 00007f09b6423155 00007f09b93463d0 00007f09b9358576 00007f09b93690d7 00007f09b9369058
0924 14:31:33.097227 (+   212us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:33.097285 (+    58us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:33.097372 (+    87us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:33.097443 (+    71us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:33.097575 (+   132us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":99063,"prepare.queue_time_us":718733,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":88,"raft.queue_time_us":303371,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":81861888,"thread_start_us":1030136,"threads_started":2,"wal-append.queue_time_us":917659}
W20260924 14:31:33.128084 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 72042.8us
I20260924 14:31:33.082553 26077 catalog_manager.cc:5495] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 13, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:31:33.112085 31850 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260924 14:31:33.101775 31781 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:33.098564 31843 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:33.151712 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:33.138433 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:33.096365 31844 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:33.047258 31733 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:33.163980 31623 consensus_queue.cc:1046] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260924 14:31:33.136075 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:33.106913 31831 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:33.115610 31809 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:33.160094 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:33.101699 31845 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:33.169139 31733 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:33.143074 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:33.142232 31266 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:33.170526 31431 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:33.082666 31688 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:33.115144 31346 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260924 14:31:33.137050 31541 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:33.142472 31861 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260924 14:31:33.180203 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.255296 27191 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:33.181082 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.182665 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.161902 31781 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:33.256565 27190 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:33.183880 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.257259 26295 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.181996 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.120249 31811 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:33.155112 26406 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:33.122241 31830 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:33.164309 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:33.188975 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:33.228904 31850 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260924 14:31:33.193161 26256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1917) took 1210 ms. Trace:
I20260924 14:31:33.271896 27185 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:33.271116 26256 rpcz_store.cc:276] 0924 14:31:31.982303 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:32.048080 (+ 65777us) service_pool.cc:227] Handling call
0924 14:31:32.054465 (+  6385us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:32.605729 (+551264us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:32.605758 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:32.643815 (+ 38057us) log.cc:842] Serialized 26 byte log entry
0924 14:31:33.090928 (+447113us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:33.091165 (+   237us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:33.091550 (+   385us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:33.192917 (+101367us) raft_consensus.cc:1671] finished
0924 14:31:33.192940 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:33.193095 (+   155us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":16419,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":51,"thread_start_us":893872,"threads_started":2,"wal-append.queue_time_us":454676}
I20260924 14:31:33.193836 26077 catalog_manager.cc:5495] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 10, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 10 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:33.262921 26324 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:33.196877 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:33.273659 26401 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:33.198371 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:33.191848 31623 consensus_queue.cc:1225] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260924 14:31:33.191821 31849 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260924 14:31:33.201776 26400 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 12: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:33.187160 26399 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:33.202346 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.288349 27188 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:33.203208 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:33.168936 31833 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:33.206317 26398 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 7 FOLLOWER]: Advancing to term 9
I20260924 14:31:33.169895 31852 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:33.212957 26252 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:31:33.144119 31846 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260924 14:31:33.185128 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.261014 26295 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:33.192745 26397 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:33.265645 27194 raft_consensus.cc:2354] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15.
I20260924 14:31:33.224586 31392 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:33.311230 31392 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:33.313870 31392 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:33.280184 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:33.314641 26406 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:33.314857 27191 raft_consensus.cc:2354] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15.
I20260924 14:31:33.248868 31519 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:33.288810 26399 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260924 14:31:33.156116 31573 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.292605 26077 catalog_manager.cc:5495] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 3 to 5, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 5 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:33.323951 26077 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 12, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 12 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:33.297216 26252 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:33.302394 26295 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.327898 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.199810 31279 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:33.301548 27192 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:33.278913 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:33.332710 27190 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:33.319883 26398 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:31:33.293079 31454 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:33.337761 26077 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 11, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:31:33.306180 31866 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260924 14:31:33.282375 31856 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:33.337670 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 144639us
I20260924 14:31:33.317929 31541 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:33.275264 31562 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260924 14:31:33.344127 26224 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:33.351930 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.352553 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:33.358032 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.360702 26077 catalog_manager.cc:5495] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:33.362910 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:33.357918 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.364267 27191 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:33.349121 26401 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
W20260924 14:31:33.351776 26371 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.857s (SENT)
I20260924 14:31:33.360095 27192 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:33.367597 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.366343 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.333067 31451 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:33.373268 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1301) took 1467 ms. Trace:
W20260924 14:31:33.373028 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 272087us
I20260924 14:31:33.376724 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.333602 31329 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:33.372356 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.338537 31573 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:33.384840 26224 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:33.388240 26298 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:33.392815 26221 connection.cc:232] RPC call timeout handler was delayed by -1.0486156260003554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
I20260924 14:31:33.394044 26252 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:33.366811 31541 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:33.373847 31833 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:33.380853 31454 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:33.378183 26253 rpcz_store.cc:276] 0924 14:31:31.906211 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:32.027234 (+121023us) service_pool.cc:227] Handling call
0924 14:31:32.028482 (+  1248us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:32.047561 (+ 19079us) raft_consensus.cc:1520] Early marking committed up to index 7
0924 14:31:32.108811 (+ 61250us) log.cc:842] Serialized 14 byte log entry
0924 14:31:32.747932 (+639121us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:32.759392 (+ 11460us) log.cc:842] Serialized 26 byte log entry
0924 14:31:32.760092 (+   700us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:32.760146 (+    54us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:32.760244 (+    98us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:33.373081 (+612837us) raft_consensus.cc:1671] finished
0924 14:31:33.373102 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:33.373216 (+   114us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":41565,"prepare.queue_time_us":69900,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":9315,"raft.queue_time_us":42201,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":13225,"thread_start_us":590697,"threads_started":1,"wal-append.queue_time_us":845297}
I20260924 14:31:33.402401 26224 leader_election.cc:398] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14.
I20260924 14:31:33.376660 31818 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:33.383718 31335 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:33.411381 31335 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:33.411957 31335 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:33.413245 31869 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260924 14:31:33.405256 31855 log.cc:924] Time spent T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260924 14:31:33.415897 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 64201.1us
I20260924 14:31:33.408578 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1916) took 1426 ms. Trace:
I20260924 14:31:33.417269 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1764) took 1133 ms. Trace:
I20260924 14:31:33.329202 31519 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:33.409632 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:33.420331 26323 raft_consensus.cc:2354] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14.
W20260924 14:31:33.410969 31870 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260924 14:31:33.389137 31808 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:33.395601 31783 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:33.417836 26403 rpcz_store.cc:276] 0924 14:31:32.284236 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:32.329550 (+ 45314us) service_pool.cc:227] Handling call
0924 14:31:32.329895 (+   345us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:32.371280 (+ 41385us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:32.371312 (+    32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:32.374078 (+  2766us) log.cc:842] Serialized 26 byte log entry
0924 14:31:32.455999 (+ 81921us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4800088d58. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:31:32.967669 (+511670us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:32.967710 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:32.967781 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:33.416877 (+449096us) raft_consensus.cc:1671] finished
0924 14:31:33.416907 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:33.417191 (+   284us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20437,"raft.queue_time_us":122,"raft.run_cpu_time_us":3049,"raft.run_wall_time_us":125768,"spinlock_wait_cycles":24430976,"thread_start_us":588682,"threads_started":1,"wal-append.queue_time_us":749042}
I20260924 14:31:33.403028 31454 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:33.408015 26224 leader_election.cc:401] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260924 14:31:33.395201 31830 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:33.402652 31833 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:33.417672 26258 rpcz_store.cc:276] 0924 14:31:31.982018 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:32.037492 (+ 55474us) service_pool.cc:227] Handling call
0924 14:31:32.037831 (+   339us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:32.606078 (+568247us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:32.606125 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:32.613911 (+  7786us) log.cc:842] Serialized 26 byte log entry
0924 14:31:33.051010 (+437099us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:33.051060 (+    50us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:33.051133 (+    73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:33.408360 (+357227us) raft_consensus.cc:1671] finished
0924 14:31:33.408382 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:33.408518 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1427,"raft.queue_time_us":32932,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":946345,"threads_started":2,"wal-append.queue_time_us":537933}
W20260924 14:31:33.472134 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 191337us
I20260924 14:31:33.397763 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:33.474853 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 77182.7us
I20260924 14:31:33.475875 26221 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.478922 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.425995 26327 raft_consensus.cc:3032] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:33.426172 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:33.479838 26327 raft_consensus.cc:729] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:33.402051 31541 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:33.429179 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:33.479842 26258 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:31:33.429844 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:33.441098 31872 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260924 14:31:33.445340 31781 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:33.449522 31856 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:33.420408 31329 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:31:33.454232 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1795) took 2329 ms. Trace:
I20260924 14:31:33.495486 26404 rpcz_store.cc:276] 0924 14:31:31.125375 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.974832 (+849457us) service_pool.cc:227] Handling call
0924 14:31:32.997405 (+1022573us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:33.006333 (+  8928us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:33.454002 (+447669us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002fcbd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:33.454160 (+   158us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2350719744}
I20260924 14:31:33.455583 26331 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Stepping down as leader of term 15
W20260924 14:31:33.432163 31720 consensus_meta.cc:287] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.655s	user 0.013s	sys 0.000s
I20260924 14:31:33.471812 31830 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:33.426891 31451 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:33.457336 31454 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:33.471894 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 69675us
I20260924 14:31:33.473035 27190 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:33.483551 26295 leader_election.cc:398] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15.
I20260924 14:31:33.398036 31811 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:33.506700 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.485773 26221 leader_election.cc:398] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14.
I20260924 14:31:33.484513 26324 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:31:33.492070 26327 consensus_queue.cc:260] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:33.507879 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.496621 26331 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:33.482370 31833 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:33.482367 31828 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:33.488076 31875 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:33.501346 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.430194 31818 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260924 14:31:33.506986 26295 leader_election.cc:398] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15.
I20260924 14:31:33.519757 26295 leader_election.cc:401] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260924 14:31:33.508476 26256 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:33.508558 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:33.509450 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:33.511337 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:33.511920 26253 raft_consensus.cc:1226] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:33.491478 31844 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election won for term 18
W20260924 14:31:33.509582 31721 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.024s	user 0.014s	sys 0.000s
I20260924 14:31:33.516204 27194 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:33.517987 31783 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:33.518813 31808 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:33.531900 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:33.522552 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:33.523218 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 130495us
I20260924 14:31:33.524859 26404 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:33.513367 27186 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:33.521765 27188 raft_consensus.cc:2362] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14. Re-sending same reply.
I20260924 14:31:33.504208 31830 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:33.535286 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.515833 26331 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:33.536901 26331 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:33.514168 31329 raft_consensus.cc:686] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:33.521248 31875 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:33.531230 31856 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15.
I20260924 14:31:33.505465 31720 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:33.560511 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.561735 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:33.509796 31451 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:33.531174 31844 raft_consensus.cc:686] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:33.544761 31735 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:33.515077 31811 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.580030 27181 raft_consensus.cc:2354] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 18. Current term is 19.
I20260924 14:31:33.575110 26256 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:33.576640 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1155) took 1033 ms. Trace:
W20260924 14:31:33.581029 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 69790.8us
I20260924 14:31:33.581408 26331 rpcz_store.cc:276] 0924 14:31:32.543728 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:32.701259 (+157531us) service_pool.cc:227] Handling call
0924 14:31:32.742518 (+ 41259us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:33.455465 (+712947us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54003ad7a0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92de730 00007f09b9350300 00007f09b9351371 00007f09b934dedf 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc
0924 14:31:33.576178 (+120713us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:33.576229 (+    51us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:33.576290 (+    61us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:33.576334 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:33.576421 (+    87us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:33.576568 (+   147us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":844,"raft.queue_time_us":968,"spinlock_wait_cycles":510199040}
I20260924 14:31:33.515697 31843 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260924 14:31:33.529750 31721 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:33.560488 31875 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:33.544437 31783 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.589421 27125 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:33.574623 31885 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:33.587328 31678 consensus_queue.cc:1046] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260924 14:31:33.592094 31885 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:33.592748 31885 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:33.548252 31808 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:33.597869 31808 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:33.598766 31833 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.599839 31721 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:33.600804 31451 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:33.600891 31833 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:33.569782 31547 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:33.601619 31547 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:33.602016 31547 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:33.604040 31519 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:33.605990 31454 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.608246 31454 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:33.608919 31454 consensus_queue.cc:1046] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:33.610183 31454 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:33.610347 31809 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:33.611119 31809 raft_consensus.cc:3032] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:33.611526 31809 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:33.612185 31783 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:33.612468 31809 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:33.613690 31844 consensus_queue.cc:237] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:33.614099 31809 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:33.615499 31445 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
W20260924 14:31:33.616487 31889 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260924 14:31:33.614013 31811 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:33.586529 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 76646.5us
I20260924 14:31:33.578572 31818 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:33.586292 31735 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:31:33.640872 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 80509.4us
I20260924 14:31:33.647001 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.650710 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.651350 26254 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:33.653239 26253 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260924 14:31:33.652246 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.632865 31368 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:33.657213 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.654588 26369 leader_election.cc:398] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9.
I20260924 14:31:33.658725 26369 leader_election.cc:401] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:33.659307 26251 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:33.635667 31174 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:33.670964 26255 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:33.671094 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.647666 31329 consensus_queue.cc:237] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.667754 31783 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9.
I20260924 14:31:33.611330 31541 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:33.669253 31888 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:33.648792 31818 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:33.675256 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.694427 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.661345 26369 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:33.695269 26257 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:33.679953 31174 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:33.681556 31335 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.684042 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.703626 26251 raft_consensus.cc:2354] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:33.685312 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.620422 31830 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.692595 26328 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:33.669066 31720 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:33.697049 31783 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:33.696523 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:33.697535 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.698490 31541 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:33.699249 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.700110 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.696357 26252 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:33.719803 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:33.675670 31368 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:33.723244 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1152) took 1434 ms. Trace:
I20260924 14:31:33.736459 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.737027 26330 rpcz_store.cc:276] 0924 14:31:32.289062 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:32.297624 (+  8562us) service_pool.cc:227] Handling call
0924 14:31:32.777433 (+479809us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:32.778275 (+   842us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:33.723016 (+944741us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54002774d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:33.723181 (+   165us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1087700736}
I20260924 14:31:33.737748 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.738961 26251 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:33.691951 31888 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:31:33.718618 31699 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.750s	user 0.000s	sys 0.017s
I20260924 14:31:33.752650 31895 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:33.753180 31895 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:33.753585 31895 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:33.753860 31335 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:33.756235 31174 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:33.756915 31894 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:33.757778 31894 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:33.723034 31547 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:33.740653 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44207.2us
I20260924 14:31:33.761348 26369 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:33.744812 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.762126 26255 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 8 FOLLOWER]: Advancing to term 9
I20260924 14:31:33.762617 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.763078 26250 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:33.745316 31818 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:33.764067 31368 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:33.745991 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.748152 31888 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.751791 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.783435 26254 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:33.761435 31547 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:33.742204 31735 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:33.765838 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.792198 26250 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:33.750424 31830 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:33.794047 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:33.770368 26257 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:33.767503 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.772806 26255 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:31:33.801618 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.802155 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.781141 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.735881 31783 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:33.789690 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:33.794445 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:33.805807 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.797870 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.807076 26250 raft_consensus.cc:2396] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260924 14:31:33.800379 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:33.808250 26257 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:33.784883 31721 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:33.812942 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:33.813529 26330 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:33.803678 26252 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:31:33.799635 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 233489us
I20260924 14:31:33.805928 26331 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:33.796186 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:33.816246 26324 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:33.816387 26255 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:33.775098 27182 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:33.810527 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:33.800061 31888 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:33.823891 26257 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
W20260924 14:31:33.817330 31696 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.032s	sys 0.007s
I20260924 14:31:33.810482 31783 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:33.857350 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 59525.9us
I20260924 14:31:33.858421 26221 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:33.860244 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 298546us
I20260924 14:31:33.784447 31885 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260924 14:31:33.824790 31844 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:33.867601 31907 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:33.853029 31696 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.033s	sys 0.007s
I20260924 14:31:33.819361 31541 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:33.868010 26330 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:33.862811 26371 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.860433 31721 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:33.880523 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:33.881100 31887 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:33.882073 31834 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:33.882517 31834 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:33.883436 31834 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.884317 31894 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:33.890020 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 77750.9us
I20260924 14:31:33.892220 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:33.903290 26369 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:33.878290 31844 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:33.904510 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:33.906533 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1295) took 2002 ms. Trace:
I20260924 14:31:33.909139 26249 rpcz_store.cc:276] 0924 14:31:31.904443 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:31.905741 (+  1298us) service_pool.cc:227] Handling call
0924 14:31:31.906115 (+   374us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:33.061007 (+1154892us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:33.906184 (+845177us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000c26d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:33.906453 (+   269us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2651366144}
I20260924 14:31:33.916177 26330 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:33.918079 27186 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:33.916699 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:33.879845 31521 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260924 14:31:33.887396 31541 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:33.914561 31892 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election won for term 14
W20260924 14:31:33.935261 31898 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260924 14:31:33.936095 31721 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.962561 31892 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:33.963618 31892 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:33.938040 31875 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:33.940110 31904 consensus_queue.cc:1046] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260924 14:31:33.966305 31904 raft_consensus.cc:3032] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:33.966665 31904 raft_consensus.cc:729] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:33.940711 31808 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:33.915782 31906 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:33.945502 31908 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:33.946780 27194 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "64e0b4058f18400dab940d3832586746"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:33.933761 31834 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:33.922679 26221 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:33.952739 26254 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:33.973806 31844 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:33.974591 31844 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:33.975749 31844 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:33.981381 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 58798.4us
I20260924 14:31:33.921209 31818 raft_consensus.cc:2741] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260924 14:31:33.983916 31809 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:33.986495 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:33.987635 31809 raft_consensus.cc:2741] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260924 14:31:33.994673 26256 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
W20260924 14:31:33.997174 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 89379.1us
I20260924 14:31:33.998469 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:33.972805 31828 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14.
I20260924 14:31:33.993553 31893 raft_consensus.cc:2741] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260924 14:31:34.007496 27194 raft_consensus.cc:595] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:34.002272 31908 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:34.010111 31875 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:33.996448 31904 consensus_queue.cc:260] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:33.993029 31808 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:34.012099 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:34.019387 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:34.019671 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 156970us
I20260924 14:31:34.027855 31907 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:34.028353 31904 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:34.001120 31906 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:34.029999 26330 raft_consensus.cc:2354] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16.
I20260924 14:31:34.021607 31895 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.004248 31809 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260924 14:31:34.024091 31908 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:34.031703 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:34.031751 26221 leader_election.cc:398] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16.
I20260924 14:31:34.052480 26253 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:34.054875 26221 leader_election.cc:401] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260924 14:31:34.037340 31623 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:34.061277 26371 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:34.040824 31896 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260924 14:31:34.058908 31907 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:34.061334 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:34.037003 31851 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.950s	user 0.002s	sys 0.003s
I20260924 14:31:34.062248 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:34.063661 31623 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:34.054376 31818 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:34.064018 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:34.043043 31906 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.036983 31541 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:34.049693 31808 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:34.065544 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:34.067899 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:34.081001 27188 raft_consensus.cc:3032] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:34.081398 26402 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:34.081974 26405 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:34.075598 26403 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:34.068475 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:34.069686 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:34.070252 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:34.068547 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 274567us
I20260924 14:31:34.084025 31039 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:34.084427 31039 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:34.084772 31039 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:34.085671 31688 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:34.081502 27188 raft_consensus.cc:729] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:34.085947 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:34.066502 31895 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:34.087289 31887 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:34.082863 27189 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 21.
I20260924 14:31:34.086979 27188 consensus_queue.cc:260] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:34.089677 31909 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.153s	user 0.000s	sys 0.002s
I20260924 14:31:34.091471 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:34.091779 31887 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:34.092176 26406 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:34.092814 31895 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:34.093226 31895 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:31:34.093554 31903 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.154s	user 0.002s	sys 0.000s
I20260924 14:31:34.094059 31895 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.095758 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:34.057677 31844 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:34.096820 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:34.096833 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:34.100852 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:34.110060 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:34.120530 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:34.121526 26403 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:34.123227 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:34.123374 26404 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:34.123833 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:34.120081 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:34.124967 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:34.125001 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:34.042362 31783 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.127954 26254 raft_consensus.cc:2354] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9.
I20260924 14:31:34.072007 31721 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader election won for term 9
I20260924 14:31:34.125730 26403 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:34.074662 31885 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:34.127049 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:34.127161 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:34.124058 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:34.120061 31623 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:34.132117 31918 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:31:34.132962 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:34.133489 31174 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:34.134554 31039 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:34.127099 31266 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:34.121915 31911 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:34.112706 31888 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:34.139784 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:34.098572 31818 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:34.144862 31912 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260924 14:31:34.177168 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52300.2us
I20260924 14:31:34.145983 26399 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:34.145807 31731 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:34.147696 31720 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:31:34.148157 26404 raft_consensus.cc:1226] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: []
W20260924 14:31:34.150148 31919 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260924 14:31:34.153118 26251 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:34.156584 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:34.157859 26371 leader_election.cc:398] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 8. Current term is 9.
I20260924 14:31:34.190837 26406 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:34.186267 27186 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
W20260924 14:31:34.175894 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 54902.1us
I20260924 14:31:34.162218 31809 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:34.145385 31783 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:34.176466 26403 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:34.083195 31907 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:34.184715 27194 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:34.200621 27194 raft_consensus.cc:595] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:34.201005 27194 raft_consensus.cc:603] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:34.177873 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:34.185802 27191 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:31:34.172894 26402 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:34.194053 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:34.202584 26224 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:34.175715 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:34.177608 31912 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:34.179157 31911 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:34.205691 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:34.205716 26256 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:34.177807 31818 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:34.206647 31912 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:31:34.214480 31911 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:34.195624 31888 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:34.215328 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:34.227298 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:34.211252 31890 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260924 14:31:34.208436 31892 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:34.230645 31908 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:34.231417 31907 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:34.231717 31908 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:34.233223 31908 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:34.211002 31828 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:31:34.211583 31720 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:34.236936 31720 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:34.238971 31875 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.239744 31875 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.240326 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:34.241261 26251 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:34.245674 31828 raft_consensus.cc:686] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:34.246758 31828 consensus_queue.cc:237] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:34.250409 31890 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:34.250917 31890 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:34.251884 31890 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.254751 31279 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.256098 31844 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16.
I20260924 14:31:34.257197 26249 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:34.257017 31875 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:34.260450 31039 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:34.261003 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:34.262051 26403 raft_consensus.cc:2354] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12.
I20260924 14:31:34.262362 31912 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:34.208053 31174 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:34.263607 26397 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260924 14:31:34.264320 26406 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260924 14:31:34.259141 31781 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260924 14:31:34.265798 31781 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:34.266186 26252 raft_consensus.cc:1226] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:34.266307 31781 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:31:34.179008 31266 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:34.268218 31912 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:34.269217 31266 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:34.270089 27194 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:34.270870 27194 raft_consensus.cc:595] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:34.264418 31174 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:34.278877 31894 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:34.279526 31894 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:34.280634 31894 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.281986 31894 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:34.286340 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:34.287214 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 60768us
I20260924 14:31:34.291428 26256 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:31:34.294759 26297 connection.cc:232] RPC call timeout handler was delayed by -1.1360525580003014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
W20260924 14:31:34.295469 26297 connection.cc:232] RPC call timeout handler was delayed by -1.0964525580002373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21070000000000003s for a response
I20260924 14:31:34.300612 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:34.302351 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:34.351526 27126 leader_election.cc:398] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12.
I20260924 14:31:34.351951 27126 leader_election.cc:401] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260924 14:31:34.215606 31731 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:34.350500 26251 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
W20260924 14:31:34.352304 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 55718.3us
I20260924 14:31:34.354422 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:34.358501 27191 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:34.354518 26328 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260924 14:31:34.354820 27194 raft_consensus.cc:1261] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260924 14:31:34.360373 27194 pending_rounds.cc:76] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 12
I20260924 14:31:34.360723 27194 pending_rounds.cc:97] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260924 14:31:34.296500 31541 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.356138 31562 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.359287 27191 raft_consensus.cc:595] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:34.370801 27191 raft_consensus.cc:603] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:34.360122 26324 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:34.361068 27194 raft_consensus.cc:2866] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:34.355213 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:34.372157 26256 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 13
I20260924 14:31:34.356032 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:34.362398 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 63812us
I20260924 14:31:34.374691 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:34.363299 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:34.296711 31266 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.356750 31731 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:34.379434 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:34.411870 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:34.418341 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47951.5us
I20260924 14:31:34.423959 31858 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:34.424347 31875 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.424978 31875 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:34.426121 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 905142us
W20260924 14:31:34.428028 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 139800us
I20260924 14:31:34.419932 26256 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:34.364142 31767 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:34.432511 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:34.432787 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:34.436738 27191 raft_consensus.cc:3037] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:34.433530 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:34.437510 26404 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260924 14:31:34.437981 26404 pending_rounds.cc:76] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 7
I20260924 14:31:34.438449 26404 pending_rounds.cc:97] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260924 14:31:34.438752 26404 raft_consensus.cc:2866] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:34.437839 26406 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:34.381119 31834 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.449358 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:34.367027 31368 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260924 14:31:34.441732 31547 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12.
I20260924 14:31:34.452412 31678 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260924 14:31:34.453118 31897 log.cc:924] Time spent T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.570s	user 0.000s	sys 0.003s
W20260924 14:31:34.454281 31917 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260924 14:31:34.297268 31279 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:34.454962 27125 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:34.451504 31875 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:34.456671 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:34.459715 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:34.466171 26331 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:34.466637 27188 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:34.467206 26406 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:34.468559 26406 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:31:34.377684 31541 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:34.473318 31541 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:34.473932 31541 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:34.474432 31541 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:34.473361 31279 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:34.430613 31688 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260924 14:31:34.481046 26327 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:34.492517 31541 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:34.494168 31541 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 14:31:34.427042 31346 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:34.495798 31346 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:34.497021 31346 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.376715 31828 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.074s
I20260924 14:31:34.499437 31733 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260924 14:31:34.427194 31266 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:34.426916 31645 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:34.456020 26255 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:34.504740 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 1481) took 1326 ms. Trace:
I20260924 14:31:34.505087 27188 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:34.505208 27181 rpcz_store.cc:276] 0924 14:31:33.178521 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:33.181654 (+  3133us) service_pool.cc:227] Handling call
0924 14:31:34.463817 (+1282163us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b540000f354. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:34.504656 (+ 40839us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":739630976}
I20260924 14:31:34.508430 26327 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260924 14:31:34.515996 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1151) took 2228 ms. Trace:
I20260924 14:31:34.522732 27190 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:34.523597 27190 raft_consensus.cc:595] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:34.381508 31562 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:34.521654 26327 rpcz_store.cc:276] 0924 14:31:32.287842 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:32.287994 (+   152us) service_pool.cc:227] Handling call
0924 14:31:32.288409 (+   415us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:33.425876 (+1137467us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400409320. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92de730 00007f09b9350300 00007f09b9351371 00007f09b934dedf 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc
0924 14:31:34.463401 (+1037525us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b5800a94cb0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92c9f63 00007f09b932daba 00007f09b93453af 00007f09b9346b4e 00007f09b9350c89 00007f09b9351556 00007f09b934dedf 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684
0924 14:31:34.513631 (+ 50230us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:34.515921 (+  2290us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1149374976}
I20260924 14:31:34.520576 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:34.460367 31852 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:34.529402 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:34.532232 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 77344.1us
I20260924 14:31:34.536995 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:34.551402 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 357383us
I20260924 14:31:34.553618 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:31:34.556399 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:31:34.557374 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:34.571635 27191 raft_consensus.cc:2454] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
W20260924 14:31:34.522240 31121 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260924 14:31:34.575078 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:34.575724 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 55246.5us
I20260924 14:31:34.575944 27188 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:34.577446 26224 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:34.511890 31645 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:34.584108 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:34.522595 31733 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:34.587247 26252 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:34.596758 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:34.531785 31915 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:34.462752 31834 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:34.617803 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:34.618714 26399 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:34.628041 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50675.8us
I20260924 14:31:34.582965 31678 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:34.450335 31888 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.583863 31828 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:34.530633 31845 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260924 14:31:34.517529 31892 consensus_queue.cc:1035] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.075s
W20260924 14:31:34.584025 31735 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.022s	sys 0.015s
I20260924 14:31:34.643004 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:34.583379 31645 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:34.584226 31721 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:34.663818 26399 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:34.665648 26399 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:34.667749 27188 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260924 14:31:34.615061 31454 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:34.583410 31852 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
W20260924 14:31:34.669035 31735 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.023s	sys 0.015s
I20260924 14:31:34.648950 31828 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:34.699649 27123 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.503s (SENT)
I20260924 14:31:34.700513 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:34.666765 31892 consensus_queue.cc:1225] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260924 14:31:34.616184 31733 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.580173 31731 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:34.709693 27123 leader_election.cc:334] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.559s (SENT)
I20260924 14:31:34.710994 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:34.711225 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 299404us
I20260924 14:31:34.718335 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:34.721592 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:34.734576 26297 leader_election.cc:398] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:34.746742 26297 leader_election.cc:401] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260924 14:31:34.693090 31863 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.691557 31454 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:34.693630 31887 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:34.697005 31828 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:34.875492 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 308618us
I20260924 14:31:34.781412 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:34.861231 31781 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.594s	user 0.007s	sys 0.005s
W20260924 14:31:34.779983 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 94065.4us
I20260924 14:31:34.778460 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:34.808888 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 99240.6us
W20260924 14:31:34.821213 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 89145.9us
W20260924 14:31:34.762678 31919 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260924 14:31:34.741834 31844 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.741343 31677 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260924 14:31:34.877803 31818 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.008s
I20260924 14:31:34.853513 31733 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:34.728183 31266 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:34.915935 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:34.808588 31941 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13.
I20260924 14:31:34.741128 31678 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:34.849354 31830 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.861537 31562 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:34.912493 26297 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:34.854529 31895 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:34.876214 31346 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:34.900048 31174 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.025s	sys 0.017s
W20260924 14:31:35.023110 31174 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.025s	sys 0.018s
I20260924 14:31:34.859081 31121 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.024214 31121 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:34.955876 27126 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:35.028098 31121 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:34.763763 31852 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 21 LEADER]: Stepping down as leader of term 21
I20260924 14:31:35.034864 31852 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:35.036558 31852 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:34.820379 31368 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:34.764503 31735 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:34.979815 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:35.040201 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:34.716369 31519 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.232s
W20260924 14:31:34.871743 31930 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
W20260924 14:31:34.981705 26295 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.027s	sys 0.038s
W20260924 14:31:35.042469 26295 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.027s	sys 0.039s
I20260924 14:31:35.000888 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:34.918166 31834 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:34.955690 31934 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:34.874421 31863 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:35.021265 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 108864us
I20260924 14:31:35.054935 26297 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:34.975304 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 59256us
I20260924 14:31:35.056311 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:35.033162 31346 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:34.893222 31454 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:35.051620 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 96773.4us
W20260924 14:31:35.074710 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 445412us
I20260924 14:31:34.920590 31781 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:34.898120 31887 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:35.140571 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 85735.1us
I20260924 14:31:34.979827 31818 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:34.770998 31721 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:35.154177 26332 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:35.145143 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.048491 31852 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:31:35.050184 31941 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260924 14:31:35.143250 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 57873.8us
W20260924 14:31:35.146219 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 84096.3us
I20260924 14:31:35.157531 26295 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:35.062196 31915 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:35.148504 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.150688 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.218421 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:35.152216 26329 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:31:35.153260 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.066932 31368 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:35.158917 27189 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260924 14:31:35.153574 31606 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260924 14:31:35.085453 31121 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.164337 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.237254 26330 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:35.165109 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.174187 31767 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:35.175772 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:35.188959 31645 consensus_meta.cc:287] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.002s
I20260924 14:31:35.080412 31562 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:35.075663 31934 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:35.017407 31830 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:35.083249 31335 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:35.192052 31928 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.333s	user 0.002s	sys 0.001s
I20260924 14:31:35.192308 26405 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260924 14:31:35.082540 31174 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
W20260924 14:31:35.288793 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 379427us
W20260924 14:31:35.197878 31936 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.338s	user 0.000s	sys 0.001s
I20260924 14:31:35.199329 31623 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:35.200883 31882 raft_consensus.cc:3032] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:35.242995 31767 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:34.980346 31844 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:35.208470 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.332114 26328 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
W20260924 14:31:35.220909 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 63509.4us
I20260924 14:31:35.333801 26295 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:35.011018 31931 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.144s
I20260924 14:31:35.140841 31834 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:35.074384 31925 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.008s	sys 0.012s
I20260924 14:31:35.146867 31733 raft_consensus.cc:2781] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:35.234179 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:35.235980 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.195295 31890 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:35.249636 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.016s	sys 0.006s
I20260924 14:31:35.260197 31781 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 22 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:35.247313 31121 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:35.235567 31857 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:35.264288 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.410301 26330 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:35.252163 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:35.278508 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:35.199406 31346 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:35.231408 31858 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:35.228608 31846 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:35.374223 31875 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:35.080833 31929 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.180s
I20260924 14:31:35.307806 26252 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:35.308459 26400 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:35.302893 26404 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:31:35.336313 26295 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:35.158912 31735 consensus_queue.cc:1046] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.098s
I20260924 14:31:35.279140 31519 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.348439 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:35.392122 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.016s	sys 0.006s
W20260924 14:31:35.379218 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 145147us
I20260924 14:31:35.251219 31645 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:35.309857 31882 raft_consensus.cc:729] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:35.399286 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:35.348047 31894 consensus_meta.cc:287] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.066s	user 0.011s	sys 0.006s
I20260924 14:31:35.402922 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1568) took 1281 ms. Trace:
I20260924 14:31:35.255376 31562 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:35.373029 31925 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.008s	sys 0.014s
I20260924 14:31:35.364863 31845 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.412377 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:35.439572 31541 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:35.452762 31933 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.534198 26405 rpcz_store.cc:276] 0924 14:31:34.121098 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.121216 (+   118us) service_pool.cc:227] Handling call
0924 14:31:34.121539 (+   323us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:35.286943 (+1165404us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:35.402627 (+115684us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54004f0e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:35.402856 (+   229us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2461934848}
I20260924 14:31:35.349126 31945 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:35.436614 26256 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260924 14:31:35.227351 31678 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:35.192062 31818 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:35.307726 31335 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:35.398418 31921 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:35.448707 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.408454 31733 raft_consensus.cc:686] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:31:35.474963 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 138739us
I20260924 14:31:35.579679 26330 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:35.479110 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.373386 31834 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:35.534256 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 55985.9us
I20260924 14:31:35.537712 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:35.537638 26329 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 19
W20260924 14:31:35.466333 31944 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.505s	user 0.000s	sys 0.001s
I20260924 14:31:35.524067 31895 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
I20260924 14:31:35.554163 26397 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260924 14:31:35.433970 31857 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:35.581626 26295 leader_election.cc:398] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 18. Current term is 19.
I20260924 14:31:35.483560 31519 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:35.592329 31950 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260924 14:31:35.576187 31943 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.458s	user 0.001s	sys 0.000s
I20260924 14:31:35.593430 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:35.594048 26399 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.036s	sys 0.025s
W20260924 14:31:35.638597 26399 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.036s	sys 0.025s
I20260924 14:31:35.448751 31846 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:35.599850 26326 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:35.553692 31845 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:35.602716 26329 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:35.471357 31454 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:35.588296 31947 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.429s	user 0.000s	sys 0.001s
I20260924 14:31:35.605988 26327 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:35.580726 31894 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.613375 26295 leader_election.cc:401] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:35.541424 31941 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:35.614229 26397 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:35.615123 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:35.627128 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:35.459882 31858 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:35.596181 31921 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:35.639875 27123 leader_election.cc:334] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.891s (SENT)
I20260924 14:31:35.597970 31934 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:35.592365 31830 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:35.560665 31541 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:35.624626 31940 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.470s	user 0.001s	sys 0.000s
I20260924 14:31:35.598166 31887 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:35.671533 26324 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:35.709687 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:35.560899 31933 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:35.460584 31875 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:35.732673 26397 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:35.717221 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:35.765415 26328 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:35.731827 31547 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:35.735291 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 95474.1us
W20260924 14:31:35.770516 27123 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.895s (SENT)
I20260924 14:31:35.716398 26330 raft_consensus.cc:2362] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15. Re-sending same reply.
I20260924 14:31:35.738561 26254 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:35.649013 31931 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.781309 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:35.785724 26328 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:35.775174 31731 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.012s	sys 0.060s
I20260924 14:31:35.787992 31121 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:35.788502 27192 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260924 14:31:35.790010 31368 raft_consensus.cc:979] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:31:35.790295 27182 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:35.793409 26328 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260924 14:31:35.795300 26328 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260924 14:31:35.600116 31882 consensus_queue.cc:260] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:35.751485 31830 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:35.642911 31863 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:35.752921 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1562) took 1681 ms. Trace:
I20260924 14:31:35.714766 31956 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:35.717635 31454 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.766518 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:35.777128 31945 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.740744 31941 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 18. Current term is 19.
I20260924 14:31:35.777158 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:35.811033 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.812471 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:35.813831 31844 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:35.737236 31894 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:35.824446 26401 rpcz_store.cc:276] 0924 14:31:34.071182 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.071337 (+   155us) service_pool.cc:227] Handling call
0924 14:31:34.071732 (+   395us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:34.081409 (+  9677us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:34.081456 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:34.468374 (+386918us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b1800049b40. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a
0924 14:31:34.469337 (+   963us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b2c0013ed68. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:34.469858 (+   521us) log.cc:842] Serialized 26 byte log entry
0924 14:31:34.688526 (+218668us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:34.688572 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:34.688655 (+    83us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:35.752606 (+1063951us) raft_consensus.cc:1671] finished
0924 14:31:35.752639 (+    33us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:35.752845 (+   206us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":814,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":22018,"spinlock_wait_cycles":51043968,"thread_start_us":218251,"threads_started":1,"wal-append.queue_time_us":402004}
W20260924 14:31:35.794523 31731 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.012s	sys 0.061s
W20260924 14:31:35.825001 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 237818us
W20260924 14:31:35.798203 31952 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.380s	user 0.000s	sys 0.001s
I20260924 14:31:35.839601 26329 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:35.807660 31005 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260924 14:31:35.780058 31699 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:35.714479 27126 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.030s	sys 0.051s
W20260924 14:31:35.868214 27126 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.030s	sys 0.051s
W20260924 14:31:35.868655 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 708969us
I20260924 14:31:35.842065 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1161) took 1314 ms. Trace:
I20260924 14:31:35.871590 26331 rpcz_store.cc:276] 0924 14:31:34.527945 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.528078 (+   133us) service_pool.cc:227] Handling call
0924 14:31:34.570653 (+ 42575us) raft_consensus.cc:1457] Updating replica for 3 ops
0924 14:31:34.579337 (+  8684us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:34.579384 (+    47us) raft_consensus.cc:1525] Triggering prepare for 3 ops
0924 14:31:34.623919 (+ 44535us) log.cc:842] Serialized 78 byte log entry
0924 14:31:35.146629 (+522710us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:35.241361 (+ 94732us) log.cc:842] Serialized 14 byte log entry
0924 14:31:35.241705 (+   344us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:35.241944 (+   239us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:35.841776 (+599832us) raft_consensus.cc:1671] finished
0924 14:31:35.841810 (+    34us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:35.841991 (+   181us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9576,"prepare.queue_time_us":94204,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":99407,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":13852,"thread_start_us":461637,"threads_started":1,"wal-append.queue_time_us":743589}
I20260924 14:31:35.788534 31857 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:35.798990 31888 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:35.864881 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 52868.7us
I20260924 14:31:35.742230 31733 consensus_queue.cc:237] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:35.742560 31921 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:35.792950 31547 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:35.896950 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:35.897756 26401 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 9.
I20260924 14:31:35.899349 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:35.906693 27192 raft_consensus.cc:1203] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 13.10->[14.11-18.13]   Dedup: 14.11->[16.12-18.13]
I20260924 14:31:35.907310 27192 pending_rounds.cc:76] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Aborting all ops after (but not including) 11
I20260924 14:31:35.907662 27192 pending_rounds.cc:97] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260924 14:31:35.907924 27192 raft_consensus.cc:2866] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:31:35.908116 31946 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260924 14:31:35.909283 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1496) took 1477 ms. Trace:
I20260924 14:31:35.909705 27189 rpcz_store.cc:276] 0924 14:31:34.432349 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.432479 (+   130us) service_pool.cc:227] Handling call
0924 14:31:34.449251 (+ 16772us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:35.230889 (+781638us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:35.909006 (+678117us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b54000028d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:35.909203 (+   197us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1631463168}
I20260924 14:31:35.912245 31547 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:35.912708 31941 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:35.848712 31945 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:35.913213 31941 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:35.913668 31941 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:35.914777 31945 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:35.896509 31888 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:35.917368 31888 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:35.841583 31846 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:35.907047 31005 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.841408 31809 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:35.841828 31863 raft_consensus.cc:1067] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
W20260924 14:31:35.917478 31623 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.004s
I20260924 14:31:35.907310 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:35.879185 31781 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:35.924152 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:35.944147 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:35.797613 31931 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:35.846750 31454 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:35.925141 26405 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:35.883895 26329 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:35.929615 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:35.962644 26401 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:35.933138 26400 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:35.880491 31645 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:35.887521 31828 consensus_meta.cc:287] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.983s	user 0.008s	sys 0.008s
I20260924 14:31:35.842931 31830 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:35.894114 31818 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:35.971554 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 390037us
W20260924 14:31:35.947777 31957 log.cc:924] Time spent T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.342s	user 0.001s	sys 0.001s
I20260924 14:31:35.944969 26326 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:35.889403 31699 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:35.965709 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:35.993021 26397 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:35.932379 31834 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:35.991469 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.001547 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.002175 27189 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:35.965405 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.003311 27191 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:36.004379 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:35.932226 31335 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:35.944824 31846 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.947928 31781 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:35.998024 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:36.034874 26401 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 22: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:35.848384 31956 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:36.005122 26400 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:36.018790 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:35.970705 31929 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:35.843602 31858 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:35.944365 31809 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:36.036196 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.039690 27186 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
W20260924 14:31:35.885099 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 419638us
W20260924 14:31:36.040675 26253 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.001s	sys 0.012s
I20260924 14:31:36.041054 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:36.042550 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:35.932739 31005 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:36.041949 31335 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:35.991950 31519 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.002691 31845 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:36.139667 31960 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
W20260924 14:31:36.140944 31966 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260924 14:31:36.039098 31781 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:36.040333 31939 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:35.998075 31974 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:36.184298 26323 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:36.261965 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1159) took 1092 ms. Trace:
I20260924 14:31:36.023121 31834 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:36.186543 26253 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.001s	sys 0.012s
I20260924 14:31:36.043354 31893 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.196226 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:36.033517 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.039517 31888 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:36.224140 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 183191us
W20260924 14:31:36.065534 31852 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.823s	user 0.014s	sys 0.000s
I20260924 14:31:36.231451 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:35.991115 31828 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:36.229698 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.227241 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:36.238062 31431 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:36.238369 31875 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:36.050338 31541 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.240773 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.000073 31818 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:36.308696 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.245925 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:36.232877 31972 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.267s	user 0.002s	sys 0.002s
I20260924 14:31:36.243794 31975 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:36.257229 31939 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:36.045526 31174 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.190s
I20260924 14:31:36.039654 31846 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:36.185982 31623 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:36.224293 31721 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.254721 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:36.229781 31904 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:36.262506 26323 rpcz_store.cc:276] 0924 14:31:35.169914 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.594721 (+424807us) service_pool.cc:227] Handling call
0924 14:31:36.261906 (+667185us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:36.351115 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:36.351732 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 105941us
I20260924 14:31:36.352289 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:36.352527 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:36.353155 26323 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:36.235664 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.298445 26325 raft_consensus.cc:2396] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260924 14:31:36.299847 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1178) took 1113 ms. Trace:
I20260924 14:31:36.354719 26326 rpcz_store.cc:276] 0924 14:31:35.186055 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.001033 (+814978us) service_pool.cc:227] Handling call
0924 14:31:36.001383 (+   350us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:36.001594 (+   211us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:36.006188 (+  4594us) log.cc:842] Serialized 14 byte log entry
0924 14:31:36.299370 (+293182us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:36.299456 (+    86us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:36.299516 (+    60us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:36.299601 (+    85us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:36.299768 (+   167us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":164,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":5028,"raft.queue_time_us":150,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":19979,"thread_start_us":292666,"threads_started":1}
I20260924 14:31:36.357484 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:36.246995 31963 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.470s	user 0.000s	sys 0.002s
I20260924 14:31:36.286180 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.267601 31894 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.411064 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.309497 27185 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
W20260924 14:31:36.319849 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 123739us
I20260924 14:31:36.052593 31929 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:36.321753 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:36.250878 27194 raft_consensus.cc:2379] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:36.238574 31335 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:36.331669 31975 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:36.343941 31939 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:36.348546 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 121429us
W20260924 14:31:36.360021 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 105406us
I20260924 14:31:36.354138 26250 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:36.367678 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:36.412806 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 58340.2us
I20260924 14:31:36.262465 31863 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.261703 31971 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.425640 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:36.308971 31431 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:36.349217 31368 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:36.346211 31969 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:36.431114 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:36.459404 26323 raft_consensus.cc:2362] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19. Re-sending same reply.
I20260924 14:31:36.438069 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:36.339177 31875 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:36.442255 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1789) took 1759 ms. Trace:
I20260924 14:31:36.469758 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:36.469777 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:36.472534 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:36.285955 31834 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:36.468000 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1980) took 1230 ms. Trace:
I20260924 14:31:36.475576 26253 rpcz_store.cc:276] 0924 14:31:35.237319 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.237565 (+   246us) service_pool.cc:227] Handling call
0924 14:31:35.237920 (+   355us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:35.238293 (+   373us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:35.238321 (+    28us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:35.238890 (+   569us) log.cc:842] Serialized 52 byte log entry
0924 14:31:36.265248 (+1026358us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:36.265292 (+    44us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:36.265366 (+    74us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:36.458510 (+193144us) raft_consensus.cc:1671] finished
0924 14:31:36.458533 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:36.467913 (+  9380us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":5151,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":4995,"thread_start_us":970272,"threads_started":1,"wal-append.queue_time_us":1052059}
I20260924 14:31:36.370730 31541 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:36.367110 31735 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:36.421207 31645 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.300730 31888 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:36.451913 31977 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260924 14:31:36.412108 31852 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:36.429538 27126 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 1.756s (SENT) [suppressed 28 similar messages]
I20260924 14:31:36.483538 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:36.472395 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 236021us
I20260924 14:31:36.470307 26399 rpcz_store.cc:276] 0924 14:31:34.682786 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.688306 (+  5520us) service_pool.cc:227] Handling call
0924 14:31:34.735480 (+ 47174us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:34.876394 (+140914us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:34.876439 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:34.910265 (+ 33826us) log.cc:842] Serialized 26 byte log entry
0924 14:31:35.639022 (+728757us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:35.639063 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:35.639129 (+    66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:36.369241 (+730112us) raft_consensus.cc:1671] finished
0924 14:31:36.369264 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:36.425131 (+ 55867us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":159677,"raft.queue_time_us":141485,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":22951,"thread_start_us":567000,"threads_started":1,"wal-append.queue_time_us":780420}
I20260924 14:31:36.299191 31893 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:36.451301 31956 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.477998 31857 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:36.452591 31809 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.451220 31335 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.491586 31735 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:36.450690 31828 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:36.427731 31975 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:36.449535 31929 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.351146 31623 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:36.449462 31818 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.504958 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:36.464493 31921 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.511525 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.479285 31979 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:36.359087 31721 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:36.466790 31971 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:36.550376 31890 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.148s	user 0.011s	sys 0.000s
I20260924 14:31:36.550541 31623 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.550894 31892 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.551100 31818 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:36.551605 31818 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:36.551470 31892 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:36.551934 31971 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:36.552094 31818 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:36.479805 31969 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:36.513054 31852 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:36.554176 31969 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.554848 31431 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.538791 31929 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:36.555733 31834 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:36.556588 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1160) took 1386 ms. Trace:
W20260924 14:31:36.557773 31767 consensus_meta.cc:287] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.192s	user 0.011s	sys 0.000s
I20260924 14:31:36.558401 31767 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:36.562728 31941 consensus_meta.cc:287] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.649s	user 0.008s	sys 0.000s
I20260924 14:31:36.280464 31974 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:36.513643 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:36.577806 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 64233.1us
I20260924 14:31:36.579893 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:36.483654 31521 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.524150 31735 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:36.536340 31978 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260924 14:31:36.499100 31982 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:36.492388 31980 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:36.483634 31454 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:36.423800 31991 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:36.502964 31541 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:36.578706 31941 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:36.630430 31980 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:36.632457 26330 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:36.502507 31558 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.633351 31982 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:36.472433 31863 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:36.529327 31857 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260924 14:31:36.534689 31968 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260924 14:31:36.635641 31857 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:36.635741 31863 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:36.636116 31857 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:36.631134 31991 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:36.636628 27191 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:31:36.534391 31997 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:36.546428 31979 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:36.637313 31997 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:36.637715 31997 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:36.531128 27123 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:36.638821 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 107776us
I20260924 14:31:36.639060 31893 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:36.640112 31888 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:36.640319 31980 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.640849 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:36.496290 31951 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.422s	user 0.001s	sys 0.004s
I20260924 14:31:36.552484 31721 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.642926 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1834) took 2439 ms. Trace:
I20260924 14:31:36.643649 31995 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:36.643355 26402 rpcz_store.cc:276] 0924 14:31:34.203884 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.204087 (+   203us) service_pool.cc:227] Handling call
0924 14:31:34.204472 (+   385us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:34.593916 (+389444us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:34.593964 (+    48us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:34.624294 (+ 30330us) log.cc:842] Serialized 52 byte log entry
0924 14:31:35.064618 (+440324us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:35.064658 (+    40us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:35.064754 (+    96us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:36.642692 (+1577938us) raft_consensus.cc:1671] finished
0924 14:31:36.642715 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:36.642869 (+   154us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":22201,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":1861,"thread_start_us":763629,"threads_started":2,"wal-append.queue_time_us":836788}
I20260924 14:31:36.645578 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:36.646186 31995 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:36.556981 26328 rpcz_store.cc:276] 0924 14:31:35.170221 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.815937 (+645716us) service_pool.cc:227] Handling call
0924 14:31:35.816404 (+   467us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:35.818680 (+  2276us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:35.818721 (+    41us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:35.839710 (+ 20989us) log.cc:842] Serialized 26 byte log entry
0924 14:31:36.251964 (+412254us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:36.252077 (+   113us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:36.252411 (+   334us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:36.556375 (+303964us) raft_consensus.cc:1671] finished
0924 14:31:36.556397 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:36.556532 (+   135us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":142,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":102,"thread_start_us":350744,"threads_started":1,"wal-append.queue_time_us":604728}
I20260924 14:31:36.647508 31999 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.648113 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:36.648064 31999 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:36.648995 26328 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:36.650058 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:36.650406 31999 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.643179 26326 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:36.502733 31830 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:36.468447 31931 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.482762 31875 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:36.652029 31979 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.653388 31875 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:36.372319 31904 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:36.654959 26324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1158) took 1485 ms (client timeout 1503 ms). Trace:
I20260924 14:31:36.655483 31904 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.473222 31699 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:36.634809 31781 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:36.656109 26404 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1844) took 1801 ms (client timeout 2014 ms). Trace:
I20260924 14:31:36.656802 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:36.655369 26324 rpcz_store.cc:269] 0924 14:31:35.169561 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.218082 (+ 48521us) service_pool.cc:227] Handling call
0924 14:31:36.654779 (+1436697us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b540001dbd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:36.654905 (+   126us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":547623808}
I20260924 14:31:36.658022 27181 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15. Ops: []
W20260924 14:31:36.656500 26404 rpcz_store.cc:269] 0924 14:31:34.854430 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.000451 (+146021us) service_pool.cc:227] Handling call
0924 14:31:36.655927 (+1655476us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540035cfd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:36.656032 (+   105us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":596975872}
I20260924 14:31:36.651217 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:36.666383 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:36.590951 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 73680.7us
W20260924 14:31:36.612311 31987 log.cc:924] Time spent T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260924 14:31:36.669600 31781 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:36.613771 31846 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:36.608696 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.620952 31852 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:36.621755 27188 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 18: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:36.673774 27182 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:36.673830 31939 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:36.674970 26327 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:36.623714 31767 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.627887 26258 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260924 14:31:36.590363 31005 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.629048 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.684351 26251 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 9.
I20260924 14:31:36.527171 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:36.678712 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:36.633255 31558 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:36.521835 31958 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.482s	user 0.003s	sys 0.010s
I20260924 14:31:36.503986 31645 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:36.572624 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1841) took 1996 ms. Trace:
I20260924 14:31:36.714607 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:36.664824 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:36.647128 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:36.678246 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.027s	sys 0.038s
I20260924 14:31:36.588418 31521 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:36.587638 31174 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260924 14:31:36.687122 27190 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.006s	sys 0.000s
I20260924 14:31:36.688468 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:36.690440 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:36.641781 31941 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.677328 31830 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.641119 31982 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:36.714489 26398 rpcz_store.cc:276] 0924 14:31:34.575949 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.662517 (+ 86568us) service_pool.cc:227] Handling call
0924 14:31:34.687669 (+ 25152us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:34.693813 (+  6144us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:34.693872 (+    59us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:35.175936 (+482064us) spinlock_profiling.cc:243] Waited 5.93 ms on lock 0x7b18004b5bc0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a
0924 14:31:35.177302 (+  1366us) spinlock_profiling.cc:243] Waited 9.19 ms on lock 0x7b2c0013ed68. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:35.177931 (+   629us) log.cc:842] Serialized 52 byte log entry
0924 14:31:35.447305 (+269374us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:35.849631 (+402326us) log.cc:842] Serialized 14 byte log entry
0924 14:31:35.853813 (+  4182us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:35.853916 (+   103us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:36.572365 (+718449us) raft_consensus.cc:1671] finished
0924 14:31:36.572387 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:36.572545 (+   158us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31878,"prepare.queue_time_us":789564,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":12638,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":39308,"spinlock_wait_cycles":34784384,"thread_start_us":572949,"threads_started":2,"wal-append.queue_time_us":533660}
I20260924 14:31:36.716946 26402 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:31:36.717121 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.027s	sys 0.038s
I20260924 14:31:36.740476 26298 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:36.744550 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.742266 26323 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:36.747140 26404 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260924 14:31:36.747685 26404 pending_rounds.cc:76] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 8
W20260924 14:31:36.717340 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 51060.9us
I20260924 14:31:36.655107 31818 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:36.720952 27190 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.007s	sys 0.000s
W20260924 14:31:36.687901 31733 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.004s	sys 0.001s
I20260924 14:31:36.597833 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.757791 26253 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:36.723909 31174 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:36.724041 31879 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:36.759011 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.762594 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.759718 26257 raft_consensus.cc:1203] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Deduplicated request from leader. Original: 11.7->[13.8-13.8]   Dedup: 13.8->[]
I20260924 14:31:36.764019 26257 raft_consensus.cc:1226] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260924 14:31:36.764743 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.763203 26253 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:36.765724 26258 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
W20260924 14:31:36.757426 31733 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.005s	sys 0.002s
I20260924 14:31:36.775803 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.776885 26258 raft_consensus.cc:2379] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:31:36.734226 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 304690us
I20260924 14:31:36.778121 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.747963 26404 pending_rounds.cc:97] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260924 14:31:36.805375 26404 raft_consensus.cc:2866] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:36.690946 31852 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:36.755872 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.806613 26250 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:36.758383 26251 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260924 14:31:36.725836 27189 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:36.766516 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.591328 31975 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:36.772459 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:36.808226 26253 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
W20260924 14:31:36.772666 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 302980us
I20260924 14:31:36.828907 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:36.744434 31846 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.738899 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:36.799928 26402 raft_consensus.cc:2454] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:31:36.827134 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.828949 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:36.727620 31005 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260924 14:31:36.830233 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:36.840456 26369 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:36.757110 31875 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:36.826927 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1971) took 2291 ms. Trace:
I20260924 14:31:36.764065 31174 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:36.715853 27125 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:36.840260 26298 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:36.845970 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.763893 31879 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:36.852629 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.745481 31830 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:36.847597 26255 rpcz_store.cc:276] 0924 14:31:34.535970 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.542316 (+  6346us) service_pool.cc:227] Handling call
0924 14:31:34.543169 (+   853us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:34.615496 (+ 72327us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:34.615546 (+    50us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:34.870639 (+255093us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b2c001c3658. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b7d31e11 00007f09b7d41442 00007f09b9389c11 00007f09b934e58a 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:34.871147 (+   508us) log.cc:842] Serialized 26 byte log entry
0924 14:31:35.250972 (+379825us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:35.251012 (+    40us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:35.251201 (+   189us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:36.774353 (+1523152us) raft_consensus.cc:1671] finished
0924 14:31:36.774376 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:36.826839 (+ 52463us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":51339,"raft.queue_time_us":63338,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":23553,"spinlock_wait_cycles":23286144,"thread_start_us":379420,"threads_started":1,"wal-append.queue_time_us":773985}
I20260924 14:31:36.848374 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:36.849697 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.758558 32002 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:36.853484 26253 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:36.870844 26253 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260924 14:31:36.846961 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 77613.1us
I20260924 14:31:36.855985 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1853) took 1049 ms. Trace:
I20260924 14:31:36.873328 26404 rpcz_store.cc:276] 0924 14:31:35.806640 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.746244 (+939604us) service_pool.cc:227] Handling call
0924 14:31:36.746645 (+   401us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:36.855663 (+109018us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:36.855910 (+   247us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:36.874709 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:36.849258 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:36.851961 26249 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:36.861116 26250 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:36.828250 31454 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.836397 31931 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:36.827019 31888 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:36.829975 31852 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:36.851727 31988 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260924 14:31:36.854423 31879 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:36.854126 31767 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:36.893190 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:36.904157 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 55904.6us
I20260924 14:31:36.851244 31904 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260924 14:31:36.914660 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:36.920550 32006 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.202s	user 0.002s	sys 0.000s
W20260924 14:31:36.923056 26325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1672) took 1772 ms (client timeout 1833 ms). Trace:
W20260924 14:31:36.923676 26325 rpcz_store.cc:269] 0924 14:31:35.150370 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.150498 (+   128us) service_pool.cc:227] Handling call
0924 14:31:36.922756 (+1772258us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54004100d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:36.922974 (+   218us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2437336832}
W20260924 14:31:36.926461 26252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 1973) took 2145 ms (client timeout 2037 ms). Trace:
W20260924 14:31:36.926993 26252 rpcz_store.cc:269] 0924 14:31:34.781025 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.781149 (+   124us) service_pool.cc:227] Handling call
0924 14:31:36.926202 (+2145053us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b54004c9d54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:36.926336 (+   134us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":989075712}
I20260924 14:31:36.935674 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 1850) took 1178 ms. Trace:
I20260924 14:31:36.936088 26397 rpcz_store.cc:276] 0924 14:31:35.757656 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.780972 (+ 23316us) service_pool.cc:227] Handling call
0924 14:31:36.935523 (+1154551us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54002fb7d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:36.935620 (+    97us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":483423616}
I20260924 14:31:36.937611 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1175) took 1762 ms. Trace:
I20260924 14:31:36.937990 26331 rpcz_store.cc:276] 0924 14:31:35.174932 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.947087 (+772155us) service_pool.cc:227] Handling call
0924 14:31:35.994082 (+ 46995us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:35.994340 (+   258us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:35.994377 (+    37us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:35.994435 (+    58us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:35.994488 (+    53us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:35.994537 (+    49us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:36.937457 (+942920us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54005bbdd0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:36.937559 (+   102us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":57855872}
I20260924 14:31:36.941133 31720 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:36.941648 31720 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:36.942014 31720 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:36.943436 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1165) took 1771 ms. Trace:
I20260924 14:31:36.854746 31996 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:36.943885 26330 rpcz_store.cc:276] 0924 14:31:35.171707 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.867844 (+696137us) service_pool.cc:227] Handling call
0924 14:31:35.921862 (+ 54018us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:36.632842 (+710980us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:36.942749 (+309907us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54002765d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:36.943231 (+   482us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b54002765d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:36.943364 (+   133us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1677109888}
I20260924 14:31:36.925160 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:36.965445 26253 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:36.950309 26249 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:36.965616 26325 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:36.964525 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:36.912406 26250 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260924 14:31:36.971340 26250 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260924 14:31:36.899886 31995 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:36.889031 31733 raft_consensus.cc:2781] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:31:36.912158 26251 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:36.967945 26404 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:36.854929 31929 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260924 14:31:36.855085 31721 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260924 14:31:36.978547 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:36.906840 31852 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:36.953266 31990 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.422s	user 0.003s	sys 0.000s
I20260924 14:31:37.048584 26330 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:37.049460 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1162) took 1147 ms. Trace:
I20260924 14:31:37.049826 26330 rpcz_store.cc:276] 0924 14:31:35.901962 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.977176 (+1075214us) service_pool.cc:227] Handling call
0924 14:31:37.048146 (+ 70970us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.049116 (+   970us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.049397 (+   281us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:36.858039 31875 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:36.848351 31846 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:36.908965 32007 raft_consensus.cc:2741] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260924 14:31:36.986415 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1180) took 1799 ms. Trace:
I20260924 14:31:36.989341 31996 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:37.062772 31996 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:37.065099 31996 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:36.989768 31999 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:37.065846 31999 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:37.067960 31999 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.994515 31924 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260924 14:31:36.995997 32003 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260924 14:31:36.997690 31975 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:36.964635 31735 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:36.906291 31541 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.128s
I20260924 14:31:37.070451 31735 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:37.070778 31645 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:37.062402 26324 rpcz_store.cc:276] 0924 14:31:35.186850 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.657669 (+1470819us) service_pool.cc:227] Handling call
0924 14:31:36.657982 (+   313us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:36.658360 (+   378us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:36.658397 (+    37us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:36.661697 (+  3300us) log.cc:842] Serialized 26 byte log entry
0924 14:31:36.762770 (+101073us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:36.762807 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:36.762868 (+    61us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:36.986139 (+223271us) raft_consensus.cc:1671] finished
0924 14:31:36.986159 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:36.986345 (+   186us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2379,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"thread_start_us":100660,"threads_started":1,"wal-append.queue_time_us":247383}
I20260924 14:31:37.078276 26325 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:37.047360 31558 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:37.079037 31933 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:37.025524 31995 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:36.891501 32002 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:37.107883 32002 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:37.032199 31521 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:37.110014 26325 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260924 14:31:37.032866 31562 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.967689 31941 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:36.972458 31939 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:36.964442 31931 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:37.034296 31857 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:37.057999 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:37.059803 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:37.059931 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.060590 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:37.060899 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.042063 31984 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260924 14:31:37.061774 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.002259 31997 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:37.081530 26324 raft_consensus.cc:1226] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260924 14:31:37.084640 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.124595 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:37.087728 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.089146 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.096481 31846 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:36.902040 31454 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:37.099140 31875 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:37.003847 31980 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:37.105829 31995 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:36.968734 31767 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:37.114810 27182 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:37.115607 27186 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:37.118965 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:36.909466 31888 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:37.125360 26256 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:37.129467 26330 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:37.041531 31733 raft_consensus.cc:686] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:36.973268 31893 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.127s
I20260924 14:31:37.132493 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.143401 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:37.147929 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:37.166553 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 450739us
W20260924 14:31:37.200678 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57426.5us
I20260924 14:31:37.195536 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1163) took 1280 ms. Trace:
I20260924 14:31:37.167382 26251 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:37.203119 26330 rpcz_store.cc:276] 0924 14:31:35.915301 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.050694 (+1135393us) service_pool.cc:227] Handling call
0924 14:31:37.128150 (+ 77456us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.195207 (+ 67057us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.195476 (+   269us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:37.045814 31904 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
I20260924 14:31:37.205042 26325 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
W20260924 14:31:37.150687 32006 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260924 14:31:37.210337 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:37.131690 31931 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:37.132418 31818 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:37.224752 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:37.195945 31939 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:37.046269 31954 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.220s
I20260924 14:31:37.234714 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:37.231529 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:37.237984 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:37.242673 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:37.046720 26404 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:37.152496 31830 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:37.210203 31933 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:37.248951 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:37.237185 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:37.215328 31895 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:37.255224 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:37.258697 26330 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:37.244951 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.262899 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:37.230583 31392 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:37.267412 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:37.315311 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47975.4us
I20260924 14:31:37.315666 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.314167 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.317705 26404 raft_consensus.cc:2396] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260924 14:31:37.318871 27185 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:37.319589 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:37.320833 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:37.321142 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.251847 31933 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:37.324653 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1182) took 2137 ms. Trace:
I20260924 14:31:37.258306 31818 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:37.331164 26330 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:37.333238 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:37.333988 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.334003 26325 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:37.259690 31721 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:37.336086 27191 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:37.335981 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:37.322249 31623 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:37.335172 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1795) took 1459 ms. Trace:
I20260924 14:31:37.344782 26398 rpcz_store.cc:276] 0924 14:31:35.875857 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.774676 (+898819us) service_pool.cc:227] Handling call
0924 14:31:36.775080 (+   404us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:36.775546 (+   466us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:36.775589 (+    43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:36.833662 (+ 58073us) log.cc:842] Serialized 26 byte log entry
0924 14:31:37.133614 (+299952us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:37.133668 (+    54us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.133773 (+   105us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:37.334958 (+201185us) raft_consensus.cc:1671] finished
0924 14:31:37.334979 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.335118 (+   139us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":621,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":54668,"thread_start_us":237237,"threads_started":1,"wal-append.queue_time_us":486884}
I20260924 14:31:37.269423 31392 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:37.348066 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.350071 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.350500 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:37.353652 26330 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:37.323377 31893 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:37.326769 26331 rpcz_store.cc:276] 0924 14:31:35.198056 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.939097 (+1741041us) service_pool.cc:227] Handling call
0924 14:31:36.939418 (+   321us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:36.955588 (+ 16170us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:37.034771 (+ 79183us) log.cc:842] Serialized 14 byte log entry
0924 14:31:37.323699 (+288928us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:37.323752 (+    53us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:37.323803 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.324484 (+   681us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.324594 (+   110us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":39769,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":1212,"thread_start_us":287306,"threads_started":1}
I20260924 14:31:37.248051 31939 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:37.373910 31939 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:37.353886 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.337546 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.376835 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.377493 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1852) took 1591 ms. Trace:
I20260924 14:31:37.377578 26404 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:37.357210 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:37.378023 26400 rpcz_store.cc:276] 0924 14:31:35.785937 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.045669 (+259732us) service_pool.cc:227] Handling call
0924 14:31:36.183730 (+138061us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:36.451882 (+268152us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:36.451932 (+    50us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:36.451992 (+    60us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:36.452039 (+    47us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.377240 (+925201us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54001c1954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:37.377289 (+    49us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.377417 (+   128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":616004352}
I20260924 14:31:37.378775 26398 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:37.357313 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.380409 31767 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:37.380501 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:37.381300 26255 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:37.381358 31857 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:37.346594 31890 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260924 14:31:37.382252 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:37.383677 31720 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:37.385167 27188 raft_consensus.cc:2379] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:37.386049 31720 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:37.341217 31733 consensus_queue.cc:237] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:37.389809 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1851) took 1626 ms. Trace:
I20260924 14:31:37.351469 31623 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:37.390175 26401 rpcz_store.cc:276] 0924 14:31:35.763691 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.035752 (+272061us) service_pool.cc:227] Handling call
0924 14:31:36.230371 (+194619us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:36.512388 (+282017us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:36.512439 (+    51us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:36.512500 (+    61us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:36.512550 (+    50us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.389567 (+877017us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5400217854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:37.389614 (+    47us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.389754 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":647926016}
I20260924 14:31:37.385097 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.391521 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:37.391765 31623 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:37.336833 31818 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:37.392758 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.342106 31521 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:37.393400 31521 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:37.393783 31521 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:37.393110 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:37.318611 31996 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260924 14:31:37.395476 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.396171 31996 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:37.396703 31996 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:37.397496 31996 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:37.398665 31996 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:37.399714 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1847) took 2327 ms. Trace:
I20260924 14:31:37.400516 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:37.401803 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:37.402112 26224 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:37.402496 26253 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 11
I20260924 14:31:37.403990 31890 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:37.404210 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.405110 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:37.405422 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.405949 26397 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 18: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:37.405532 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.406633 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.406765 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.365517 26252 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:37.268537 31924 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260924 14:31:37.408206 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:37.350134 31392 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:37.408802 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:37.409964 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.410176 31346 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:37.410764 31346 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:37.414041 31875 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:37.414490 31875 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:37.415418 26398 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:37.415834 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.416589 27188 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:37.366879 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:37.417953 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:37.418646 26331 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:37.343514 31431 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:37.419770 31431 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:37.419515 31392 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:37.420159 31431 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:37.368019 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:37.418661 32012 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260924 14:31:37.428388 31392 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:37.428608 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1181) took 2241 ms. Trace:
I20260924 14:31:37.428978 26328 rpcz_store.cc:276] 0924 14:31:35.187211 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.684072 (+1496861us) service_pool.cc:227] Handling call
0924 14:31:36.684529 (+   457us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:36.684812 (+   283us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:36.684859 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:36.685396 (+   537us) log.cc:842] Serialized 26 byte log entry
0924 14:31:36.946259 (+260863us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:36.946308 (+    49us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:36.946415 (+   107us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:37.428392 (+481977us) raft_consensus.cc:1671] finished
0924 14:31:37.428415 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.428551 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":221135,"threads_started":1,"wal-append.queue_time_us":409499}
I20260924 14:31:37.430831 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:37.368999 26403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 1776) took 3083 ms (client timeout 1756 ms). Trace:
I20260924 14:31:37.371773 26399 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:37.431737 26399 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:31:37.432297 31949 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 2.032s	user 0.006s	sys 0.003s
I20260924 14:31:37.423058 26251 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:31:37.432686 26399 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:37.327195 31830 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260924 14:31:37.433696 26399 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:37.433832 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:37.434631 26251 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 17
I20260924 14:31:37.392943 31818 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:37.431633 26403 rpcz_store.cc:269] 0924 14:31:34.285791 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.286080 (+   289us) service_pool.cc:227] Handling call
0924 14:31:37.368775 (+3082695us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54001f5ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:37.368926 (+   151us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4068145024}
I20260924 14:31:37.437022 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:37.437763 26403 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:37.329454 31933 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:37.405102 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.360219 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:37.400148 26405 rpcz_store.cc:276] 0924 14:31:35.072377 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.560787 (+488410us) service_pool.cc:227] Handling call
0924 14:31:35.921976 (+361189us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:35.960217 (+ 38241us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.399509 (+1439292us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54002fc6d0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:37.399653 (+   144us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":778890496}
I20260924 14:31:37.440796 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.419373 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:37.441596 26405 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:37.442459 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.443044 26405 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:37.443673 26399 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260924 14:31:37.366036 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 1982) took 1963 ms. Trace:
I20260924 14:31:37.455868 26254 rpcz_store.cc:276] 0924 14:31:35.402996 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.403123 (+   127us) service_pool.cc:227] Handling call
0924 14:31:35.439981 (+ 36858us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:35.774076 (+334095us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.365112 (+1591036us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b5400381dd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:37.365974 (+   862us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":685530368}
I20260924 14:31:37.444099 26328 raft_consensus.cc:2396] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260924 14:31:37.444751 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:37.443325 26224 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:37.439185 27191 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:37.364267 26371 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:37.447662 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.449939 26401 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:37.450783 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.312531 31999 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
W20260924 14:31:37.527268 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 119145us
I20260924 14:31:37.451862 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1840) took 2941 ms. Trace:
I20260924 14:31:37.265379 31895 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:37.343696 31828 consensus_queue.cc:1046] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260924 14:31:37.453452 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.357534 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:37.416525 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.541237 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:37.541525 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.542097 26397 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:37.543876 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 100649us
I20260924 14:31:37.452672 31516 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260924 14:31:37.564735 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 114885us
W20260924 14:31:37.462644 31956 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.028s	sys 0.029s
W20260924 14:31:37.567473 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 209971us
I20260924 14:31:37.568363 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.568548 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:37.570087 32002 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:37.570225 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:37.570816 32002 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:37.458220 26399 pending_rounds.cc:76] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 6
W20260924 14:31:37.463546 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 514432us
I20260924 14:31:37.353415 31971 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:37.479997 26405 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:37.486815 26251 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:37.490285 26403 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:37.474874 26253 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
W20260924 14:31:37.529067 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 122267us
I20260924 14:31:37.528064 26406 rpcz_store.cc:276] 0924 14:31:34.510169 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:34.515428 (+  5259us) service_pool.cc:227] Handling call
0924 14:31:34.533947 (+ 18519us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:34.534310 (+   363us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:34.534354 (+    44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:34.570493 (+ 36139us) log.cc:842] Serialized 26 byte log entry
0924 14:31:34.881397 (+310904us) raft_consensus.cc:1631] Marking committed up to 4
0924 14:31:34.881452 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:34.881548 (+    96us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:37.258518 (+2376970us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:37.433322 (+174804us) raft_consensus.cc:1671] finished
0924 14:31:37.433343 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.451790 (+ 18447us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":33939,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":3130,"spinlock_wait_cycles":63073152,"thread_start_us":274998,"threads_started":1,"wal-append.queue_time_us":663287}
I20260924 14:31:37.312726 32007 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.147s
W20260924 14:31:37.533134 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 168954us
I20260924 14:31:37.566381 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.620393 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.361543 31893 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:37.451867 27190 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.036s	sys 0.058s
W20260924 14:31:37.625957 27190 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.036s	sys 0.058s
I20260924 14:31:37.621906 26399 pending_rounds.cc:97] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260924 14:31:37.626965 26399 raft_consensus.cc:2866] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:37.626825 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 965) took 1663 ms. Trace:
I20260924 14:31:37.621270 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.629395 26403 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:37.630831 27126 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:37.632625 26323 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.023s	sys 0.063s
W20260924 14:31:37.633313 26323 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.023s	sys 0.063s
I20260924 14:31:37.633728 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1800) took 1747 ms. Trace:
I20260924 14:31:37.635546 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.636445 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:37.589339 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:37.621506 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:37.627919 27190 rpcz_store.cc:276] 0924 14:31:35.963607 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.963791 (+   184us) service_pool.cc:227] Handling call
0924 14:31:35.996861 (+ 33070us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:35.997088 (+   227us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:36.041963 (+ 44875us) log.cc:842] Serialized 14 byte log entry
0924 14:31:36.852672 (+810709us) log.cc:842] Serialized 14 byte log entry
0924 14:31:37.626432 (+773760us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:37.626506 (+    74us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:37.626562 (+    56us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.626624 (+    62us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.626761 (+   137us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":59542,"prepare.queue_time_us":65017,"prepare.run_cpu_time_us":13356,"prepare.run_wall_time_us":172070,"raft.queue_time_us":93614,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":5101,"spinlock_wait_cycles":6837504,"thread_start_us":1530035,"threads_started":3,"wal-append.queue_time_us":610642}
I20260924 14:31:37.640107 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1802) took 1138 ms. Trace:
I20260924 14:31:37.640825 26403 rpcz_store.cc:276] 0924 14:31:36.501546 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.638840 (+1137294us) service_pool.cc:227] Handling call
0924 14:31:37.639388 (+   548us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.639640 (+   252us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:37.639677 (+    37us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:37.639742 (+    65us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:37.639788 (+    46us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.639875 (+    87us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.640008 (+   133us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:37.574528 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:37.621503 26401 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:37.628490 26369 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:37.644660 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 55479.3us
I20260924 14:31:37.644788 26369 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:37.645452 26403 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:35444
I20260924 14:31:37.647230 26401 raft_consensus.cc:1226] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:37.647900 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1805) took 1130 ms. Trace:
I20260924 14:31:37.642640 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1801) took 1756 ms. Trace:
I20260924 14:31:37.648294 26401 rpcz_store.cc:276] 0924 14:31:36.517146 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.643823 (+1126677us) service_pool.cc:227] Handling call
0924 14:31:37.647057 (+  3234us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.647669 (+   612us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.647848 (+   179us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260924 14:31:37.622426 31956 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.033s	sys 0.034s
I20260924 14:31:37.652200 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:37.645687 26400 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:37.650372 26403 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:31:37.649115 26369 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:37.567752 31924 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:37.574041 31846 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:37.661075 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.415338 31875 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:37.655086 27126 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:37.618965 31828 raft_consensus.cc:3032] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:37.565064 31997 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:37.682019 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 102947us
I20260924 14:31:37.683766 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:37.684554 26406 raft_consensus.cc:1226] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:37.655997 26403 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:37.622506 32002 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:37.729507 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1798) took 1852 ms. Trace:
I20260924 14:31:37.729898 26406 rpcz_store.cc:276] 0924 14:31:35.876782 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.637643 (+1760861us) service_pool.cc:227] Handling call
0924 14:31:37.684341 (+ 46698us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.729256 (+ 44915us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.729450 (+   194us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:37.649292 26398 rpcz_store.cc:276] 0924 14:31:35.886223 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.641124 (+1754901us) service_pool.cc:227] Handling call
0924 14:31:37.641807 (+   683us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.642066 (+   259us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:37.642223 (+   157us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:37.642294 (+    71us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:37.642354 (+    60us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.642427 (+    73us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.642564 (+   137us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:37.732774 26398 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260924 14:31:37.634495 26405 rpcz_store.cc:276] 0924 14:31:35.885908 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.632779 (+1746871us) service_pool.cc:227] Handling call
0924 14:31:37.633187 (+   408us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.633408 (+   221us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:37.633438 (+    30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:37.633478 (+    40us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:37.633514 (+    36us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.633567 (+    53us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.633664 (+    97us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:37.567507 31368 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:37.635012 31893 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:37.623925 31890 raft_consensus.cc:2741] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260924 14:31:37.652972 31989 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:37.576577 31516 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:37.754357 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:37.653168 31904 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:37.451923 31931 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:37.651460 31929 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260924 14:31:37.744246 31953 thread.cc:636] apply [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.013s	sys 0.017s
W20260924 14:31:37.747361 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 92364.6us
I20260924 14:31:37.624128 31454 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:37.658581 31945 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:37.687754 31888 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:37.719187 31846 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:37.872406 31846 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:37.637936 31721 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.108s
I20260924 14:31:37.653599 32027 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:37.876626 32027 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:37.877002 32027 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:31:37.826565 32017 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260924 14:31:37.761139 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:37.887135 27190 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:37.763144 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:37.639493 31971 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:37.846639 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:37.718604 31828 raft_consensus.cc:729] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:37.761960 31516 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:37.755752 31999 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:31:37.769577 31953 thread.cc:602] apply [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.014s	sys 0.017s
I20260924 14:31:37.840127 27125 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:37.852120 31879 consensus_meta.cc:287] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.880s	user 0.011s	sys 0.000s
I20260924 14:31:37.746199 31994 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260924 14:31:37.854532 31991 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:37.858760 31852 consensus_meta.cc:287] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.787s	user 0.007s	sys 0.001s
I20260924 14:31:37.762084 31956 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:37.861567 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1168) took 2688 ms. Trace:
I20260924 14:31:37.862406 31346 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:37.928267 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.838001 31454 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:37.931703 31368 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:37.932094 31454 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:37.926023 31879 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:37.932807 31454 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:37.864849 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1515) took 1184 ms. Trace:
I20260924 14:31:37.933693 31852 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:37.778982 31174 consensus_meta.cc:287] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.885s	user 0.009s	sys 0.001s
I20260924 14:31:37.934774 31879 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:37.705449 31924 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:37.759770 32022 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:37.935499 31828 consensus_queue.cc:260] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:37.879904 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1702) took 1132 ms. Trace:
I20260924 14:31:37.936970 31828 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:37.937098 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1797) took 2060 ms. Trace:
I20260924 14:31:37.935861 26330 raft_consensus.cc:2396] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260924 14:31:37.935192 31174 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:37.763151 26403 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:37.940513 31174 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:37.940774 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1190) took 1745 ms. Trace:
I20260924 14:31:37.940935 26403 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:37.941510 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:35444 (request call id 1804) took 1426 ms. Trace:
I20260924 14:31:37.941880 26403 rpcz_store.cc:276] 0924 14:31:36.515253 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.643257 (+1128004us) service_pool.cc:227] Handling call
0924 14:31:37.941433 (+298176us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:37.943830 27192 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260924 14:31:37.943686 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.898146 31857 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:37.944396 26398 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 18. Ops: []
W20260924 14:31:37.914970 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 137396us
I20260924 14:31:37.944731 31857 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:37.945501 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:37.946079 27186 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17. Ops: []
W20260924 14:31:37.863399 31982 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.104s	user 0.012s	sys 0.000s
I20260924 14:31:37.941210 27189 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260924 14:31:37.947621 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1796) took 2071 ms. Trace:
I20260924 14:31:37.948113 26399 rpcz_store.cc:276] 0924 14:31:35.876156 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.835151 (+958995us) service_pool.cc:227] Handling call
0924 14:31:37.371642 (+536491us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.627759 (+256117us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.947418 (+319659us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b54001f5ed0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:37.947567 (+   149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1232746880}
I20260924 14:31:37.934309 27181 rpcz_store.cc:276] 0924 14:31:36.680602 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.081994 (+401392us) service_pool.cc:227] Handling call
0924 14:31:37.082465 (+   471us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:37.082745 (+   280us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:37.082780 (+    35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:37.103618 (+ 20838us) log.cc:842] Serialized 26 byte log entry
0924 14:31:37.644625 (+541007us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:37.644655 (+    30us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.644746 (+    91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:37.864607 (+219861us) raft_consensus.cc:1671] finished
0924 14:31:37.864628 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.864786 (+   158us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":493538,"threads_started":1,"wal-append.queue_time_us":736719}
I20260924 14:31:37.928066 31956 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:37.950464 31956 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:37.953330 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.017s	sys 0.020s
W20260924 14:31:37.953733 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.018s	sys 0.021s
I20260924 14:31:37.937605 26402 rpcz_store.cc:276] 0924 14:31:35.876489 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.837584 (+961095us) service_pool.cc:227] Handling call
0924 14:31:36.837988 (+   404us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:36.901930 (+ 63942us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:36.901961 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:36.902415 (+   454us) log.cc:842] Serialized 26 byte log entry
0924 14:31:37.112294 (+209879us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:37.112349 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.367564 (+255215us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x7b5400476654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:37.367613 (+    49us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:37.936754 (+569141us) raft_consensus.cc:1671] finished
0924 14:31:37.936775 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.937021 (+   246us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":153,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":7257,"spinlock_wait_cycles":146172160,"thread_start_us":209542,"threads_started":1,"wal-append.queue_time_us":630368}
I20260924 14:31:37.935828 31852 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:37.936079 31924 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:37.888626 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 119384us
W20260924 14:31:37.889556 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 344635us
I20260924 14:31:37.936523 26258 rpcz_store.cc:276] 0924 14:31:36.743244 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.838789 (+ 95545us) service_pool.cc:227] Handling call
0924 14:31:36.845279 (+  6490us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:37.097972 (+252693us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:37.098008 (+    36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:37.098459 (+   451us) log.cc:842] Serialized 26 byte log entry
0924 14:31:37.570975 (+472516us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:37.571859 (+   884us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.874977 (+303118us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54003822d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:37.875048 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:37.875113 (+    65us) raft_consensus.cc:1671] finished
0924 14:31:37.875132 (+    19us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.875418 (+   286us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":580393344,"thread_start_us":472142,"threads_started":1,"wal-append.queue_time_us":665816}
I20260924 14:31:37.958922 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:37.899881 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 136818us
I20260924 14:31:37.963615 31346 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:37.966009 31346 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:37.966475 31346 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:37.969497 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:37.895326 31862 consensus_queue.cc:1046] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260924 14:31:37.927434 26332 rpcz_store.cc:276] 0924 14:31:35.172699 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.891270 (+718571us) service_pool.cc:227] Handling call
0924 14:31:35.892111 (+   841us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:35.892809 (+   698us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:35.892847 (+    38us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:35.893424 (+   577us) log.cc:842] Serialized 26 byte log entry
0924 14:31:37.097707 (+1204283us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b1800041380. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:31:37.109958 (+ 12251us) spinlock_profiling.cc:243] Waited 6.42 ms on lock 0x7b48000c21d8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:31:37.533268 (+423310us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:37.533311 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.533383 (+    72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:37.861350 (+327967us) raft_consensus.cc:1671] finished
0924 14:31:37.861374 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.861505 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":15520,"raft.run_wall_time_us":1213262,"spinlock_wait_cycles":20175104,"thread_start_us":435347,"threads_started":1,"wal-append.queue_time_us":1941845}
I20260924 14:31:37.946946 31857 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:37.971882 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:37.974180 26224 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:37.941267 26323 rpcz_store.cc:276] 0924 14:31:36.195338 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.100770 (+905432us) service_pool.cc:227] Handling call
0924 14:31:37.108637 (+  7867us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:37.109091 (+   454us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:37.109130 (+    39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:37.109687 (+   557us) log.cc:842] Serialized 26 byte log entry
0924 14:31:37.634107 (+524420us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:37.659747 (+ 25640us) log.cc:842] Serialized 14 byte log entry
0924 14:31:37.681419 (+ 21672us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:37.681535 (+   116us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:37.940541 (+259006us) raft_consensus.cc:1671] finished
0924 14:31:37.940564 (+    23us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:37.940703 (+   139us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":25713,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":1353,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":6138,"thread_start_us":524029,"threads_started":1,"wal-append.queue_time_us":828793}
I20260924 14:31:37.889448 31945 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:37.932018 31895 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:37.988495 31991 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:37.912835 32021 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:37.898895 31521 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:38.007236 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61831.4us
W20260924 14:31:38.015110 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 58127.3us
I20260924 14:31:37.999069 31830 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:37.996338 31892 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:37.947716 31982 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:38.048466 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 426997us
I20260924 14:31:38.031620 32036 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:37.997061 31368 raft_consensus.cc:686] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:38.056761 26249 raft_consensus.cc:2379] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 18: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:38.014351 31857 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:38.049109 32033 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260924 14:31:38.011234 27190 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
W20260924 14:31:38.061863 31431 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.641s	user 0.007s	sys 0.004s
I20260924 14:31:37.990949 31346 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:38.051167 31558 raft_consensus.cc:3032] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:38.078009 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:38.079659 26298 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:38.062494 31882 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:37.995093 31818 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:38.065969 27185 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:38.068274 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:38.038640 31991 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:38.075438 26397 consensus_meta.cc:287] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.503s	user 0.005s	sys 0.002s
I20260924 14:31:38.039203 31728 consensus_queue.cc:1046] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260924 14:31:38.086062 26397 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:38.086997 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:38.082230 31996 consensus_meta.cc:287] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.683s	user 0.013s	sys 0.000s
I20260924 14:31:38.096020 31728 raft_consensus.cc:3032] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:38.106216 31728 raft_consensus.cc:729] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:38.107995 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:38.107352 31728 consensus_queue.cc:260] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.108700 26252 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:38.109501 31728 raft_consensus.cc:3037] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:38.088747 31852 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:38.120410 31346 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:38.083753 31431 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:38.124219 31431 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:38.091794 31834 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:38.125265 31834 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:38.126228 31834 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.091017 31941 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:38.127497 31941 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:38.128342 31941 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.132030 31368 consensus_queue.cc:237] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:38.133764 31893 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:38.134827 31996 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:38.135519 31996 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.093384 31733 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260924 14:31:38.137944 31996 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:38.138717 26250 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:38.138705 31846 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.140816 31846 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:38.098551 32027 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.100715 31982 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:38.067246 32036 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:38.090040 31997 raft_consensus.cc:2741] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260924 14:31:38.083011 31558 raft_consensus.cc:729] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:38.104148 31971 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:38.104147 32021 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:38.142704 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.145635 26399 raft_consensus.cc:1261] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:31:38.161417 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:38.147460 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 309559us
I20260924 14:31:38.151463 26252 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:38.155146 31516 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:38.090601 31991 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:38.158666 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 68934.9us
I20260924 14:31:38.164109 26397 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:38.173766 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.069540 31857 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:38.144333 31846 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:38.183427 26399 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:31:38.185531 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.194267 26397 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:38.160118 31989 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:38.160259 32021 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:38.160261 31942 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:38.201860 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:38.196168 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.001s	sys 0.019s
I20260924 14:31:38.192734 31781 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:38.207208 27125 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:38.209218 31933 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.747s	user 0.010s	sys 0.000s
I20260924 14:31:38.200603 31893 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:38.213043 32021 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:38.240136 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:38.240850 27190 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 22.
I20260924 14:31:38.197568 31862 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260924 14:31:38.242170 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:38.248091 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 216182us
I20260924 14:31:38.224658 31519 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:38.203136 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:38.252586 26399 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:38.253743 31852 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260924 14:31:38.254271 31852 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:38.254611 31852 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Advancing to term 23
I20260924 14:31:38.244711 27123 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:38.235603 31971 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:38.283557 31519 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:38.284629 31519 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:38.285313 31863 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:38.285876 31863 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:38.286705 31863 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:38.214851 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:38.288962 31728 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:38.289072 32019 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:38.290566 31728 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:38.290958 31728 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:38.199029 31846 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:38.224761 31781 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:38.214092 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.001s	sys 0.019s
I20260924 14:31:38.291575 31989 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.292106 31857 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:38.184804 26369 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.010s	sys 0.024s
I20260924 14:31:38.312064 31781 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:38.312680 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:38.312680 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:38.313666 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 661502us
W20260924 14:31:38.313855 32052 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260924 14:31:38.314636 31846 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:38.314828 32027 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:38.242025 31945 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:38.307775 31971 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:38.242226 31521 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:38.309309 31516 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:38.310442 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:38.321677 26369 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.010s	sys 0.024s
I20260924 14:31:38.322938 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:38.320717 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:38.223002 31942 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.320842 32023 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:38.212888 31558 consensus_queue.cc:260] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.339278 31980 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:38.234629 31933 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.339600 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:38.340365 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 253463us
I20260924 14:31:38.376127 26298 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:38.376695 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1862) took 1713 ms. Trace:
I20260924 14:31:38.377270 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:38.342939 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:38.355374 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:38.355585 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1701) took 1612 ms. Trace:
I20260924 14:31:38.379307 26257 rpcz_store.cc:276] 0924 14:31:36.742906 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.743602 (+   696us) service_pool.cc:227] Handling call
0924 14:31:36.759623 (+ 16021us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:36.766262 (+  6639us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:38.355363 (+1589101us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b540002b550. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:38.355519 (+   156us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":35917568}
W20260924 14:31:38.356426 26326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1192) took 2143 ms (client timeout 1910 ms). Trace:
I20260924 14:31:38.380514 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:38.380467 26326 rpcz_store.cc:269] 0924 14:31:36.213343 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:36.472244 (+258901us) service_pool.cc:227] Handling call
0924 14:31:38.356205 (+1883961us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54007d37d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:38.356344 (+   139us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":390214400}
I20260924 14:31:38.381755 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:38.382241 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:38.358443 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 709339us
I20260924 14:31:38.361934 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:38.318199 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1586) took 1292 ms. Trace:
I20260924 14:31:38.363576 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:38.365585 26399 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:38.385074 26325 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 19: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:38.338578 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:38.337090 31857 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:38.382366 26326 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:38.356755 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:38.386189 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:38.384222 26405 rpcz_store.cc:276] 0924 14:31:37.025887 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.734226 (+708339us) service_pool.cc:227] Handling call
0924 14:31:37.734490 (+   264us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.734706 (+   216us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:38.150802 (+416096us) log.cc:842] Serialized 14 byte log entry
0924 14:31:38.317799 (+166997us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:38.317876 (+    77us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:38.317941 (+    65us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:38.318032 (+    91us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:38.318148 (+   116us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":34621,"prepare.queue_time_us":532201,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":86,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":16568,"thread_start_us":476774,"threads_started":2}
I20260924 14:31:38.391587 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:38.396770 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:38.229600 32036 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:38.363965 31521 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:38.336714 31956 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:38.405104 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:38.411800 26404 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:38.400698 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:38.389128 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:38.413615 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:38.397192 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1863) took 1564 ms. Trace:
I20260924 14:31:38.337383 31924 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.395522 31846 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:38.390992 32048 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:38.403162 31893 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:38.404618 31834 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:38.417268 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:38.467648 26328 raft_consensus.cc:2396] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260924 14:31:38.403507 32027 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260924 14:31:38.396188 31516 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:38.405457 31985 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election won for term 16
W20260924 14:31:38.469775 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53026.3us
I20260924 14:31:38.414744 26400 rpcz_store.cc:276] 0924 14:31:36.833049 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.655703 (+822654us) service_pool.cc:227] Handling call
0924 14:31:37.728920 (+ 73217us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.729141 (+   221us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:38.178616 (+449475us) log.cc:842] Serialized 14 byte log entry
0924 14:31:38.396500 (+217884us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:38.396650 (+   150us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:38.396755 (+   105us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:38.397015 (+   260us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:38.397135 (+   120us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":505205,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":3161,"raft.queue_time_us":411,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":6698,"thread_start_us":664167,"threads_started":2}
I20260924 14:31:38.498147 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:38.499392 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:38.460024 31941 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:38.474291 31454 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
W20260924 14:31:38.506033 32055 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260924 14:31:38.506390 31454 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:38.507969 32057 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260924 14:31:38.365008 31728 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:38.367674 32023 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.509465 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:38.509752 31728 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:38.511649 32023 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:38.466641 31688 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.013s	sys 0.020s
W20260924 14:31:38.492300 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 75134.4us
I20260924 14:31:38.395180 31980 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.417611 31783 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:38.398372 31857 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.502188 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:38.502562 31941 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:38.502604 31996 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:38.496317 31834 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:38.480747 31931 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:38.504501 31924 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:38.504804 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0756687d99e643c094531124c9438376"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:38.505630 31985 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:38.530795 27189 raft_consensus.cc:595] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:38.497977 31346 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:38.531244 27189 raft_consensus.cc:603] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:38.504443 31895 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260924 14:31:38.528534 31834 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:38.526021 31996 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:38.535434 31996 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:38.541344 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:38.509057 31454 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:38.500897 32043 raft_consensus.cc:2741] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260924 14:31:38.544059 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:38.552516 27189 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:38.512923 31688 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.013s	sys 0.020s
I20260924 14:31:38.513176 31893 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:38.583062 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 226407us
I20260924 14:31:38.583413 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:38.583958 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:38.540073 31783 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:38.589973 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:38.529315 31931 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:38.593272 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:38.593658 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:38.601269 31971 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:38.601490 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:38.542127 31346 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:38.606527 31956 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:38.607496 32023 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:38.607925 32023 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:38.608296 32023 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:38.539505 31941 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:38.613523 31346 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:38.614481 31941 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:38.615113 31941 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:38.597659 31893 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.617269 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:38.617486 31893 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:38.617890 26400 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:38.619128 26255 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260924 14:31:38.622635 26254 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260924 14:31:38.626367 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:38.632511 27189 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:31:38.646554 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:38.598846 31783 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:38.653254 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:38.653892 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:38.598676 31931 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:38.648283 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:38.647320 26254 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:38.658282 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:38.659046 26254 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:38.660252 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:38.660957 26254 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
I20260924 14:31:38.661878 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:38.662519 26254 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:38.664587 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:38.608104 31818 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:38.666960 26254 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260924 14:31:38.688493 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:38.628683 31942 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:38.668681 31979 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260924 14:31:38.670989 32062 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260924 14:31:38.665359 31941 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:38.672919 31980 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:38.666635 32059 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:38.680109 32015 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.578s	user 0.002s	sys 0.001s
I20260924 14:31:38.637269 31991 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:38.648098 31933 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:38.686470 27186 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5581950b69104df485f842c6b78b1047"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:38.710919 27186 raft_consensus.cc:595] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:38.686705 26251 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:38.686949 31521 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:38.630779 31845 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:38.659020 31783 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:38.729946 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 214732us
I20260924 14:31:38.731530 26258 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 11. Current term is 12. Ops: []
W20260924 14:31:38.732692 26224 leader_election.cc:334] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583): Timed out: RequestConsensusVote RPC to 127.22.151.196:45583 timed out after 2.062s (SENT)
I20260924 14:31:38.733273 26224 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:38.703049 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:38.698125 31882 consensus_meta.cc:287] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.608s	user 0.005s	sys 0.007s
W20260924 14:31:38.687186 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 187853us
I20260924 14:31:38.748287 26224 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:38.750478 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:38.739509 32060 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.267s	user 0.000s	sys 0.002s
I20260924 14:31:38.700500 31834 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.743140 31818 consensus_queue.cc:1046] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260924 14:31:38.754132 27191 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260924 14:31:38.754709 26329 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:38.765153 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1164) took 2849 ms. Trace:
I20260924 14:31:38.768350 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.769541 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.771275 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.708282 32059 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.754395 31431 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:38.792585 26255 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260924 14:31:38.794523 32074 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260924 14:31:38.794202 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:38.767024 32016 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.107s	user 0.002s	sys 0.000s
I20260924 14:31:38.756877 31818 raft_consensus.cc:3032] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:38.837821 26252 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260924 14:31:38.795436 26403 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:31:38.774526 26330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1176) took 1433 ms (client timeout 1507 ms). Trace:
W20260924 14:31:38.754114 32065 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260924 14:31:38.696627 31985 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.779697 26250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 2011) took 1424 ms. Trace:
I20260924 14:31:38.765552 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1171) took 3591 ms. Trace:
W20260924 14:31:38.797302 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 400609us
I20260924 14:31:38.843428 26327 rpcz_store.cc:276] 0924 14:31:35.173662 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.908247 (+734585us) service_pool.cc:227] Handling call
0924 14:31:36.224074 (+315827us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:36.675333 (+451259us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:38.765312 (+2089979us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b540001ead4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:38.765495 (+   183us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1036539648}
I20260924 14:31:38.783756 26324 rpcz_store.cc:276] 0924 14:31:35.915612 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.077674 (+1162062us) service_pool.cc:227] Handling call
0924 14:31:37.081192 (+  3518us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:37.124802 (+ 43610us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:38.764930 (+1640128us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b540012e150. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:38.765078 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7012608}
I20260924 14:31:38.770584 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.799674 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:38.762748 26253 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:38.805461 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:38.754444 31971 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election won for term 18
W20260924 14:31:38.841030 26330 rpcz_store.cc:269] 0924 14:31:37.340723 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.444472 (+103749us) service_pool.cc:227] Handling call
0924 14:31:38.774319 (+1329847us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b540012eb54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:38.774472 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1054197504}
I20260924 14:31:38.757736 32080 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:38.844360 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:38.845635 27191 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:38.850534 27191 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:38.850261 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.852092 26399 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:38.846868 26224 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:38.843348 26250 rpcz_store.cc:276] 0924 14:31:37.354970 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.355225 (+   255us) service_pool.cc:227] Handling call
0924 14:31:37.359234 (+  4009us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:38.139392 (+780158us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:38.779438 (+640046us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b5400195ad4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:38.779642 (+   204us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1792007936}
I20260924 14:31:38.846731 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:38.849570 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:38.853116 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.851713 27191 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.858369 27191 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 18
I20260924 14:31:38.861871 26224 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:38.842754 31818 raft_consensus.cc:729] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:38.795790 31857 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:38.776288 32061 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260924 14:31:38.864598 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:38.778496 32077 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:38.882791 32077 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:38.883318 32077 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:38.776623 31852 consensus_meta.cc:287] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.003s
I20260924 14:31:38.759708 31933 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.076s
I20260924 14:31:38.867779 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.868741 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:38.766009 31895 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260924 14:31:38.888188 31933 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:38.887843 31852 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.888553 31895 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:38.888649 31933 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:38.889061 31895 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:38.889775 31933 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.889923 31895 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:38.775969 31894 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:38.891160 31933 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:38.891191 31895 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:38.892638 31852 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:38.891640 31894 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.873549 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:38.875970 26406 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: []
W20260924 14:31:38.876601 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 230344us
I20260924 14:31:38.879329 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:38.778749 32048 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.090s
I20260924 14:31:38.866336 31431 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:38.866074 31945 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:38.774711 31346 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.897688 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:38.770370 32085 consensus_queue.cc:1046] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260924 14:31:38.872359 31521 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:38.901262 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:38.901140 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:38.795217 31931 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.907552 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:38.917769 26330 raft_consensus.cc:2354] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14.
I20260924 14:31:38.918075 26327 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:31:38.905084 27191 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:38.802018 31887 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260924 14:31:38.870519 31677 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:38.910627 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:38.912314 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:38.876127 31971 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:38.919838 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:38.864176 31728 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.045s
I20260924 14:31:38.920244 26256 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260924 14:31:38.920924 26403 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 22.
I20260924 14:31:38.921828 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:38.922547 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:38.923719 32084 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260924 14:31:38.906451 32002 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:38.930845 26325 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:38.900898 32080 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:38.871551 32036 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260924 14:31:38.876789 32024 consensus_queue.cc:1046] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
W20260924 14:31:38.958125 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 60588.2us
I20260924 14:31:38.975466 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:38.975519 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 200959us
W20260924 14:31:38.922533 32073 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260924 14:31:38.958400 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:38.958974 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:38.949322 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:38.980134 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:38.915971 31521 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:38.961845 26256 pending_rounds.cc:76] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 8
I20260924 14:31:38.962930 26399 raft_consensus.cc:2379] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:38.981446 26256 pending_rounds.cc:97] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260924 14:31:38.916535 31996 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.931483 26400 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:38.981879 26256 raft_consensus.cc:2866] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:38.979818 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:38.967521 31892 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:38.971032 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:39.033740 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 62800.3us
I20260924 14:31:38.983028 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:38.983510 31346 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:38.983695 31521 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:38.980872 26330 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:38.978231 32080 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:38.919188 31945 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:38.948975 31677 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.994091 32059 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:38.998551 31931 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:38.961167 31857 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:39.004228 32093 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260924 14:31:38.975565 32027 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:39.008617 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:39.094087 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 85542.7us
I20260924 14:31:39.009289 32075 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:39.009231 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:39.011523 31335 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260924 14:31:39.013458 26249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55802 (request call id 2010) took 1697 ms (client timeout 2037 ms). Trace:
I20260924 14:31:38.377238 26401 rpcz_store.cc:276] 0924 14:31:36.662913 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.658498 (+995585us) service_pool.cc:227] Handling call
0924 14:31:37.730717 (+ 72219us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.730957 (+   240us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:38.143048 (+412091us) log.cc:842] Serialized 14 byte log entry
0924 14:31:38.375493 (+232445us) log.cc:842] Serialized 14 byte log entry
0924 14:31:38.376043 (+   550us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:38.376181 (+   138us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:38.376367 (+   186us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:38.376479 (+   112us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:38.376624 (+   145us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":521573,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":854,"thread_start_us":545166,"threads_started":2}
I20260924 14:31:39.021356 31783 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:39.022962 32072 log.cc:924] Time spent T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.395s	user 0.001s	sys 0.000s
I20260924 14:31:39.024134 32022 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:39.052924 31834 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:39.108136 31431 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:39.048211 31880 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260924 14:31:39.125005 31971 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.100608 32027 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:38.982335 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:39.032114 27126 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:38.986917 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:39.130146 27191 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Advancing to term 23
I20260924 14:31:39.041478 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:39.046566 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:38.918673 31991 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:39.066695 31945 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:39.088802 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:38.981156 31941 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:39.045953 31882 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260924 14:31:39.097067 26249 rpcz_store.cc:269] 0924 14:31:37.316244 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.316539 (+   295us) service_pool.cc:227] Handling call
0924 14:31:39.013221 (+1696682us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b54004c9d54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:39.013387 (+   166us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1648330112}
W20260924 14:31:39.089504 32021 consensus_meta.cc:287] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.839s	user 0.012s	sys 0.002s
I20260924 14:31:39.156284 32027 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:39.156368 32021 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:39.104095 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:39.027972 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50424us
I20260924 14:31:39.109440 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:39.160708 26221 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:39.161664 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:39.162745 27181 raft_consensus.cc:2354] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18.
I20260924 14:31:39.164273 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:39.165529 26295 leader_election.cc:398] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18.
I20260924 14:31:39.165962 26326 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:39.166158 26295 leader_election.cc:401] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:39.110340 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:39.052361 32089 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260924 14:31:39.113037 32022 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:39.115907 26298 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:39.117424 31677 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:39.159276 32021 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:39.118675 31894 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:39.132483 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:39.176945 26400 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:39.137118 26402 raft_consensus.cc:2354] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14.
W20260924 14:31:39.138592 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 106611us
I20260924 14:31:39.139124 27190 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
W20260924 14:31:39.123234 32023 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.000s
I20260924 14:31:39.045485 31892 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.100550 32059 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:39.146529 27125 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:39.120612 32080 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:39.154714 31996 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:39.167841 27191 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 23.
I20260924 14:31:39.144902 31982 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:39.185371 26401 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260924 14:31:39.206064 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 306153us
I20260924 14:31:39.178102 31882 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260924 14:31:39.197827 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51374.9us
I20260924 14:31:39.181116 31346 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 18
I20260924 14:31:39.209051 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:39.209884 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:39.214468 27190 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 23: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:39.215641 32077 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.180403 31677 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:39.216327 26405 raft_consensus.cc:1226] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260924 14:31:39.216682 31677 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.218804 31677 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:39.223941 31783 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:39.202569 31924 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:39.225355 31924 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:39.227957 26369 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.833s (SENT)
W20260924 14:31:39.229161 26369 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 2.011s (SENT)
I20260924 14:31:39.231040 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:39.232259 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:39.205008 31368 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:39.233415 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:39.233980 31982 raft_consensus.cc:686] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:39.234951 31982 consensus_queue.cc:237] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.237303 31996 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:39.238029 31996 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:39.213886 31882 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:39.238911 31882 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:39.244360 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 997) took 1929 ms. Trace:
I20260924 14:31:39.244843 27192 rpcz_store.cc:276] 0924 14:31:37.315229 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.317038 (+  1809us) service_pool.cc:227] Handling call
0924 14:31:37.335230 (+ 18192us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:37.944313 (+609083us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:39.244073 (+1299760us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400260a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:39.244294 (+   221us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1399032576}
I20260924 14:31:39.246623 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:39.248070 26404 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1583) took 2254 ms (client timeout 2062 ms). Trace:
W20260924 14:31:39.248509 26404 rpcz_store.cc:269] 0924 14:31:36.993285 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.395159 (+401874us) service_pool.cc:227] Handling call
0924 14:31:39.247848 (+1852689us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002fc6d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:39.247996 (+   148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2320610048}
I20260924 14:31:39.248987 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:39.239161 31996 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:39.252871 32036 raft_consensus.cc:3032] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:39.253348 32036 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:39.254235 32036 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:39.256798 31558 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:39.260501 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 971) took 2202 ms. Trace:
I20260924 14:31:39.260926 27194 rpcz_store.cc:276] 0924 14:31:37.058124 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.090279 (+ 32155us) service_pool.cc:227] Handling call
0924 14:31:37.090644 (+   365us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.222366 (+131722us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:37.240486 (+ 18120us) log.cc:842] Serialized 14 byte log entry
0924 14:31:37.747754 (+507268us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:37.747832 (+    78us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:37.747899 (+    67us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:39.260239 (+1512340us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540065f754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:39.260292 (+    53us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:39.260421 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":118936,"prepare.queue_time_us":18474,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":57645,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":8281,"spinlock_wait_cycles":302227328,"thread_start_us":466651,"threads_started":1,"wal-append.queue_time_us":756962,"wal-append.run_cpu_time_us":2010,"wal-append.run_wall_time_us":1083598}
I20260924 14:31:39.160861 31941 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:39.177597 31974 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:39.170636 31991 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:39.181296 32027 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:39.210273 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:39.194355 26400 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:39.184460 32059 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:39.266268 26397 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260924 14:31:39.184535 32023 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:39.267510 26405 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:39.289485 32023 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:39.255407 32036 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:39.266072 31887 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:39.266659 31783 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:39.271832 32059 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:39.279316 31933 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:39.303737 31924 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:39.332581 32043 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Stepping down as leader of term 17
W20260924 14:31:39.331404 32103 log.cc:924] Time spent T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260924 14:31:39.349473 32059 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:39.355827 32059 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:39.355988 32077 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:39.343195 32043 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:39.357725 32043 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.359050 32043 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:39.167917 26221 leader_election.cc:398] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14.
I20260924 14:31:39.413540 26221 leader_election.cc:401] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260924 14:31:39.175114 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 59333.3us
I20260924 14:31:39.424467 26327 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:39.171234 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 60991us
W20260924 14:31:39.414901 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 252944us
I20260924 14:31:39.433838 26256 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:39.434206 26295 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:39.434930 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:39.439575 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:39.435675 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:39.440943 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:39.442709 26252 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 18: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:39.442886 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:39.444176 26401 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:39.444458 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:39.446122 26399 raft_consensus.cc:2396] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260924 14:31:39.449841 26252 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:39.453706 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:39.459602 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:39.462466 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:39.463363 26252 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:39.465502 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:39.469664 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:39.477067 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:39.484025 31266 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:39.485374 31266 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:39.486750 31941 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:39.492759 31941 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:39.488423 31924 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:39.491007 31892 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:39.312170 31933 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:39.510587 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 45313us
I20260924 14:31:39.495405 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:39.499984 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:39.494606 31431 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:39.546098 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 81698.8us
I20260924 14:31:39.504009 31890 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:39.495428 31941 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:39.505475 31974 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:39.504408 31954 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:39.536945 31335 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 23 FOLLOWER]: Leader election won for term 23
I20260924 14:31:39.504951 31971 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:39.503949 31893 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:39.496047 31924 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.563905 31924 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:39.565059 31335 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 23 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:39.566972 26258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 2017) took 1526 ms. Trace:
W20260924 14:31:39.567847 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 72062.2us
I20260924 14:31:39.569581 31266 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:39.567266 31335 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:39.498692 31368 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260924 14:31:39.496383 31980 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.094s
I20260924 14:31:39.498010 31892 consensus_queue.cc:1046] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.106s
I20260924 14:31:39.552696 31890 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:39.598134 26252 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260924 14:31:39.605815 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50025.6us
I20260924 14:31:39.596444 27182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 972) took 2537 ms. Trace:
I20260924 14:31:39.571415 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1811) took 2246 ms. Trace:
I20260924 14:31:39.587760 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:39.588268 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 76052.7us
I20260924 14:31:39.598575 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:39.588157 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 375915us
I20260924 14:31:39.635320 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:39.635486 31985 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260924 14:31:39.635964 26224 leader_election.cc:398] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14.
I20260924 14:31:39.636579 26252 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:39.636873 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:39.567426 26258 rpcz_store.cc:276] 0924 14:31:38.040340 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:38.040865 (+   525us) service_pool.cc:227] Handling call
0924 14:31:38.044347 (+  3482us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:38.732204 (+687857us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:39.566257 (+834053us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54006d2ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:39.566701 (+   444us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1579620224}
I20260924 14:31:39.599120 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:39.641866 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 86046.5us
I20260924 14:31:39.555750 31974 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:39.622820 27182 rpcz_store.cc:276] 0924 14:31:37.058424 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.091722 (+ 33298us) service_pool.cc:227] Handling call
0924 14:31:37.092025 (+   303us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.166829 (+ 74804us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:39.595861 (+2429032us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b540001b8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:39.596212 (+   351us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":51425664}
I20260924 14:31:39.605381 31890 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:39.650401 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50086 (request call id 999) took 2301 ms. Trace:
I20260924 14:31:39.623317 26398 rpcz_store.cc:276] 0924 14:31:37.325133 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.743308 (+418175us) service_pool.cc:227] Handling call
0924 14:31:37.743876 (+   568us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:37.944876 (+201000us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:39.570894 (+1626018us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400476b54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:39.571220 (+   326us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":460733184}
I20260924 14:31:39.604537 31892 raft_consensus.cc:3032] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Stepping down as leader of term 10
W20260924 14:31:39.588425 32026 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.746s	user 0.003s	sys 0.001s
I20260924 14:31:39.651393 31974 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:39.652817 31985 raft_consensus.cc:3032] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:39.653671 26256 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:39.605051 31933 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:39.596385 32115 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:39.596002 31893 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:39.658110 31892 raft_consensus.cc:729] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:39.667872 32021 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:39.675015 26249 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260924 14:31:39.676399 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:39.679035 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:39.737908 26254 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:39.739018 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:39.739173 31893 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:39.739979 26250 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 9.
I20260924 14:31:39.679905 31924 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:39.684145 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:39.702050 27182 raft_consensus.cc:1261] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260924 14:31:39.704502 26252 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:31:39.707763 31818 consensus_queue.cc:260] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:39.708616 32022 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.009s
W20260924 14:31:39.744304 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52433.3us
I20260924 14:31:39.712412 27186 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "898441fbc17a42e09c7639e00843680b"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:39.714198 31834 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:39.716022 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:39.728209 27194 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260924 14:31:39.655165 27185 rpcz_store.cc:276] 0924 14:31:37.320752 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.320898 (+   146us) service_pool.cc:227] Handling call
0924 14:31:39.599911 (+2279013us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b540031f954. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:39.622551 (+ 22640us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1697223424}
W20260924 14:31:39.735208 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 79422us
I20260924 14:31:39.733294 31985 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:39.751981 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:39.753911 32115 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:39.754307 32118 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:31:39.756361 27186 raft_consensus.cc:595] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:39.746270 32021 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:39.745999 31834 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:39.777823 31845 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:39.780335 32066 log.cc:924] Time spent T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.277s	user 0.000s	sys 0.005s
I20260924 14:31:39.781360 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:39.781819 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:39.784631 27181 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:31:39.785868 27189 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:39.759979 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:39.778743 31845 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:39.798297 31985 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.803722 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:39.804075 31882 consensus_meta.cc:287] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.565s	user 0.013s	sys 0.007s
I20260924 14:31:39.805704 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:39.805970 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 53356.7us
W20260924 14:31:39.806527 31895 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.899s	user 0.008s	sys 0.015s
I20260924 14:31:39.808182 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:39.809454 32115 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:39.812423 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:39.808408 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:39.813278 31521 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.765s	user 0.013s	sys 0.008s
W20260924 14:31:39.813915 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 98164.7us
I20260924 14:31:39.755507 31892 consensus_queue.cc:260] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.846735 27182 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:39.757586 32022 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.841630 31431 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14.
I20260924 14:31:39.838145 31980 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:39.861343 26249 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:39.863056 26406 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:39.864310 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:39.864516 32022 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:39.865407 31895 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:39.865758 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:39.866075 31895 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:39.868363 31895 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:39.869956 31845 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:39.871699 32034 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:39.872066 26405 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:39.872347 32034 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:39.874563 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:39.874893 32034 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:39.875631 32089 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:39.877110 32075 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:39.877322 26397 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:39.877509 32075 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:39.878293 32116 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.119s
I20260924 14:31:39.878734 26402 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:39.881201 32043 consensus_meta.cc:287] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.522s	user 0.017s	sys 0.002s
W20260924 14:31:39.882360 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 77483.9us
I20260924 14:31:39.883258 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:39.884382 31699 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:39.885090 31699 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:39.878896 32075 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:39.886982 32075 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:39.757218 32091 consensus_queue.cc:1035] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260924 14:31:39.849526 31521 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:39.869982 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:39.909544 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:39.854444 31882 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:39.934512 31834 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:39.966084 31834 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:39.966549 31882 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:39.969812 32036 consensus_meta.cc:287] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.003s
I20260924 14:31:39.913055 26258 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:31:39.922253 31346 consensus_meta.cc:287] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.694s	user 0.009s	sys 0.008s
I20260924 14:31:39.892899 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:39.985898 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 79752.2us
I20260924 14:31:39.991148 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:39.970968 31974 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:39.940586 31985 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
W20260924 14:31:40.013186 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 79708.7us
I20260924 14:31:40.014879 26256 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:39.931697 31828 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:39.887365 31699 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:40.032518 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 47577.4us
I20260924 14:31:40.033540 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1187) took 1130 ms. Trace:
I20260924 14:31:40.033721 31890 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:40.034227 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:39.908054 31980 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260924 14:31:40.036638 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.023118 26403 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:40.034087 26327 rpcz_store.cc:276] 0924 14:31:38.902769 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:38.918720 (+ 15951us) service_pool.cc:227] Handling call
0924 14:31:40.033254 (+1114534us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b5400498754. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:40.033452 (+   198us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1064862592}
I20260924 14:31:40.016718 31954 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:40.050666 26251 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
W20260924 14:31:40.051218 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 168032us
I20260924 14:31:40.052851 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:40.065227 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 48492.5us
I20260924 14:31:40.054997 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:40.067060 26254 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:40.018893 31346 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18.
I20260924 14:31:40.074280 31279 consensus_queue.cc:1046] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260924 14:31:40.075129 32019 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:40.075224 26330 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:39.892084 32021 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:40.064289 32091 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:40.081475 32019 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:40.098433 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:40.095336 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:40.100844 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:40.101189 31845 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:40.105757 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:40.107640 31547 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:40.107820 31279 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:40.108112 31974 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:40.108590 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47028us
I20260924 14:31:40.109726 31279 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:40.110141 31279 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:40.110471 31279 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:40.115818 26400 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260924 14:31:40.116376 26400 pending_rounds.cc:76] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 9
I20260924 14:31:40.116778 26400 pending_rounds.cc:97] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260924 14:31:40.117138 26400 raft_consensus.cc:2866] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:31:40.117314 32123 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260924 14:31:40.118847 31521 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:40.095551 32019 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:40.128407 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:40.130775 26221 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:40.130872 31991 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:40.131091 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.131374 31893 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:40.072207 32024 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260924 14:31:40.123945 32091 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:40.143982 31279 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:40.145210 32091 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:40.145529 31875 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:40.147967 31431 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:40.108130 31547 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:40.153462 31547 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:40.154472 31933 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Stepping down as leader of term 18
I20260924 14:31:40.154918 31933 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:40.155925 31933 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:40.157147 31933 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:31:40.158948 32086 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.125s	user 0.003s	sys 0.000s
I20260924 14:31:40.161298 31845 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:40.161837 31845 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:40.162310 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1177) took 4976 ms. Trace:
W20260924 14:31:40.164168 26326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1205) took 1819 ms (client timeout 1971 ms). Trace:
I20260924 14:31:40.164921 31279 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:40.175041 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:40.175513 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:40.176628 26399 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:40.180147 31997 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:40.181124 31989 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:40.167943 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.182179 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1340) took 1601 ms. Trace:
I20260924 14:31:40.166918 26330 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:40.169531 31995 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:40.182920 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.183768 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:40.184357 26403 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:40.184629 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.185359 26327 raft_consensus.cc:3037] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:40.180814 31997 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:40.162719 26329 rpcz_store.cc:276] 0924 14:31:35.185592 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:35.967396 (+781804us) service_pool.cc:227] Handling call
0924 14:31:35.967833 (+   437us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:38.781302 (+2813469us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.161288 (+1379986us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b540040a720. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b92de730 00007f09b9351b42 00007f09b934df26 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772
0924 14:31:40.161895 (+   607us) spinlock_profiling.cc:243] Waited 2.79 s on lock 0x7b54005582d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:40.162106 (+   211us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6412547712}
I20260924 14:31:40.170095 31623 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:40.164719 26326 rpcz_store.cc:269] 0924 14:31:38.344737 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:38.391306 (+ 46569us) service_pool.cc:227] Handling call
0924 14:31:40.163756 (+1772450us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b5400505854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:40.163958 (+   202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1760167808}
I20260924 14:31:40.188401 31623 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:40.189141 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.189348 31997 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:40.099812 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.187696 31892 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:40.183290 31995 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:40.182642 26257 rpcz_store.cc:276] 0924 14:31:38.580502 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:38.580674 (+   172us) service_pool.cc:227] Handling call
0924 14:31:38.602262 (+ 21588us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:38.622089 (+ 19827us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:38.622134 (+    45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:38.635312 (+ 13178us) log.cc:842] Serialized 26 byte log entry
0924 14:31:38.881292 (+245980us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:38.881335 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:39.129008 (+247673us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b4800000fd8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:31:40.181696 (+1052688us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b54002882d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:40.181766 (+    70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:40.181840 (+    74us) raft_consensus.cc:1671] finished
0924 14:31:40.181853 (+    13us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:40.182006 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":6682,"raft.run_cpu_time_us":2785,"raft.run_wall_time_us":494599,"spinlock_wait_cycles":70223104,"thread_start_us":195114,"threads_started":1,"wal-append.queue_time_us":362929,"wal-append.run_cpu_time_us":2435,"wal-append.run_wall_time_us":1035141}
I20260924 14:31:40.193527 31623 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:40.194171 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:40.195602 31890 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:40.196300 31890 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:40.192338 31995 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:40.197304 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 66605.3us
W20260924 14:31:40.199677 26328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54090 (request call id 1202) took 1857 ms (client timeout 2066 ms). Trace:
W20260924 14:31:40.200075 26328 rpcz_store.cc:269] 0924 14:31:38.341746 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:38.342635 (+   889us) service_pool.cc:227] Handling call
0924 14:31:40.199243 (+1856608us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b54007ce7d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:40.199522 (+   279us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":97370368}
I20260924 14:31:40.201190 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.202425 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.203408 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:40.203658 31979 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:40.204933 31979 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:40.205688 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:40.205864 31979 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:40.207563 31005 raft_consensus.cc:3032] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:40.207991 31005 raft_consensus.cc:729] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:40.208974 31005 consensus_queue.cc:260] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:40.203519 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:40.211567 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:40.190115 26326 raft_consensus.cc:2396] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260924 14:31:40.215878 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 153640us
I20260924 14:31:40.216174 32043 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:40.216696 32043 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:40.217969 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46994.5us
I20260924 14:31:40.190099 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.222926 26325 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:40.219059 26295 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:40.218972 32043 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:40.236760 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:40.240628 31678 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:31:40.240924 31623 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:40.219280 31997 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:40.241490 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:40.242677 26251 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:40.243286 31945 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:40.243903 31945 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:40.246094 31945 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:40.241654 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.248122 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.249002 26326 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 19: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:40.249799 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.250893 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.252544 26249 raft_consensus.cc:1226] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:40.224926 26403 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:40.256747 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:40.262554 26405 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260924 14:31:40.269230 26326 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:40.271996 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:40.273144 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:40.273888 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:40.263128 26405 pending_rounds.cc:76] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 10
I20260924 14:31:40.275676 26405 pending_rounds.cc:97] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260924 14:31:40.276088 26405 raft_consensus.cc:2866] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:40.229970 31890 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:40.288272 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.286052 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.290073 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:40.286692 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:40.293298 26252 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
W20260924 14:31:40.293653 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 63737.2us
I20260924 14:31:40.295181 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:40.295383 31005 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:40.296573 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:31:40.297566 32132 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.078s	user 0.004s	sys 0.000s
I20260924 14:31:40.303470 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.163185 31845 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:40.272612 31279 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:40.305029 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.293687 26252 pending_rounds.cc:76] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 6
I20260924 14:31:40.310130 26252 pending_rounds.cc:97] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260924 14:31:40.310530 26252 raft_consensus.cc:2866] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:40.310055 26327 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:40.254469 32019 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:40.313699 31890 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:40.318719 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.319757 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.318766 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.322826 26326 raft_consensus.cc:2396] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260924 14:31:40.324182 32134 log.cc:924] Time spent T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.004s
I20260924 14:31:40.324579 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:40.326031 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.330235 32019 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:40.330957 27181 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:40.339497 31895 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:40.340363 31895 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:40.342574 26254 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
W20260924 14:31:40.343627 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 60770.7us
I20260924 14:31:40.347486 31039 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
W20260924 14:31:40.351907 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 89149.9us
I20260924 14:31:40.345537 31971 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:40.357038 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 60567.7us
I20260924 14:31:40.357168 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.366482 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.351922 31678 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:40.369894 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 95633.6us
I20260924 14:31:40.365665 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.385071 27188 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 1013) took 2978 ms. Trace:
I20260924 14:31:40.385567 27188 rpcz_store.cc:276] 0924 14:31:37.406545 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:37.417441 (+ 10896us) service_pool.cc:227] Handling call
0924 14:31:37.417796 (+   355us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:37.537934 (+120138us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:37.537974 (+    40us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:37.625241 (+ 87267us) log.cc:842] Serialized 26 byte log entry
0924 14:31:38.611343 (+986102us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b180023fda0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:31:38.771519 (+160176us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:38.771559 (+    40us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:38.771632 (+    73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:40.158006 (+1386374us) spinlock_profiling.cc:243] Waited 8.69 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:40.355291 (+197285us) raft_consensus.cc:1671] finished
0924 14:31:40.355320 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:40.384889 (+ 29569us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":197812,"raft.queue_time_us":146295,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":51194,"spinlock_wait_cycles":23512064,"thread_start_us":159948,"threads_started":1,"wal-append.queue_time_us":1308577}
I20260924 14:31:40.380275 31039 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:40.388562 31039 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:40.389148 31677 raft_consensus.cc:979] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:31:40.389819 31677 raft_consensus.cc:1067] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:31:40.367921 31971 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:40.393301 31547 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:40.393940 31931 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:40.321475 31279 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:40.394764 31931 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:31:40.388592 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 125215us
I20260924 14:31:40.398200 27182 raft_consensus.cc:2354] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15.
I20260924 14:31:40.381664 26329 raft_consensus.cc:2362] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16. Re-sending same reply.
I20260924 14:31:40.439224 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.442924 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.444101 26329 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:40.377285 31997 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:40.467815 27192 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260924 14:31:40.464443 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57195.6us
I20260924 14:31:40.460561 26403 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5581950b69104df485f842c6b78b1047"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:35444
I20260924 14:31:40.473946 26403 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:31:40.474485 26403 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:40.481060 27194 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:40.493608 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.468602 31677 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:40.501044 26329 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:40.501945 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:40.509544 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44358.8us
I20260924 14:31:40.512768 26327 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:40.499740 31996 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:40.529562 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.522559 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.529294 26398 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:40.531571 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:40.532160 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:40.532908 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:40.533514 26251 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 15
I20260924 14:31:40.533643 31818 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:40.534129 31971 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:40.511085 31933 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:40.534793 31547 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:40.535557 32021 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:40.536315 31933 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:40.536831 31933 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:40.537982 31996 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:40.538482 31996 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:40.502985 31677 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:40.545885 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1200) took 2207 ms. Trace:
I20260924 14:31:40.507959 31931 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:40.546315 26331 rpcz_store.cc:276] 0924 14:31:38.338357 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:38.338494 (+   137us) service_pool.cc:227] Handling call
0924 14:31:38.338896 (+   402us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:38.573449 (+234553us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:38.573498 (+    49us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:38.573558 (+    60us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:38.573599 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.545491 (+1971892us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b540001e5d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:40.545559 (+    68us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:40.545716 (+   157us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":538817280}
I20260924 14:31:40.547710 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.548404 26331 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:40.549482 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:40.552030 32136 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260924 14:31:40.550840 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.514113 26402 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Advancing to term 23
I20260924 14:31:40.507993 32085 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:40.583786 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.584939 26329 raft_consensus.cc:2396] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260924 14:31:40.598697 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.507953 32050 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.238s
I20260924 14:31:40.512320 31346 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.080s
I20260924 14:31:40.611552 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.594658 31875 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:40.613649 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:40.584990 32085 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 19 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:40.617542 26256 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:40.624773 26331 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 18: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:40.615749 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:40.626199 27125 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:40.620581 26403 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:40.585489 32089 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260924 14:31:40.615101 26329 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:40.607126 31279 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:40.608554 32021 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:40.641662 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:40.652051 27189 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50480 (request call id 1534) took 2023 ms (client timeout 2024 ms). Trace:
I20260924 14:31:40.655843 31677 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:40.666098 31677 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:40.666625 31677 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:40.641319 32098 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.499s	user 0.005s	sys 0.000s
I20260924 14:31:40.668761 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1711) took 1073 ms. Trace:
I20260924 14:31:40.669188 26329 rpcz_store.cc:276] 0924 14:31:39.595156 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.641374 (+1046218us) service_pool.cc:227] Handling call
0924 14:31:40.668679 (+ 27305us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:40.637164 32028 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260924 14:31:40.670265 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.670493 31005 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:40.670890 31005 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:40.671113 26329 raft_consensus.cc:2396] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260924 14:31:40.646888 26328 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
W20260924 14:31:40.663736 27189 rpcz_store.cc:269] 0924 14:31:38.628177 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:38.647984 (+ 19807us) service_pool.cc:227] Handling call
0924 14:31:40.651806 (+2003822us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540000df54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:40.651961 (+   155us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2496283392}
I20260924 14:31:40.677875 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1181) took 1798 ms. Trace:
I20260924 14:31:40.633710 31266 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:40.677723 31005 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:40.675403 26403 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:40.678601 31266 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:40.682408 31266 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:40.682824 31005 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:40.683315 31677 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:40.684504 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.685482 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1691) took 1227 ms. Trace:
I20260924 14:31:40.685859 26329 rpcz_store.cc:276] 0924 14:31:39.457834 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.684215 (+1226381us) service_pool.cc:227] Handling call
0924 14:31:40.685407 (+  1192us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:40.686877 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.694926 26327 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:40.698155 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.695940 26329 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:40.691782 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:40.702975 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 206395us
I20260924 14:31:40.703866 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.678287 26324 rpcz_store.cc:276] 0924 14:31:38.879231 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:38.879365 (+   134us) service_pool.cc:227] Handling call
0924 14:31:38.879732 (+   367us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:38.880149 (+   417us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:38.880193 (+    44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:38.894578 (+ 14385us) log.cc:842] Serialized 26 byte log entry
0924 14:31:39.033377 (+138799us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:39.033430 (+    53us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:39.033529 (+    99us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:40.640411 (+1606882us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:40.671921 (+ 31510us) raft_consensus.cc:1671] finished
0924 14:31:40.671950 (+    29us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:40.677682 (+  5732us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5751,"raft.queue_time_us":1046,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":11887,"spinlock_wait_cycles":25238784,"thread_start_us":138429,"threads_started":1,"wal-append.queue_time_us":233225}
I20260924 14:31:40.708283 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.711745 26327 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:40.712873 26331 raft_consensus.cc:2379] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:40.690668 32021 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:40.653124 32043 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:40.728330 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:40.731302 26329 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:40.732587 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.734087 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1690) took 1276 ms. Trace:
I20260924 14:31:40.734557 26329 rpcz_store.cc:276] 0924 14:31:39.457435 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.707978 (+1250543us) service_pool.cc:227] Handling call
0924 14:31:40.734008 (+ 26030us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:40.735615 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.736203 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.731395 27189 raft_consensus.cc:595] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:40.737775 27189 raft_consensus.cc:603] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:40.742175 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:40.755211 26331 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:40.757617 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 431628us
I20260924 14:31:40.759793 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.767910 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 1848) took 1522 ms. Trace:
I20260924 14:31:40.768397 26406 rpcz_store.cc:276] 0924 14:31:39.245194 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:39.246351 (+  1157us) service_pool.cc:227] Handling call
0924 14:31:40.767657 (+1521306us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b5400691ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:40.767810 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1374326144}
W20260924 14:31:40.772387 32138 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260924 14:31:40.776269 31852 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:40.777029 31852 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:40.778053 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1706) took 1185 ms. Trace:
I20260924 14:31:40.778484 26330 rpcz_store.cc:276] 0924 14:31:39.592760 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.130759 (+537999us) service_pool.cc:227] Handling call
0924 14:31:40.777829 (+647070us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b5400498254. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:40.777974 (+   145us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":80632576}
I20260924 14:31:40.779948 26330 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:40.767910 31346 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:40.780614 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1697) took 1307 ms. Trace:
I20260924 14:31:40.780903 31346 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:40.782720 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1708) took 1189 ms. Trace:
I20260924 14:31:40.783162 26328 rpcz_store.cc:276] 0924 14:31:39.593771 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.202111 (+608340us) service_pool.cc:227] Handling call
0924 14:31:40.782501 (+580390us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400498c54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:40.782645 (+   144us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":445679488}
I20260924 14:31:40.784607 26328 raft_consensus.cc:1226] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260924 14:31:40.785209 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1701) took 1310 ms. Trace:
W20260924 14:31:40.785470 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 243493us
I20260924 14:31:40.781097 26330 rpcz_store.cc:276] 0924 14:31:39.473206 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.779382 (+1306176us) service_pool.cc:227] Handling call
0924 14:31:40.779761 (+   379us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:40.780360 (+   599us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.780534 (+   174us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:40.786468 32043 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:40.788326 32139 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260924 14:31:40.788549 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 83423.2us
I20260924 14:31:40.789355 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:40.789692 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:40.790234 31890 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260924 14:31:40.790688 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 129371us
I20260924 14:31:40.791061 31890 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:40.791692 31890 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:40.792739 31890 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:40.797125 31995 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:40.785496 26328 rpcz_store.cc:276] 0924 14:31:39.474545 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.783998 (+1309453us) service_pool.cc:227] Handling call
0924 14:31:40.784389 (+   391us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:40.784965 (+   576us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.785155 (+   190us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:40.796809 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.752907 32085 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:40.795591 31845 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:40.802803 31346 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:40.799263 26328 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260924 14:31:40.789372 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 2065) took 1013 ms. Trace:
I20260924 14:31:40.813556 26329 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:40.815347 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1702) took 1329 ms. Trace:
I20260924 14:31:40.810525 26252 rpcz_store.cc:276] 0924 14:31:39.776063 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:39.792649 (+ 16586us) service_pool.cc:227] Handling call
0924 14:31:39.805043 (+ 12394us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:40.312850 (+507807us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.789139 (+476289us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b5400716454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:40.789313 (+   174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1115791616}
I20260924 14:31:40.815862 26328 rpcz_store.cc:276] 0924 14:31:39.474876 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.798445 (+1323569us) service_pool.cc:227] Handling call
0924 14:31:40.798856 (+   411us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:40.804187 (+  5331us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.804455 (+   268us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:40.756654 26331 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260924 14:31:40.830628 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.841995 26251 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:40.846017 26403 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260924 14:31:40.834798 26327 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:40.848052 26330 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:40.849327 31720 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260924 14:31:40.849643 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.837396 31852 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:40.850849 31720 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:40.850996 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.837394 31875 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260924 14:31:40.851007 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:40.839344 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:40.803387 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:40.858130 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1689) took 1380 ms. Trace:
I20260924 14:31:40.859279 26329 rpcz_store.cc:276] 0924 14:31:39.456965 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.754278 (+1297313us) service_pool.cc:227] Handling call
0924 14:31:40.807796 (+ 53518us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:40.836918 (+ 29122us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.837169 (+   251us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:40.858716 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1693) took 1379 ms. Trace:
I20260924 14:31:40.861850 26331 rpcz_store.cc:276] 0924 14:31:39.458604 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.756006 (+1297402us) service_pool.cc:227] Handling call
0924 14:31:40.756421 (+   415us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:40.838137 (+ 81716us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.838354 (+   217us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:40.816149 32050 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Leader election lost for term 17. Reason: could not achieve majority
W20260924 14:31:40.886596 26332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1698) took 1413 ms (client timeout 1694 ms). Trace:
I20260924 14:31:40.810779 32021 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:40.884286 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:40.890005 26328 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:40.890069 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:40.887235 26332 rpcz_store.cc:269] 0924 14:31:39.473586 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:39.581320 (+107734us) service_pool.cc:227] Handling call
0924 14:31:40.803111 (+1221791us) spinlock_profiling.cc:243] Waited 9.28 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:31:40.886522 (+ 83411us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":21355264}
W20260924 14:31:40.894555 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 134862us
I20260924 14:31:40.890777 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1192) took 1010 ms. Trace:
I20260924 14:31:40.898988 26328 rpcz_store.cc:276] 0924 14:31:39.879973 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.841252 (+961279us) service_pool.cc:227] Handling call
0924 14:31:40.884341 (+ 43089us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:40.890480 (+  6139us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.890710 (+   230us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:40.901712 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.903306 26402 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 23 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260924 14:31:40.905781 27123 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:40.916282 26331 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260924 14:31:40.916917 26331 pending_rounds.cc:76] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 9
I20260924 14:31:40.917181 26331 pending_rounds.cc:97] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260924 14:31:40.917485 26331 raft_consensus.cc:2866] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:40.871968 31892 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:40.940963 31677 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:40.941506 31677 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:40.941568 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:40.941915 31677 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:40.942296 27189 raft_consensus.cc:595] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:40.942688 27189 raft_consensus.cc:603] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:40.943657 26249 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:40.944873 32088 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:40.945287 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1198) took 2607 ms. Trace:
I20260924 14:31:40.945436 32088 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:40.946460 26369 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:40.945706 26323 rpcz_store.cc:276] 0924 14:31:38.337645 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:38.337811 (+   166us) service_pool.cc:227] Handling call
0924 14:31:38.361319 (+ 23508us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:38.361530 (+   211us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:38.396941 (+ 35411us) log.cc:842] Serialized 14 byte log entry
0924 14:31:40.184643 (+1787702us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b18003e0800. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd557b 00007f09b935888a
0924 14:31:40.944709 (+760066us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:40.944780 (+    71us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:40.944879 (+    99us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.944965 (+    86us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:40.945089 (+   124us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":36496,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":18063,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6586,"spinlock_wait_cycles":4674816,"thread_start_us":695841,"threads_started":2,"wal-append.queue_time_us":2245961}
I20260924 14:31:40.947522 32143 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:40.947950 32050 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:40.948036 31857 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:40.948542 32050 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:40.948640 31857 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:40.950835 32050 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:40.932717 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:40.884071 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:40.951663 31997 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:40.952105 31997 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:40.952860 31623 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:40.952993 31997 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:40.953262 32088 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:40.954701 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:31:40.955876 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1707) took 1362 ms. Trace:
I20260924 14:31:40.956305 26330 rpcz_store.cc:276] 0924 14:31:39.593315 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.799550 (+1206235us) service_pool.cc:227] Handling call
0924 14:31:40.847855 (+ 48305us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:40.855081 (+  7226us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.955645 (+100564us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b54005baed0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:40.955774 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":110241024}
I20260924 14:31:40.958698 31279 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:40.959059 32043 raft_consensus.cc:2781] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:40.959823 32043 raft_consensus.cc:686] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:40.960775 32043 consensus_queue.cc:237] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:40.961409 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:40.963392 31733 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:40.950891 31857 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:40.948918 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 64067.5us
I20260924 14:31:40.897066 31678 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
W20260924 14:31:40.966049 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 126802us
I20260924 14:31:40.964943 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:40.968029 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1191) took 1932 ms. Trace:
I20260924 14:31:40.972309 26324 rpcz_store.cc:276] 0924 14:31:39.037469 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.828447 (+1790978us) service_pool.cc:227] Handling call
0924 14:31:40.915247 (+ 86800us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:40.935221 (+ 19974us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:40.935261 (+    40us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:40.967689 (+ 32428us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:40.967748 (+    59us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.967827 (+    79us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:40.967965 (+   138us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260924 14:31:40.845625 32034 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260924 14:31:40.929463 31875 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:40.975819 26332 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:40.974089 32143 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:40.977249 31678 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:40.977596 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:40.981274 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:40.879846 26327 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 18. Ops: []
I20260924 14:31:40.989646 26328 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260924 14:31:40.987776 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 336027us
I20260924 14:31:40.977731 31266 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:40.994499 26403 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
W20260924 14:31:40.932081 31039 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.037s	sys 0.038s
I20260924 14:31:40.999711 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.001106 26252 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:40.964205 31980 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260924 14:31:41.003629 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:40.983728 31875 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:41.006562 31997 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:41.013012 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:40.933111 31933 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:41.018128 31933 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:41.021858 31678 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:41.027932 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:41.029018 26325 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 17: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:41.029522 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50660 (request call id 1194) took 1148 ms. Trace:
I20260924 14:31:41.029888 26325 rpcz_store.cc:276] 0924 14:31:39.880630 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.302038 (+421408us) service_pool.cc:227] Handling call
0924 14:31:41.027751 (+725713us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b580012e798. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63ee702 00007f09bf2deb09 00007f09be21a730 00007f09be2e079b 00007f09be2e0821 00007f09be2901b0 00007f09be28fa94 00007f09b7977c04 00007f09b7977b92
0924 14:31:41.029449 (+  1698us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6112000}
I20260924 14:31:41.030149 31266 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:41.031857 31677 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:41.045585 31677 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:40.983420 32143 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:41.046471 32143 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:41.054807 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:41.071578 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:41.072345 26252 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260924 14:31:41.076979 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 56722.6us
W20260924 14:31:40.998193 31039 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.038s	sys 0.039s
I20260924 14:31:41.077582 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:41.084143 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:41.085328 26254 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:41.086760 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 125404us
I20260924 14:31:41.087116 27186 raft_consensus.cc:2396] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260924 14:31:41.092612 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:41.071257 32028 raft_consensus.cc:979] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: : Instructing follower 45ab97b763804767846e9d27a01b561f to start an election
I20260924 14:31:41.100920 26251 raft_consensus.cc:2379] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:41.096992 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:41.109607 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:41.110816 26256 raft_consensus.cc:2379] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 18: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:41.110689 26295 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:41.119428 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:41.122963 26257 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:41.126472 27191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1550) took 1066 ms. Trace:
I20260924 14:31:41.132663 26398 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 19 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 18. Current term is 19. Ops: []
I20260924 14:31:41.119714 31857 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:41.114285 32019 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:41.122802 31852 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:41.146620 31767 raft_consensus.cc:1067] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Signalling peer 45ab97b763804767846e9d27a01b561f to start an election
W20260924 14:31:41.153081 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 57690.7us
I20260924 14:31:41.158651 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:41.160089 26332 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:41.161655 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:41.166112 26252 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260924 14:31:41.162366 26329 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:41.170243 26406 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:41.177631 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:41.178581 26330 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:41.136914 32034 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260924 14:31:41.205101 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:41.206112 26329 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 15
I20260924 14:31:41.144805 32028 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260924 14:31:41.198782 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:31:41.217859 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:41.224751 26251 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:41.242434 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:41.243901 26324 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:41.250204 26331 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:41.253916 32146 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260924 14:31:41.258430 32146 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:41.258809 32146 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:41.259768 32146 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:41.260921 32146 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:41.300204 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 146297us
I20260924 14:31:41.300993 26323 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:41.255064 31818 consensus_meta.cc:287] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.009s
I20260924 14:31:41.289764 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:41.310019 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.129151 27191 rpcz_store.cc:276] 0924 14:31:40.054384 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.054579 (+   195us) service_pool.cc:227] Handling call
0924 14:31:40.068565 (+ 13986us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:40.092500 (+ 23935us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:40.092540 (+    40us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:40.127067 (+ 34527us) log.cc:842] Serialized 26 byte log entry
0924 14:31:40.308417 (+181350us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b18002da0c0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:31:40.699414 (+390997us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:41.090067 (+390653us) log.cc:842] Serialized 14 byte log entry
0924 14:31:41.113448 (+ 23381us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:41.113555 (+   107us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:41.113615 (+    60us) raft_consensus.cc:1671] finished
0924 14:31:41.120104 (+  6489us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:41.120262 (+   158us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":90542,"raft.queue_time_us":27040,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":17137,"spinlock_wait_cycles":57746304,"thread_start_us":726966,"threads_started":2,"wal-append.queue_time_us":822131}
I20260924 14:31:41.323825 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:41.323972 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 74875.9us
W20260924 14:31:41.327695 32145 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260924 14:31:41.329355 27190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 997) took 1104 ms. Trace:
I20260924 14:31:41.329733 27190 rpcz_store.cc:276] 0924 14:31:40.224824 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.236323 (+ 11499us) service_pool.cc:227] Handling call
0924 14:31:40.277792 (+ 41469us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:40.322734 (+ 44942us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:40.322801 (+    67us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:40.350669 (+ 27868us) log.cc:842] Serialized 26 byte log entry
0924 14:31:40.904433 (+553764us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:40.904484 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:40.907225 (+  2741us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:41.329118 (+421893us) raft_consensus.cc:1671] finished
0924 14:31:41.329140 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:41.329293 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25440,"raft.queue_time_us":39844,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7984,"thread_start_us":425468,"threads_started":1,"wal-append.queue_time_us":744053}
I20260924 14:31:41.340747 26404 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260924 14:31:41.347611 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:41.341269 26404 pending_rounds.cc:76] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 9
I20260924 14:31:41.349905 26404 pending_rounds.cc:97] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260924 14:31:41.348549 27190 raft_consensus.cc:2454] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:41.350643 26254 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:41.339598 31346 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:41.350271 26404 raft_consensus.cc:2866] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:31:41.356350 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 48592.5us
I20260924 14:31:41.357236 26369 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:41.359238 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:41.361709 32085 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260924 14:31:41.362099 32149 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260924 14:31:41.362258 32085 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:41.362670 32085 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:41.366330 31931 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:41.386230 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1840) took 2155 ms. Trace:
I20260924 14:31:41.386739 26404 rpcz_store.cc:276] 0924 14:31:39.230616 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:39.250421 (+ 19805us) service_pool.cc:227] Handling call
0924 14:31:39.250726 (+   305us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:41.363688 (+2112962us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:41.385903 (+ 22215us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b54003d59d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:41.386142 (+   239us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4739335424}
I20260924 14:31:41.363933 31895 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:41.392468 26329 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:41.407857 31678 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:41.410188 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:41.413300 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:41.393230 31279 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:41.364457 31346 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:41.395175 31818 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260924 14:31:41.450138 26400 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "898441fbc17a42e09c7639e00843680b"
dest_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:35444
I20260924 14:31:41.450843 26400 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:31:41.451231 26400 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:41.404758 31991 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:41.396090 31895 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:41.459220 31857 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:41.451900 31039 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:41.458423 32153 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:41.464573 31996 consensus_meta.cc:287] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.926s	user 0.018s	sys 0.006s
I20260924 14:31:41.414446 31678 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:41.465739 31931 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:41.500674 31767 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:41.497841 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1729) took 1170 ms. Trace:
I20260924 14:31:41.505409 26328 rpcz_store.cc:276] 0924 14:31:40.327090 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.899895 (+572805us) service_pool.cc:227] Handling call
0924 14:31:40.989339 (+ 89444us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:40.990010 (+   671us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:41.497554 (+507544us) spinlock_profiling.cc:243] Waited 51.5 ms on lock 0x7b5400237650. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:41.497750 (+   196us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":118394624}
I20260924 14:31:41.465689 31279 consensus_queue.cc:1046] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
W20260924 14:31:41.508632 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 117718us
I20260924 14:31:41.460192 31991 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:41.494199 31931 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:41.521260 26327 raft_consensus.cc:1226] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:41.503841 32143 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:41.538079 26403 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260924 14:31:41.491773 32153 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:41.489728 31857 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:41.541064 26369 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:41.492820 31996 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:41.549273 26404 raft_consensus.cc:1226] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:41.550772 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:41.537189 31931 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:41.495807 31039 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:41.567871 32160 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260924 14:31:41.568387 26405 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 9. Current term is 10. Ops: []
I20260924 14:31:41.570230 27194 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260924 14:31:41.581360 26330 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:41.585704 27194 raft_consensus.cc:1203] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Deduplicated request from leader. Original: 12.9->[13.10-13.10]   Dedup: 13.10->[]
I20260924 14:31:41.586089 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:41.586522 27194 raft_consensus.cc:1226] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260924 14:31:41.580526 32164 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:41.588429 27190 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260924 14:31:41.583907 26332 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:41.620683 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:41.621275 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.622673 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.623333 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.623982 26250 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:41.624150 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.624960 26258 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 18. Current term is 19. Ops: []
I20260924 14:31:41.625406 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.621412 26254 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:41.624239 26256 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260924 14:31:41.628541 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.637751 26295 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:41.592885 31346 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:41.640623 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:41.641805 26332 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:41.598513 32164 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
W20260924 14:31:41.647905 26297 leader_election.cc:334] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.774s (SENT)
I20260924 14:31:41.650767 26297 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:41.632961 31279 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:41.666497 26254 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
W20260924 14:31:41.649868 26371 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.549s (SENT) [suppressed 22 similar messages]
I20260924 14:31:41.670328 26327 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260924 14:31:41.650015 32160 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:41.649284 32148 raft_consensus.cc:2741] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260924 14:31:41.674126 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1901) took 1265 ms. Trace:
I20260924 14:31:41.674602 26405 rpcz_store.cc:276] 0924 14:31:40.408841 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.408988 (+   147us) service_pool.cc:227] Handling call
0924 14:31:40.473162 (+ 64174us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:41.569466 (+1096304us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:41.673863 (+104397us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000997d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:41.674061 (+   198us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2500834176}
I20260924 14:31:41.676955 31346 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:41.646198 31997 consensus_queue.cc:1046] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260924 14:31:41.680001 32143 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:41.680507 32043 raft_consensus.cc:3032] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:41.680938 32043 raft_consensus.cc:729] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:41.681499 32164 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:41.681945 32043 consensus_queue.cc:260] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:41.684465 32158 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260924 14:31:41.681687 32160 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:41.686187 32161 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260924 14:31:41.687330 32160 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:41.686290 32163 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260924 14:31:41.685191 31991 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:41.689429 26401 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:41.636390 31767 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260924 14:31:41.655651 31279 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:41.694739 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 76920.1us
I20260924 14:31:41.695472 27125 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:41.659524 31678 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:41.683387 32043 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:41.709408 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:41.697611 27125 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:41.687357 31857 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:41.717617 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1905) took 1147 ms. Trace:
I20260924 14:31:41.718092 26398 rpcz_store.cc:276] 0924 14:31:40.570170 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.570358 (+   188us) service_pool.cc:227] Handling call
0924 14:31:40.592399 (+ 22041us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:41.138664 (+546265us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:41.717379 (+578715us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b540035cfd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:41.717556 (+   177us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1242037120}
I20260924 14:31:41.681145 31996 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:41.720391 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:41.652581 32166 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260924 14:31:41.698863 31279 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:41.721467 32143 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:41.702280 32160 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:41.736491 26400 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:41.754191 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:41.757795 26406 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:41.721704 32022 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:41.756464 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:41.758773 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 108923us
I20260924 14:31:41.757714 26405 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:41.764869 27126 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:41.720844 32153 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:41.759294 26398 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:41.744190 31995 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:41.773201 26400 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:41.777376 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 182965us
I20260924 14:31:41.778913 26400 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:41.760946 32022 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:41.758181 31875 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:41.806272 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.778258 32160 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:41.763203 32143 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:41.808163 26257 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
I20260924 14:31:41.810158 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:41.785187 32022 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:41.819208 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.820006 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:41.807120 26254 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:41.785511 31039 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:41.796272 31875 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:41.838891 32179 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260924 14:31:41.838507 26256 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:41.838500 31994 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:41.777128 32166 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260924 14:31:41.855232 26400 raft_consensus.cc:2396] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260924 14:31:41.816452 31996 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:41.829716 31924 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:41.841395 32019 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260924 14:31:41.868208 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 103418us
W20260924 14:31:41.851156 32146 consensus_meta.cc:287] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.590s	user 0.007s	sys 0.003s
I20260924 14:31:41.856346 32189 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:41.880169 32189 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:41.883611 32189 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:41.891094 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 92717.5us
W20260924 14:31:41.899936 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 113055us
I20260924 14:31:41.838905 31875 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:41.904799 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 262835us
I20260924 14:31:41.910765 26256 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:41.912740 26221 leader_election.cc:398] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15.
I20260924 14:31:41.915540 26221 leader_election.cc:401] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:41.875763 32182 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:41.879199 32019 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:41.930099 27123 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:41.901974 31995 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:41.903407 32189 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:41.875581 31991 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260924 14:31:41.879359 31996 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:41.936441 26255 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:41.952557 31875 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:41.933097 32019 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:41.940945 32166 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:41.952610 31818 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:41.963699 32168 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260924 14:31:41.964758 31979 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:41.973992 31995 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:41.999366 31895 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.000007 32203 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:41.891999 31924 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:41.265048 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:42.004098 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:41.991461 32177 log.cc:924] Time spent T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260924 14:31:42.004760 27194 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
W20260924 14:31:42.007295 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 85815.7us
I20260924 14:31:42.013055 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:41.993319 31720 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election won for term 16
W20260924 14:31:42.022105 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 150654us
I20260924 14:31:42.022260 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:42.022948 27191 raft_consensus.cc:3032] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:42.023267 27191 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:42.022801 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.024398 27194 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:42.024871 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.026376 27186 raft_consensus.cc:1261] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260924 14:31:41.986207 31991 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260924 14:31:41.978432 32189 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:41.979372 32188 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:42.028492 27191 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:42.031091 26297 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.892s (SENT)
I20260924 14:31:41.932404 32160 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:41.935400 32182 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.007906 31818 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.041553 31996 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:42.041765 31818 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.043104 31991 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:42.043548 31991 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:42.044395 31991 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:42.045675 32205 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15.
I20260924 14:31:42.046500 32192 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:42.047084 32192 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.049252 32192 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:42.000629 32203 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.055955 32188 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.056908 32189 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:42.025480 31875 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:42.057911 32203 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:42.058312 31875 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.060415 31875 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:42.038755 31720 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:42.062284 31720 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:42.064553 31924 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.058214 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.045827 31991 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:42.062520 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:42.065997 26326 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:42.037158 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:42.077577 27185 raft_consensus.cc:3032] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:41.995494 32166 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.027840 32143 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:42.087637 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 163912us
I20260924 14:31:41.994107 32200 consensus_queue.cc:1046] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260924 14:31:42.078224 27185 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:42.077500 32085 consensus_meta.cc:287] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.714s	user 0.004s	sys 0.009s
I20260924 14:31:42.096468 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:42.096784 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:42.091959 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 67905.8us
W20260924 14:31:42.092617 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61633.5us
I20260924 14:31:42.093596 32205 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:42.098126 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:42.100346 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:42.100574 26249 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:42.101023 26328 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:42.089898 31852 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:42.101644 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:42.102243 26249 raft_consensus.cc:2362] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14. Re-sending same reply.
I20260924 14:31:42.102072 31852 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.103312 27185 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.104280 27185 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:42.105129 32085 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.107765 32205 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:42.110029 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:42.111828 31279 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.113746 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:42.114928 32085 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:42.115382 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.110030 32205 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:42.116782 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.117750 31279 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:42.118165 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:42.119266 26253 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:42.124466 26330 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260924 14:31:42.125276 26330 pending_rounds.cc:76] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 9
I20260924 14:31:42.125591 26330 pending_rounds.cc:97] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260924 14:31:42.125881 26330 raft_consensus.cc:2866] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:42.119882 32182 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:42.128646 26325 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:42.076857 32148 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:42.131603 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.124590 26328 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:42.131775 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.132200 27185 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:42.137481 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:42.138783 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.139779 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.140555 26252 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 12: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:42.140748 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:42.089288 32204 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:42.141371 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.135614 26257 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:42.146294 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:42.147214 26256 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:42.149051 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49917.8us
I20260924 14:31:42.155606 32153 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:42.162065 32200 raft_consensus.cc:3032] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:42.141901 32204 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:42.143211 31931 consensus_meta.cc:287] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.003s
I20260924 14:31:42.129302 31335 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:42.164052 27186 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:42.166628 32166 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.167392 31997 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Stepping down as leader of term 14
W20260924 14:31:42.170763 32170 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.514s	user 0.005s	sys 0.004s
I20260924 14:31:42.160779 26253 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:42.162338 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.174880 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:42.178413 32211 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:42.180544 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:42.202792 26251 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:42.086536 32019 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.204896 32019 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.141847 32022 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.192945 31931 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.206912 31931 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:42.207605 26399 raft_consensus.cc:1226] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:42.206909 32022 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.171669 32178 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Stepping down as leader of term 12
I20260924 14:31:42.209327 32178 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:42.184185 32192 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260924 14:31:42.214365 26251 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
W20260924 14:31:42.218508 32167 log.cc:924] Time spent T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.549s	user 0.002s	sys 0.000s
W20260924 14:31:42.221901 26326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1719) took 2421 ms (client timeout 1923 ms). Trace:
W20260924 14:31:42.222541 26326 rpcz_store.cc:269] 0924 14:31:39.800784 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.353567 (+552783us) service_pool.cc:227] Handling call
0924 14:31:42.221676 (+1868109us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54007cf6d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:42.221830 (+   154us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3927440256}
I20260924 14:31:42.224092 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:42.229271 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 63620.4us
I20260924 14:31:42.229487 31606 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:42.192755 31335 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:42.233263 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1914) took 1171 ms. Trace:
I20260924 14:31:42.234606 31995 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:42.233700 26402 rpcz_store.cc:276] 0924 14:31:41.061747 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:41.068693 (+  6946us) service_pool.cc:227] Handling call
0924 14:31:41.076776 (+  8083us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:41.099913 (+ 23137us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:41.099973 (+    60us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:41.109024 (+  9051us) log.cc:842] Serialized 26 byte log entry
0924 14:31:41.874928 (+765904us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b1800474880. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd5411 00007f09b93225cf
0924 14:31:41.905193 (+ 30265us) raft_consensus.cc:1631] Marking committed up to 7
0924 14:31:41.905295 (+   102us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:41.905413 (+   118us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:42.232938 (+327525us) raft_consensus.cc:1671] finished
0924 14:31:42.232960 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:42.233199 (+   239us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":54777,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":5332,"spinlock_wait_cycles":5372288,"thread_start_us":29910,"threads_started":1,"wal-append.queue_time_us":1007064}
I20260924 14:31:42.235206 31995 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.237455 31995 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:42.238247 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 138433us
I20260924 14:31:42.240864 32171 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260924 14:31:42.244705 32173 log.cc:924] Time spent T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.571s	user 0.001s	sys 0.001s
I20260924 14:31:42.244843 26253 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:42.245030 32215 consensus_queue.cc:1046] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:42.245895 32188 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:42.246244 32184 raft_consensus.cc:3032] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:42.246668 32184 raft_consensus.cc:729] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:42.247552 32184 consensus_queue.cc:260] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.248878 32184 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:42.250790 26406 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:42.250919 32178 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:42.252413 32178 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:42.253015 31335 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:42.253994 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:42.254639 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.255276 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:42.179376 32200 raft_consensus.cc:729] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:42.255925 26253 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:42.256634 32200 consensus_queue.cc:260] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:42.257200 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:42.257790 26397 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:42.258071 32200 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:42.258548 27189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1560) took 1264 ms. Trace:
I20260924 14:31:42.179793 32204 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:42.259608 32206 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:42.260129 32206 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.260898 31547 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:42.261413 31547 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.192878 31997 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:42.262209 32206 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:42.262984 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.258939 27189 rpcz_store.cc:276] 0924 14:31:40.994082 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:40.994399 (+   317us) service_pool.cc:227] Handling call
0924 14:31:41.015424 (+ 21025us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:41.075215 (+ 59791us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:41.075254 (+    39us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:41.105650 (+ 30396us) log.cc:842] Serialized 52 byte log entry
0924 14:31:41.483247 (+377597us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:41.483296 (+    49us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:41.483386 (+    90us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:42.217552 (+734166us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:42.258251 (+ 40699us) raft_consensus.cc:1671] finished
0924 14:31:42.258271 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:42.258482 (+   211us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":63155,"raft.queue_time_us":60899,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":14154,"spinlock_wait_cycles":44846336,"thread_start_us":348851,"threads_started":1,"wal-append.queue_time_us":491027}
I20260924 14:31:42.199955 32153 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:42.264302 32153 consensus_queue.cc:1225] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260924 14:31:42.264438 31852 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:42.264904 32153 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Stepping down as leader of term 18
I20260924 14:31:42.265276 32153 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:31:42.200872 32144 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.056s	user 0.000s	sys 0.008s
I20260924 14:31:42.230824 27191 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:42.195187 31980 consensus_queue.cc:1046] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260924 14:31:42.262843 31997 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.265887 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:42.277287 26251 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:42.277493 26257 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:42.266537 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1232) took 1097 ms. Trace:
I20260924 14:31:42.256065 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:42.263768 26402 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
W20260924 14:31:42.269349 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 94534.3us
I20260924 14:31:42.280926 26400 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 15
I20260924 14:31:42.200047 26221 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:42.194857 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 68827.1us
I20260924 14:31:42.275358 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:42.284284 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 84337us
I20260924 14:31:42.244551 31994 consensus_queue.cc:1046] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260924 14:31:42.286249 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.262980 31984 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:42.287375 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:42.289987 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.263294 31547 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.280614 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.290501 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.292307 26398 raft_consensus.cc:2379] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:42.291242 26405 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:42.290184 26253 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:42.278550 26331 rpcz_store.cc:276] 0924 14:31:41.168670 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:41.296190 (+127520us) service_pool.cc:227] Handling call
0924 14:31:41.312387 (+ 16197us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:41.332237 (+ 19850us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:41.332271 (+    34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:41.339755 (+  7484us) log.cc:842] Serialized 26 byte log entry
0924 14:31:41.580527 (+240772us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:41.636514 (+ 55987us) log.cc:842] Serialized 14 byte log entry
0924 14:31:41.636839 (+   325us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:41.636924 (+    85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:42.243980 (+607056us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:42.266244 (+ 22264us) raft_consensus.cc:1671] finished
0924 14:31:42.266362 (+   118us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:42.266484 (+   122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12471,"prepare.queue_time_us":58607,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":3069,"raft.queue_time_us":20004,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":8909,"spinlock_wait_cycles":34917248,"thread_start_us":203607,"threads_started":1,"wal-append.queue_time_us":301208}
I20260924 14:31:42.292646 26256 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:42.295712 26403 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:42.298841 26254 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:42.278331 32171 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:42.277938 31931 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:42.304615 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.309027 27126 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:42.310575 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:42.343689 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.348130 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:42.329854 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:42.333316 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49580.1us
I20260924 14:31:42.350385 26397 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:42.311005 26251 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:42.311836 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.290742 32206 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:42.290774 31997 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:42.297992 31852 raft_consensus.cc:3032] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Stepping down as leader of term 9
I20260924 14:31:42.381083 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:42.384694 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:42.384344 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 73893.4us
I20260924 14:31:42.380283 26250 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
W20260924 14:31:42.384205 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 76681.4us
I20260924 14:31:42.398954 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.313303 31931 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:42.394243 26400 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:42.333387 32211 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:42.268050 32153 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.384542 32085 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:42.402184 32075 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:42.410149 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.382071 31677 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:42.377728 32213 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:42.406605 27126 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:42.294196 32166 consensus_queue.cc:1035] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260924 14:31:42.291020 31890 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.106s
I20260924 14:31:42.381373 32022 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:42.396976 31991 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:42.419854 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.420271 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.397523 31852 raft_consensus.cc:729] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:42.463303 26397 raft_consensus.cc:2379] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:42.416649 32211 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:42.464169 31852 consensus_queue.cc:260] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.464402 32211 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.465521 31852 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 9 FOLLOWER]: Advancing to term 10
W20260924 14:31:42.467886 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 89769.5us
I20260924 14:31:42.468748 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.469580 32188 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:42.470068 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:42.470887 32188 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:42.304876 31984 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.427548 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:42.427799 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.403353 31945 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:42.430871 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.431133 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.333503 32171 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.433669 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:42.495416 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 61842.5us
I20260924 14:31:42.405802 31995 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:42.407229 31931 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:42.501770 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:42.379060 32194 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260924 14:31:42.402058 32206 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:42.446990 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.456157 26400 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:42.458806 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.478211 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.479969 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:42.478525 32153 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:42.427372 31677 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:42.434427 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:42.464438 32212 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260924 14:31:42.456643 31994 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:42.416891 32205 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260924 14:31:42.462606 32022 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:42.482790 26403 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:42.505560 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:42.426970 32085 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.513193 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:42.517293 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 89859.9us
I20260924 14:31:42.501961 31980 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:42.519757 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.520001 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:42.476156 31875 consensus_queue.cc:1046] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260924 14:31:42.520826 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.525296 26371 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:42.526602 26332 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
W20260924 14:31:42.526302 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 46495.2us
I20260924 14:31:42.514277 31677 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:42.530544 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:42.494583 32075 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260924 14:31:42.512015 31995 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:42.527014 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:42.517407 32022 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:42.520778 32188 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:42.524549 31980 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:42.454774 32213 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.536233 32199 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:42.542512 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:42.547789 26324 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:42.543167 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.554947 26400 raft_consensus.cc:2396] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:42.543599 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.548260 27194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.548447 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.530038 31994 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:42.530270 32164 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:42.553555 27191 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:42.541953 32143 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:42.549641 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:42.564267 26369 leader_election.cc:334] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.923s (SENT)
I20260924 14:31:42.564990 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:42.568109 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:42.568658 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:42.566346 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.572739 26323 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260924 14:31:42.549713 31980 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:42.573849 32182 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:42.580593 31688 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260924 14:31:42.566752 31924 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:42.567303 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.584353 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.570592 27188 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:31:42.591277 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.592593 27192 raft_consensus.cc:2379] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:42.595852 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.589190 32205 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:42.598598 31890 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:42.592471 27191 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:42.602349 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.603986 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:42.606240 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 128948us
I20260924 14:31:42.610437 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.611557 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.581651 31875 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:42.584129 32143 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:42.634579 27191 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260924 14:31:42.637704 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 87669.1us
I20260924 14:31:42.653669 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:42.628511 31733 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:42.616978 31346 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:42.616827 32225 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:42.569444 32206 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.642308 32188 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:42.639832 31995 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.682118 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.683212 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.685729 26401 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:42.689011 26257 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:42.693878 31980 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.697089 32143 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.699977 32085 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:42.700531 32198 raft_consensus.cc:3032] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:42.700968 32198 raft_consensus.cc:729] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:42.704216 32213 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:42.682648 32171 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:42.708146 27189 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:42.699997 27188 raft_consensus.cc:1226] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:42.681836 32188 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:42.715545 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 104122us
I20260924 14:31:42.716562 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.661310 31984 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.708798 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:42.712284 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:42.700088 31335 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.727035 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:42.714506 32225 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:42.737166 32225 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.740381 26406 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260924 14:31:42.752970 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.656059 31994 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.760457 26224 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:42.762388 32189 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:42.757863 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:42.763095 32189 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:42.763432 32204 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.000s
I20260924 14:31:42.763681 32189 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.763909 32204 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.766088 32189 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:42.767181 27186 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:42.758070 32225 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:42.768059 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:42.755333 31980 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:42.721349 32188 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:42.732661 32075 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:42.760097 31991 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:42.766165 32204 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.765976 31677 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.714000 32199 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:42.771214 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.759591 32186 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:42.763736 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:42.732300 31890 consensus_queue.cc:1046] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260924 14:31:42.776014 31852 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:42.817129 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 46010.6us
I20260924 14:31:42.821142 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.823030 27185 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 18. Current term is 19. Ops: []
I20260924 14:31:42.814559 32186 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:42.796335 31924 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:42.740268 32205 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.096s
I20260924 14:31:42.792522 32225 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260924 14:31:42.833725 32075 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election won for term 13
W20260924 14:31:42.830643 32178 consensus_meta.cc:287] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.001s
I20260924 14:31:42.821534 32235 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:42.837131 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:42.830410 32186 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:42.863598 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:42.857033 32075 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:42.867005 26327 raft_consensus.cc:2396] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260924 14:31:42.868131 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:42.889420 32022 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.891779 27182 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:42.860849 31852 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:42.898283 32085 raft_consensus.cc:465] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Not starting pre-election -- already a leader
W20260924 14:31:42.899308 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 176013us
I20260924 14:31:42.901351 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:42.901896 26254 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260924 14:31:42.866029 32235 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:42.914706 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.921066 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:42.907986 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:42.914297 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:42.922180 26405 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:42.923018 32189 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:42.906030 31924 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.923776 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:42.924638 26332 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:42.925725 31677 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.931064 32235 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.933311 32235 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:42.901430 32199 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:42.935284 32143 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:42.941694 32146 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:42.942163 32198 consensus_queue.cc:260] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:42.858850 31931 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:42.950731 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 50254.2us
I20260924 14:31:42.953213 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:42.885437 32153 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:42.960096 32022 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:42.962981 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:42.963743 26326 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:42.964046 32153 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.964342 26328 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260924 14:31:42.967790 26398 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:42.958346 32143 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.012s
I20260924 14:31:42.972288 26295 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:42.975811 31882 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:42.975924 32198 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:42.978773 31882 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:43.009009 31982 raft_consensus.cc:3032] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:43.009462 31982 raft_consensus.cc:729] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:43.010232 31982 consensus_queue.cc:260] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:42.967779 32186 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:43.011376 31982 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:43.013471 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 304761us
I20260924 14:31:43.016356 31984 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:43.016729 31984 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:43.017072 31984 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:43.016975 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 155439us
W20260924 14:31:43.026319 32245 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260924 14:31:43.028708 32153 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:43.029827 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:43.030560 26327 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:43.030889 31931 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:43.031909 32180 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:43.032337 32180 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:43.032549 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:42.995285 32075 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:43.037555 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:43.038004 32188 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:43.039636 26325 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:43.033157 32180 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:43.048079 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:43.051474 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:43.054390 26326 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:43.051702 26331 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 20: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:43.076097 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:43.092423 32188 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:43.057698 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:43.078697 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:43.059875 32180 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:43.104827 32192 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:43.105583 32192 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:31:43.106984 26326 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:43.007076 31882 raft_consensus.cc:3037] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:43.073212 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:43.033178 26256 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:43.110335 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:43.077523 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:43.100821 26331 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:43.059027 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:43.081099 26325 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260924 14:31:43.076481 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:43.068869 26405 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:43.123858 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:43.142409 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:43.229553 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 105868us
I20260924 14:31:43.194248 26324 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
W20260924 14:31:43.222244 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 149106us
I20260924 14:31:43.231400 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:43.233404 26327 raft_consensus.cc:1203] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 12.9->[14.10-14.10]   Dedup: 14.10->[]
I20260924 14:31:43.234097 26327 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260924 14:31:43.235070 26326 raft_consensus.cc:1226] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260924 14:31:43.238497 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
W20260924 14:31:43.276026 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46064.8us
W20260924 14:31:43.309864 32249 log.cc:924] Time spent T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260924 14:31:43.236696 31945 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:43.317962 31980 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:43.221400 32182 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:43.221666 32190 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:43.264209 26324 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260924 14:31:43.316005 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 173719us
W20260924 14:31:43.372602 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 141289us
I20260924 14:31:43.368258 26256 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:43.241722 31995 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:43.395879 26328 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 12. Current term is 13. Ops: []
W20260924 14:31:43.375320 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 144008us
I20260924 14:31:43.375124 32019 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:43.403707 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:43.400660 27123 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:43.409478 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 982828us
I20260924 14:31:43.380478 32189 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:43.381242 31979 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:43.380398 31875 consensus_queue.cc:1046] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260924 14:31:43.386870 31980 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:43.445377 31945 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:43.396716 32182 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:43.381801 31984 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:43.434511 31995 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:43.383262 32190 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:43.496254 26332 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.028s	sys 0.088s
W20260924 14:31:43.496923 26332 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.029s	sys 0.088s
I20260924 14:31:43.478154 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:43.454985 31945 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:43.482760 31980 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:43.408572 32019 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:43.447580 32189 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:43.489089 26250 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:43.499006 26331 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
W20260924 14:31:43.502501 26251 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.011s	sys 0.020s
I20260924 14:31:43.503667 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:43.513091 26249 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260924 14:31:43.552451 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:43.553328 27186 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 13: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
W20260924 14:31:43.554149 26251 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.016s	sys 0.023s
I20260924 14:31:43.558681 26258 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:43.446485 32034 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260924 14:31:43.562194 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:43.462215 31890 raft_consensus.cc:3032] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Stepping down as leader of term 18
I20260924 14:31:43.447773 32215 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260924 14:31:43.564222 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:43.502537 32188 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:43.496424 32186 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:43.571905 31982 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:43.515373 31995 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:43.552901 32205 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:43.578737 27186 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260924 14:31:43.458184 32182 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:43.577682 26258 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:43.568770 31890 raft_consensus.cc:729] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:43.593250 31677 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:43.576988 31882 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:43.591863 31982 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:43.633934 32180 consensus_meta.cc:287] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.000s
I20260924 14:31:43.594100 32205 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:43.628068 31677 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:43.583221 32188 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:43.640250 31979 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:43.654474 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:43.635236 31995 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:43.655951 31882 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:43.656947 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:43.679275 26331 raft_consensus.cc:2379] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:43.598151 32235 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260924 14:31:43.678753 32146 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:43.707811 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:43.719784 31606 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:43.721011 32253 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260924 14:31:43.720770 31606 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:43.720237 32258 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260924 14:31:43.680791 31979 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:43.732955 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:43.711232 32178 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:43.738118 32252 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260924 14:31:43.650342 31890 consensus_queue.cc:260] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:43.708243 31984 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:43.730893 32198 consensus_meta.cc:287] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.724s	user 0.012s	sys 0.000s
I20260924 14:31:43.710430 31945 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:43.713328 32190 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:43.747290 32189 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:43.808367 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:43.710094 31677 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:43.811609 31677 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:43.814172 31890 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:43.823249 32182 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:43.827323 32019 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:43.828919 32254 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260924 14:31:43.831314 32205 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:43.710624 32146 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:43.833899 32146 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:43.835667 26331 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:43.837405 31982 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:43.838539 31606 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:43.741300 31882 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:43.843199 31882 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:43.844103 31979 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:43.846405 31882 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:43.743536 32188 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:43.851940 27189 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:43.937431 31984 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:43.938632 26257 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.074s	sys 0.066s
W20260924 14:31:43.939018 26257 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.074s	sys 0.066s
I20260924 14:31:43.873613 27186 raft_consensus.cc:2454] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:43.754627 26252 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:43.948022 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:43.951264 26297 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.024s	sys 0.079s
W20260924 14:31:43.951692 26297 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.024s	sys 0.079s
I20260924 14:31:43.951611 26295 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:43.956064 26295 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:43.956374 26329 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.029s	sys 0.064s
W20260924 14:31:43.957093 26329 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.029s	sys 0.064s
I20260924 14:31:43.962898 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:43.963575 26326 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:43.964725 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:43.959333 26329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1242) took 1038 ms. Trace:
I20260924 14:31:43.965965 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:43.969460 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:43.966882 26329 rpcz_store.cc:276] 0924 14:31:42.920829 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:42.920946 (+   117us) service_pool.cc:227] Handling call
0924 14:31:42.921274 (+   328us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:42.926729 (+  5455us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:42.926773 (+    44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:42.927252 (+   479us) log.cc:842] Serialized 26 byte log entry
0924 14:31:43.415238 (+487986us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:43.957643 (+542405us) log.cc:842] Serialized 14 byte log entry
0924 14:31:43.959048 (+  1405us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:43.959115 (+    67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:43.959152 (+    37us) raft_consensus.cc:1671] finished
0924 14:31:43.959163 (+    11us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:43.959282 (+   119us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4631,"raft.queue_time_us":17016,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":8099,"thread_start_us":998110,"threads_started":2,"wal-append.queue_time_us":644217}
I20260924 14:31:43.744082 32178 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:43.950981 32146 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:43.979748 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:43.972771 26326 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:43.958155 32188 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:43.938241 31984 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:43.959901 31882 raft_consensus.cc:2781] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:43.971266 32153 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:44.002636 31882 raft_consensus.cc:686] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:44.004364 31882 consensus_queue.cc:237] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.012666 31335 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.014731 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1623) took 1630 ms. Trace:
I20260924 14:31:44.015138 26406 rpcz_store.cc:276] 0924 14:31:42.384583 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:42.385825 (+  1242us) service_pool.cc:227] Handling call
0924 14:31:42.409944 (+ 24119us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:42.756557 (+346613us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:44.014479 (+1257922us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b5400123ed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:44.014671 (+   192us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":759090432}
I20260924 14:31:43.988597 32146 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.021004 32146 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.022444 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:44.023222 27189 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:44.024029 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1246) took 1054 ms. Trace:
I20260924 14:31:44.024501 26328 rpcz_store.cc:276] 0924 14:31:42.972606 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:42.974656 (+  2050us) service_pool.cc:227] Handling call
0924 14:31:42.990718 (+ 16062us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:43.396433 (+405715us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:44.023782 (+627349us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54005bb8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:44.023968 (+   186us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":907549440}
W20260924 14:31:44.028352 32244 log.cc:924] Time spent T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 1.093s	user 0.006s	sys 0.020s
I20260924 14:31:44.030812 32178 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:43.998658 31982 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.031834 31982 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.033878 31982 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.033759 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:43.988381 32188 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:44.034977 32188 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:44.040673 32243 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.106s	user 0.007s	sys 0.025s
I20260924 14:31:44.061348 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:44.062088 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 1.10889e+06us
I20260924 14:31:44.062961 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:44.064133 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:44.065141 32233 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:44.065713 32233 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.062400 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:44.062103 26398 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 13
I20260924 14:31:44.080250 32143 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:44.080739 26404 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260924 14:31:44.080902 32143 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:44.081750 31931 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.082337 31931 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.082729 32143 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.084451 31931 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.087617 32238 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.088229 32166 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.088124 32238 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.088788 32166 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.090459 32238 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:44.090977 31924 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.065280 26406 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:44.091692 31924 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.088269 32199 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.093264 26369 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.093133 32199 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:44.093864 31924 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.094430 31924 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:44.094527 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.094903 31924 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:44.095635 32085 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:44.096813 32200 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.097453 32085 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260924 14:31:44.097358 32200 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.097791 26405 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:44.098208 32085 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:44.099354 32200 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.103415 32233 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.104017 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.104955 26401 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:44.096807 31984 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:44.091005 32166 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.109535 32171 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:44.110239 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:44.110870 26404 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 17
I20260924 14:31:44.114084 32171 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:31:44.117219 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46104 (request call id 1631) took 1587 ms. Trace:
I20260924 14:31:44.117668 26403 rpcz_store.cc:276] 0924 14:31:42.530114 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:42.530274 (+   160us) service_pool.cc:227] Handling call
0924 14:31:44.116954 (+1586680us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54001e50d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:44.117139 (+   185us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2049012992}
I20260924 14:31:44.095225 32199 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:44.122027 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1754) took 1064 ms. Trace:
I20260924 14:31:44.122406 31882 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.123749 32180 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.122529 26253 rpcz_store.cc:276] 0924 14:31:43.057234 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:43.057376 (+   142us) service_pool.cc:227] Handling call
0924 14:31:43.057716 (+   340us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:43.058148 (+   432us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:43.058176 (+    28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:43.058713 (+   537us) log.cc:842] Serialized 26 byte log entry
0924 14:31:43.500216 (+441503us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:43.551845 (+ 51629us) log.cc:842] Serialized 14 byte log entry
0924 14:31:44.121295 (+569450us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b180046b9a0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09b6261990 00007f09b6423864 00007f09b6423155 00007f09b93463d0 00007f09b9358576 00007f09b93690d7 00007f09b9369058
0924 14:31:44.121587 (+   292us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:44.121672 (+    85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:44.121728 (+    56us) raft_consensus.cc:1671] finished
0924 14:31:44.121831 (+   103us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:44.121967 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4284,"prepare.queue_time_us":51652,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":572210,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":203,"spinlock_wait_cycles":12069760,"thread_start_us":441165,"threads_started":1,"wal-append.queue_time_us":565007}
I20260924 14:31:44.124796 32180 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:44.128291 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.128784 32198 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.129338 32198 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:44.131672 32189 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:44.132478 32189 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:44.134995 32189 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.135643 32182 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.136296 32182 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:44.138718 32182 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.140589 32186 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.141163 32186 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:44.140651 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:44.142580 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.143091 32186 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.143193 26258 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:44.144450 26250 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260924 14:31:44.151270 26249 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260924 14:31:44.144918 32171 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:44.145397 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.146121 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.157636 26369 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.163818 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:44.163196 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.161026 26251 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:44.199195 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.199522 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:44.200161 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 49658.8us
I20260924 14:31:44.213660 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:44.215413 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.216765 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.123078 31882 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.221700 31924 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.195020 32199 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:44.233031 26404 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:44.222821 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:44.223196 26369 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.211673 32238 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260924 14:31:44.225087 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.225689 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.226295 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.239261 26401 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 19: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:44.198864 32166 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:44.139358 32182 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.195540 32171 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:44.135676 32180 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:44.158342 31931 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.237015 26254 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260924 14:31:44.237701 27189 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
I20260924 14:31:44.239984 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.245056 26404 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:44.238559 26406 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:44.245988 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.237604 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.247162 26405 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:44.247577 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.249332 26400 raft_consensus.cc:1226] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 18. Current term is 19. Ops: []
I20260924 14:31:44.251740 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.251873 26295 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:44.248152 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.253891 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.233163 32206 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:44.254082 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.254912 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.257689 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49618 (request call id 1752) took 1391 ms. Trace:
I20260924 14:31:44.198534 32198 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:44.253181 26405 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:44.198732 32233 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260924 14:31:44.262010 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.241679 32231 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:44.261386 26369 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.263078 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:44.263191 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.241425 32166 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:44.268627 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.279963 26401 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:44.227288 32188 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.280115 26403 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:44.277385 26398 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:44.286383 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:44.262446 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 286827us
I20260924 14:31:44.260357 26257 rpcz_store.cc:276] 0924 14:31:42.866360 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:42.866501 (+   141us) service_pool.cc:227] Handling call
0924 14:31:42.903240 (+ 36739us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:42.917248 (+ 14008us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:42.989409 (+ 72161us) log.cc:842] Serialized 14 byte log entry
0924 14:31:44.142950 (+1153541us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:44.143462 (+   512us) log.cc:842] Serialized 26 byte log entry
0924 14:31:44.143632 (+   170us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:44.248734 (+105102us) log.cc:842] Serialized 14 byte log entry
0924 14:31:44.257412 (+  8678us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:44.257484 (+    72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:44.257520 (+    36us) raft_consensus.cc:1671] finished
0924 14:31:44.257534 (+    14us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:44.257639 (+   105us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":211058,"prepare.queue_time_us":123602,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":405,"raft.queue_time_us":218564,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":4728,"spinlock_wait_cycles":7173376,"thread_start_us":1049288,"threads_started":3,"wal-append.queue_time_us":1042298}
I20260924 14:31:44.251487 32171 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:44.262614 31335 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:31:44.261581 32206 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:44.258240 32143 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:44.253381 32199 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:44.295933 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.327083 26255 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:44.296352 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:44.299961 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.301353 26406 raft_consensus.cc:2354] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14.
I20260924 14:31:44.251212 32182 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:44.282275 26397 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260924 14:31:44.257340 32164 raft_consensus.cc:2741] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260924 14:31:44.305037 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.331385 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.317976 26401 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260924 14:31:44.254431 31995 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.260696 31984 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:44.320930 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.303697 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.307498 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.327286 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:44.330227 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.339388 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.345432 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.284199 32166 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:44.348762 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45184.7us
I20260924 14:31:44.350657 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.293706 32208 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:44.326386 32206 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:44.302084 32198 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260924 14:31:44.259876 31931 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.371377 26250 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:44.391479 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.374534 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.343282 26323 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:44.369256 27185 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:44.331039 32199 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:44.330987 31335 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:44.345230 32231 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:44.402072 32259 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.402729 32259 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:44.404385 32182 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.404801 32259 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:44.406410 32188 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.348151 32205 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:44.422734 32273 log.cc:924] Time spent T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260924 14:31:44.377951 32277 raft_consensus.cc:3032] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Stepping down as leader of term 18
I20260924 14:31:44.414813 32270 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260924 14:31:44.417502 32259 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.437428 32259 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.439968 26253 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260924 14:31:44.377420 32164 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:44.365659 32085 consensus_queue.cc:1046] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260924 14:31:44.414248 32204 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:44.430888 32277 raft_consensus.cc:729] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:44.385418 32198 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.383908 32190 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:44.353258 31984 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.414606 32199 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:44.346871 31995 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.456413 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:44.461794 27189 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 14
I20260924 14:31:44.080794 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260924 14:31:44.468983 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 144633us
I20260924 14:31:44.474187 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:44.446552 31980 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:44.436264 31882 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.471133 32186 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:44.457819 32204 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:44.476687 26075 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:44.479357 31335 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.478884 31931 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:44.459230 32259 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.479369 32204 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:44.463163 32198 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 9 FOLLOWER]: Advancing to term 10
I20260924 14:31:44.455950 32260 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:44.488404 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:44.468154 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 93689.8us
I20260924 14:31:44.484324 31995 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.480239 32277 consensus_queue.cc:260] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.493750 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.493906 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:44.497875 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.500494 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.501171 26406 raft_consensus.cc:2379] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:31:44.501266 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 174062us
I20260924 14:31:44.493753 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.491209 31984 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.511605 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:44.511680 31995 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:44.515265 32259 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:44.485757 32260 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.514603 27189 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:44.516984 31767 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260924 14:31:44.531699 32260 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.532399 32277 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:44.526669 32259 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:44.536520 32259 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:44.537665 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.540308 32206 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.542291 32206 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.545615 32143 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:44.520323 31945 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:44.516791 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.550130 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:44.550761 32284 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:44.550974 27126 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:44.521250 31979 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:44.522012 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.523262 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.516258 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.526957 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.525835 32153 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:44.528810 32282 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:31:44.501843 31980 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:44.558020 26406 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:44.558910 27191 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:31:44.551230 32284 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.567978 32206 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.587663 32284 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:44.587805 32206 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:44.591679 26077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55200 (request call id 130) took 34335 ms (client timeout 15000 ms). Trace:
I20260924 14:31:44.552177 31995 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:44.552114 32171 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:44.579480 32019 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260924 14:31:44.586591 32260 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:44.600414 31882 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:44.605515 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:44.608270 32171 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:44.608186 31882 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:44.608798 26405 raft_consensus.cc:2379] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:44.564459 32153 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.579671 31945 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:44.610214 31882 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:44.610610 32153 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.574868 31982 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:44.599031 27192 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260924 14:31:44.600420 32198 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.580403 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:44.611397 31982 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:44.601305 32235 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:44.612946 32198 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.612845 32235 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:44.613101 31982 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:44.613618 26257 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:31:44.613888 26221 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:44.602264 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.615321 32272 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:44.615760 32272 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:44.615908 32235 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:44.616093 32272 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:44.617053 31995 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:44.576565 32166 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.603760 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.603039 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.577001 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:44.577263 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.604517 26401 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:31:44.577469 31980 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:44.602435 32188 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260924 14:31:44.605863 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.580315 31606 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260924 14:31:44.629446 32285 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:44.583110 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.630615 32166 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:44.630947 26404 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 10.
I20260924 14:31:44.569461 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.631945 26330 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:44.632494 32199 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.633661 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.566120 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:44.578486 32204 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:44.575336 31979 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.619971 26328 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:44.634909 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 68851.6us
I20260924 14:31:44.611312 31945 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:44.637130 32171 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:44.637571 32171 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:44.637910 32171 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:44.640514 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:44.597407 26075 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 13 to 14, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 14 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:44.626000 26402 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:44.626591 26249 raft_consensus.cc:2396] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260924 14:31:44.626756 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:44.643360 26252 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:44.593935 32206 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.633718 26398 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 20: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:44.629108 32259 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:44.647010 26250 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:44.647936 32259 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:44.648856 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.601794 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.636358 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.644353 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:44.652889 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:44.651813 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:44.656144 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:44.566939 31875 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:44.657033 26325 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:44.574538 27185 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:44.661569 26328 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:44.662139 32199 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:44.645566 32198 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:31:44.662695 31995 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.663813 32166 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:44.664770 32204 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:44.671816 27126 leader_election.cc:398] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14.
I20260924 14:31:44.674266 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.688165 27126 leader_election.cc:401] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:44.688843 26252 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:44.676302 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:44.665450 31677 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:44.678754 32085 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.678779 32198 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:44.685185 26295 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:44.686064 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:44.690371 26328 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:44.652606 32277 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:44.691816 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.658470 31875 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:44.702011 26221 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:44.698632 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 363189us
I20260924 14:31:44.712845 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:44.683995 32206 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:44.726845 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:44.727862 26252 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:44.731180 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.735692 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.658504 32284 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:44.739466 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.743304 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.746423 27189 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:44.747093 27189 raft_consensus.cc:595] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:44.748255 26328 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:44.748629 26404 raft_consensus.cc:1261] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260924 14:31:44.749687 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:44.696434 32198 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:44.751909 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.752708 26255 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:44.756160 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:44.697866 32166 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:44.762923 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:44.730489 32186 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:44.757411 26297 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:44.713115 31677 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:44.685361 32271 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260924 14:31:44.764952 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:44.757390 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 240331us
I20260924 14:31:44.725308 31875 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:44.725540 32260 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader election won for term 10
I20260924 14:31:44.801188 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:44.803874 27191 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: []
W20260924 14:31:44.807018 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 49713.1us
I20260924 14:31:44.816589 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.737632 32188 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:44.736678 32206 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:44.858991 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:44.859545 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.819418 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:44.851262 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 50155.6us
I20260924 14:31:44.755622 32146 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:44.853005 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.759462 31945 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:44.616452 26077 rpcz_store.cc:269] 0924 14:31:10.255897 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:10.295491 (+ 39594us) service_pool.cc:227] Handling call
0924 14:31:44.591432 (+34295941us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:33.944264 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:33.945210 (+   946us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:33.945285 (+    75us) write_op.cc:435] Acquired schema lock
0924 14:31:33.945312 (+    27us) tablet.cc:660] Decoding operations
0924 14:31:34.016218 (+ 70906us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:34.016332 (+   114us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:34.016380 (+    48us) tablet.cc:683] Acquiring locks for 26 operations
0924 14:31:34.018716 (+  2336us) tablet.cc:699] Row locks acquired
0924 14:31:34.018749 (+    33us) write_op.cc:260] PREPARE: finished
0924 14:31:34.051221 (+ 32472us) write_op.cc:270] Start()
0924 14:31:34.060345 (+  9124us) write_op.cc:276] Timestamp: P: 1790260294018861 usec, L: 0
0924 14:31:34.074426 (+ 14081us) op_driver.cc:331] REPLICATION: starting
0924 14:31:34.095722 (+ 21296us) log.cc:842] Serialized 10315 byte log entry
0924 14:31:37.110774 (+3015052us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b18002de500. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09bcd5e8a8 00007f09bcd60a78 00007f09bcd60a28 00007f09bcd609a8 00007f09bcd60963 00007f09bcd5fc4c 00007f09bf357206 00007f09bf34f808 00007f09b935a170
0924 14:31:37.110833 (+    59us) op_driver.cc:444] REPLICATION: finished
0924 14:31:37.417993 (+307160us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b64085fe 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09bcd5e410
0924 14:31:37.966523 (+548530us) write_op.cc:301] APPLY: starting
0924 14:31:37.966625 (+   102us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:38.551324 (+584699us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:38.592145 (+ 40821us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:39.358103 (+765958us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:39.358644 (+   541us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=52,mrs_lookups=0
0924 14:31:39.358715 (+    71us) write_op.cc:312] APPLY: finished
0924 14:31:39.359339 (+   624us) log.cc:842] Serialized 227 byte log entry
0924 14:31:39.569543 (+210204us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:39.570550 (+  1007us) write_op.cc:454] Released schema lock
0924 14:31:39.570741 (+   191us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:39.571374 (+   633us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:39.571407 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35148,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":802978,"apply.run_cpu_time_us":54985,"apply.run_wall_time_us":1695684,"cfile_cache_hit":106,"cfile_cache_hit_bytes":112911,"num_ops":26,"prepare.queue_time_us":198957,"prepare.run_cpu_time_us":106003,"prepare.run_wall_time_us":450297,"raft.queue_time_us":944195,"raft.run_cpu_time_us":51857,"raft.run_wall_time_us":2464724,"replication_time_us":1402959,"spinlock_wait_cycles":23732736,"thread_start_us":2406065,"threads_started":5,"wal-append.queue_time_us":376037,"wal-append.run_cpu_time_us":27995,"wal-append.run_wall_time_us":3094116}]]}
I20260924 14:31:44.699530 32085 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.747957 32019 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:44.863878 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 136709us
I20260924 14:31:44.864090 26326 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:44.928539 27125 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:44.866984 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.929417 26330 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:44.930526 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:44.931977 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.933197 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:44.867996 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.868888 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.869796 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.936795 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:44.871194 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.858225 32186 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:44.874513 32189 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.697005 32143 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260924 14:31:44.730683 32277 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:44.862133 32260 raft_consensus.cc:686] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:44.893101 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:44.761801 32261 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:44.894164 26256 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260924 14:31:44.763940 32075 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:44.936928 26327 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:44.895239 26328 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
W20260924 14:31:44.898165 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 184555us
I20260924 14:31:44.898017 32235 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260924 14:31:44.926103 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.739521 32178 consensus_queue.cc:1046] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.123s
I20260924 14:31:44.865039 26324 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:44.876782 32238 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:44.882726 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:44.943202 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.861892 32206 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:44.965545 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.952565 26328 pending_rounds.cc:76] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 10
I20260924 14:31:44.968911 26328 pending_rounds.cc:97] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260924 14:31:44.969251 26328 raft_consensus.cc:2866] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:44.969453 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.955391 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:44.852717 31997 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.067s
I20260924 14:31:44.972448 32200 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:44.950268 26402 raft_consensus.cc:2379] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 10: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:44.973696 32200 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:44.959545 26404 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260924 14:31:44.974175 32200 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:44.974403 26404 pending_rounds.cc:76] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 9
I20260924 14:31:44.974679 26404 pending_rounds.cc:97] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260924 14:31:44.974905 26404 raft_consensus.cc:2866] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:44.900522 32236 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:44.980331 32236 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.982856 32236 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.985061 32225 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:44.985842 32225 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:44.986507 32225 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:44.861654 31688 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.091s
I20260924 14:31:44.927698 31882 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:44.993593 31882 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:44.960572 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:45.001289 31882 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:44.946754 31984 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:45.015115 27191 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "6c22052275034c369b7f7dca086fbd1e"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:45.039055 27191 raft_consensus.cc:595] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Received request to transfer leadership to TS 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:45.039479 27191 raft_consensus.cc:603] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:45.014360 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:44.966177 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 253396us
I20260924 14:31:45.040480 26256 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:45.005920 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:44.952416 32075 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:45.041836 26401 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:45.005331 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 252842us
I20260924 14:31:44.958608 32235 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:45.033854 32188 raft_consensus.cc:686] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:45.077394 26369 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:45.079859 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:45.084558 32188 consensus_queue.cc:237] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:45.086071 32261 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:45.087387 32261 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260924 14:31:45.080487 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 109684us
I20260924 14:31:44.964545 32238 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:45.092415 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:44.963798 31890 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:44.945618 32189 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:45.133553 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 50694.6us
W20260924 14:31:45.139320 32171 consensus_meta.cc:287] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.501s	user 0.006s	sys 0.005s
I20260924 14:31:45.093256 31979 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:45.141245 26224 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:45.142503 32295 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
W20260924 14:31:45.143298 32292 log.cc:924] Time spent T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
W20260924 14:31:45.144033 32299 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260924 14:31:45.144419 32085 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:44.967692 32166 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:45.034699 32182 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:45.085757 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:45.152489 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:45.159073 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 66742.3us
W20260924 14:31:45.172945 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 112326us
I20260924 14:31:45.076591 32075 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:45.075497 32272 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:45.043502 32200 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.080943 32235 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:45.146021 31945 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260924 14:31:45.135306 32276 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:45.076378 32190 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.091506 31984 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:45.179600 32238 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:44.963588 31677 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:45.226792 26401 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:45.243129 32284 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:45.171957 32261 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260924 14:31:45.144929 32300 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260924 14:31:45.179893 26256 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
W20260924 14:31:45.243702 31982 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.002s	sys 0.023s
W20260924 14:31:45.253862 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 168184us
W20260924 14:31:45.329355 31982 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.003s	sys 0.023s
I20260924 14:31:45.329993 26298 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
W20260924 14:31:45.330605 32303 log.cc:924] Time spent T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260924 14:31:45.332415 26298 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:45.332528 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:45.109126 32164 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.142s
I20260924 14:31:45.333896 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:45.256934 26258 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260924 14:31:45.245802 32293 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.486s	user 0.001s	sys 0.000s
W20260924 14:31:45.233610 27186 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.005s	sys 0.016s
W20260924 14:31:45.335606 32301 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.328s	user 0.001s	sys 0.000s
I20260924 14:31:45.252702 32182 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:45.336462 32310 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:45.337847 32182 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:45.202909 31890 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.338083 32235 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:45.339973 31890 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:45.340477 32308 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260924 14:31:45.342787 26250 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:45.290364 32322 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.343653 32236 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:45.344004 32322 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.344323 32236 consensus_queue.cc:1035] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:45.346135 32322 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:45.346514 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:45.346765 32277 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:45.247380 26075 catalog_manager.cc:5495] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 13 to 14, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 14 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:45.351702 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:45.352272 32272 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:45.354044 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:45.167744 32260 consensus_queue.cc:237] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:45.206197 32186 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:45.202154 32307 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260924 14:31:45.216567 32171 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:45.357440 26221 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:45.318212 31875 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:45.264240 32276 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.368539 26252 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:45.320904 32200 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:45.263279 32317 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:45.322432 32189 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:45.322711 32198 consensus_meta.cc:287] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.000s
I20260924 14:31:45.262732 31335 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260924 14:31:45.323462 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:45.386857 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:45.328378 32238 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:45.335778 27186 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.006s	sys 0.016s
I20260924 14:31:45.265803 32190 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.354734 26406 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:45.367431 26404 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260924 14:31:45.291484 32313 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:45.335134 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:45.427148 26254 raft_consensus.cc:2396] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260924 14:31:45.377209 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:45.427292 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:45.428869 26325 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:45.427807 26331 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:45.432246 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:45.385567 32322 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:45.452310 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:45.380949 26250 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260924 14:31:45.381994 32326 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:45.423868 32204 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14.
I20260924 14:31:45.382856 31875 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:45.363433 32261 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:45.496511 32261 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.498965 32261 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:45.324148 31984 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:45.501362 32238 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:45.409673 32075 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:45.502780 32171 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:45.505021 32075 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:45.371685 32276 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:45.425750 26404 pending_rounds.cc:76] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 10
I20260924 14:31:45.368448 32310 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.484120 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:45.491197 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.486187 32192 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:45.492614 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.456131 32313 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:45.506502 26404 pending_rounds.cc:97] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260924 14:31:45.514637 26404 raft_consensus.cc:2866] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:45.507190 27186 raft_consensus.cc:2379] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 15: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:45.508728 27190 raft_consensus.cc:2379] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 19: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:45.508155 27191 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260924 14:31:45.520176 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.389825 32198 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:45.522341 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:45.526005 27190 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: []
I20260924 14:31:45.525226 26330 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:45.529795 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.531172 26329 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:45.533327 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:45.534679 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:45.534945 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:45.535629 27190 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 20: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:45.535699 26325 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:45.536612 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.507699 32192 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:45.530649 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 269981us
I20260924 14:31:45.490111 32322 raft_consensus.cc:686] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:45.540473 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:45.508540 32313 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:45.542289 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:45.517542 32261 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.543054 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:45.590570 26328 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:45.591244 26250 raft_consensus.cc:1226] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260924 14:31:45.632692 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:45.611825 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 394804us
W20260924 14:31:45.635542 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 494347us
I20260924 14:31:45.638413 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:45.640511 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:45.641840 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:45.642354 26328 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:45.649761 26224 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:45.654107 26329 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260924 14:31:45.656927 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:45.662763 26327 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:45.664202 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:45.665073 26404 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:45.665421 26258 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:45.677006 27185 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260924 14:31:45.674925 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:45.685105 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:45.686902 26258 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:45.589423 32313 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:45.690340 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:45.691231 26256 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:45.698611 26224 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:45.700963 26404 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:45.505916 26075 catalog_manager.cc:5495] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 8 to 9, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 9 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:45.707602 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.506733 32310 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.667660 32190 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:45.490481 32326 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:45.636549 32261 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.743355 27126 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:45.679086 32285 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.022s	sys 0.011s
I20260924 14:31:45.711011 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.709789 27125 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:45.715310 26258 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:45.745765 27189 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:45.710522 27182 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:45.664047 32205 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:45.684449 32189 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260924 14:31:45.737196 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.737172 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:45.737574 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.768167 26330 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:45.736734 26224 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:45.746481 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:45.661502 32182 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260924 14:31:45.739099 26256 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
I20260924 14:31:45.748333 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:45.699043 32235 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:45.754736 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:45.707041 31875 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:45.776616 26324 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260924 14:31:45.772090 27182 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:45.781109 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 44463.2us
I20260924 14:31:45.782320 32190 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:45.782203 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:45.782776 32190 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:45.783250 32190 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:45.755906 32285 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.023s	sys 0.011s
I20260924 14:31:45.792138 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:45.777807 32206 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.730s	user 0.011s	sys 0.000s
I20260924 14:31:45.800266 32192 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:45.800421 32326 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:45.800961 31875 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:45.802225 27125 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:45.804952 32335 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:45.806592 32310 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:45.783833 31890 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.809928 27182 raft_consensus.cc:2454] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:45.807126 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1232) took 1307 ms. Trace:
W20260924 14:31:45.813537 32304 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.707s	user 0.014s	sys 0.000s
I20260924 14:31:45.808506 26224 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:45.846838 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 44694.8us
W20260924 14:31:45.854033 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 57136.9us
I20260924 14:31:45.819938 32333 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.855746 32333 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:45.820984 32336 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260924 14:31:45.824275 32337 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260924 14:31:45.770498 32290 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:45.859372 32290 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:45.860316 32290 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.770781 32267 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:45.826172 26075 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 9 to 14, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 14 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:45.863794 32267 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:45.864739 32267 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:45.864490 26075 catalog_manager.cc:5495] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 5 to 10, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 10 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:45.846251 26257 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:45.868441 32335 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:45.870005 26075 catalog_manager.cc:5495] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 8 to 11, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 11 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:45.855304 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:45.872524 26075 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 6 to 10, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 10 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:45.874670 32186 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:45.875244 32186 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.875643 32186 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:45.874590 26075 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 12 to 14, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:45.880848 32182 consensus_queue.cc:1046] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:45.881712 32182 consensus_queue.cc:1225] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260924 14:31:45.882628 26224 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:45.883515 32335 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:45.886296 26224 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:45.891743 32335 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:45.893833 32186 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:45.896461 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:45.899881 27182 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:45.901966 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:45.905972 32186 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:45.909502 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.909732 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:45.915642 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:45.920907 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:45.846288 26323 rpcz_store.cc:276] 0924 14:31:44.500096 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:44.500238 (+   142us) service_pool.cc:227] Handling call
0924 14:31:44.524440 (+ 24202us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:44.666769 (+142329us) raft_consensus.cc:1520] Early marking committed up to index 11
0924 14:31:44.697249 (+ 30480us) log.cc:842] Serialized 14 byte log entry
0924 14:31:45.590175 (+892926us) log.cc:842] Serialized 14 byte log entry
0924 14:31:45.593030 (+  2855us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:45.612793 (+ 19763us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:45.612845 (+    52us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:45.806883 (+194038us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540001e0d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:45.806933 (+    50us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:45.807069 (+   136us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":667,"prepare.queue_time_us":20106,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":130013,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":49463,"spinlock_wait_cycles":326744320,"thread_start_us":891984,"threads_started":3,"wal-append.queue_time_us":267264}
I20260924 14:31:45.924450 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:45.926556 26221 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:45.916563 26256 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:45.930168 26297 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:45.839358 31875 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.933771 27185 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:45.845891 31979 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.769666 32322 consensus_queue.cc:237] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.933823 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:45.777498 32276 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:45.851958 32171 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:45.950469 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:45.932981 32192 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:45.940110 31875 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:45.933127 32260 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:45.934495 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:45.959479 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:45.936532 32285 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:46.161289 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 228477us
I20260924 14:31:45.828711 32206 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:45.941390 32186 raft_consensus.cc:2781] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:45.930958 32198 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.043263 26402 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:45.900666 31890 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.967355 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:46.154822 32261 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:45.953907 32345 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:31:46.101502 32344 log.cc:924] Time spent T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260924 14:31:46.102056 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 151313us
W20260924 14:31:46.102867 32349 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260924 14:31:46.103888 32340 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260924 14:31:46.105226 32347 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260924 14:31:46.105931 32339 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260924 14:31:46.043066 32260 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:45.954250 31979 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:46.134621 31677 raft_consensus.cc:3032] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:46.135039 27185 raft_consensus.cc:595] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:46.061317 32236 raft_consensus.cc:2781] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:46.096149 32225 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:46.148433 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 218310us
I20260924 14:31:45.933466 32341 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:46.236379 32335 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:46.237119 32335 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:45.967250 32171 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:46.241627 32171 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:46.116060 32184 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:46.244061 32184 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:46.246732 32184 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:46.248118 32276 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.250976 31677 raft_consensus.cc:729] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:46.162096 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 202753us
I20260924 14:31:45.967609 26251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 2170) took 1347 ms. Trace:
I20260924 14:31:46.170491 27188 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 12: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:46.178675 32206 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:46.179244 26253 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:46.169428 32189 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:46.181234 32190 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:46.183740 31890 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:46.192178 26295 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:46.151991 32333 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.198652 27185 raft_consensus.cc:603] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:46.169792 32290 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:46.195237 32186 raft_consensus.cc:686] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:46.207178 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1393) took 1230 ms. Trace:
I20260924 14:31:46.286710 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:46.288295 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:46.289139 26329 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:31:46.216950 32348 log.cc:924] Time spent T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.395s	user 0.002s	sys 0.000s
I20260924 14:31:46.199407 32260 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:46.273717 32345 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:46.233738 31875 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:46.235108 32225 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:46.328353 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:46.330147 32225 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:46.254324 26251 rpcz_store.cc:276] 0924 14:31:44.619912 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:44.620060 (+   148us) service_pool.cc:227] Handling call
0924 14:31:44.628430 (+  8370us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:44.634552 (+  6122us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:44.634595 (+    43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:44.645010 (+ 10415us) log.cc:842] Serialized 26 byte log entry
0924 14:31:44.877727 (+232717us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:44.877784 (+    57us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:44.878354 (+   570us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:45.812883 (+934529us) spinlock_profiling.cc:243] Waited 8.38 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:45.958623 (+145740us) raft_consensus.cc:1671] finished
0924 14:31:45.958645 (+    22us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:45.967540 (+  8895us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5764,"raft.queue_time_us":6490,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":19274112,"thread_start_us":193549,"threads_started":1,"wal-append.queue_time_us":389669}
I20260924 14:31:46.196691 32198 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:46.277135 32335 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:46.292989 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 100896us
I20260924 14:31:46.294104 26254 raft_consensus.cc:1226] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
W20260924 14:31:46.279958 32313 consensus_meta.cc:287] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.003s
I20260924 14:31:46.350071 32313 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:46.298031 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:46.352129 26326 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:46.285162 26255 rpcz_store.cc:276] 0924 14:31:44.977030 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:45.039206 (+ 62176us) service_pool.cc:227] Handling call
0924 14:31:45.039545 (+   339us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:45.218274 (+178729us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:45.218315 (+    41us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:45.242814 (+ 24499us) log.cc:842] Serialized 52 byte log entry
0924 14:31:45.527144 (+284330us) raft_consensus.cc:1631] Marking committed up to 6
0924 14:31:45.527185 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:45.527250 (+    65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:46.206907 (+679657us) raft_consensus.cc:1671] finished
0924 14:31:46.206939 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:46.207110 (+   171us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":51847,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":42978,"thread_start_us":419613,"threads_started":2,"wal-append.queue_time_us":472279}
I20260924 14:31:46.299120 26330 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:46.275050 32189 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:46.299309 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:46.228605 32354 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260924 14:31:46.300364 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.297628 32075 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:46.301980 32366 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260924 14:31:46.297804 32285 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:46.309787 32206 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:46.311098 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.252125 31677 consensus_queue.cc:260] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:46.278676 32236 raft_consensus.cc:686] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:46.281975 32290 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:46.336764 32317 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:46.338227 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:46.346901 26329 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
W20260924 14:31:46.348097 32346 log.cc:924] Time spent T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.551s	user 0.006s	sys 0.000s
I20260924 14:31:46.348127 32372 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:46.349426 32312 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:46.368425 26326 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:46.368616 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:46.380838 27185 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
I20260924 14:31:46.378391 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:46.384275 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.397413 27185 raft_consensus.cc:595] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:46.397845 27185 raft_consensus.cc:603] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:46.399220 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.400655 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:46.402441 32319 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260924 14:31:46.403522 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.369171 32189 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:46.388517 32290 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:46.413486 31979 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:46.414103 31875 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.416375 32190 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:46.404856 32312 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:46.400313 32372 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:46.417729 32236 consensus_queue.cc:237] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:46.386418 32206 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:46.392881 32317 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:46.399983 31677 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:46.365490 32198 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:46.436075 27188 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac"
dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
mode: GRACEFUL
new_leader_uuid: "45ab97b763804767846e9d27a01b561f"
 from {username='slave'} at 127.0.0.1:41190
W20260924 14:31:46.436681 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106428us
I20260924 14:31:46.437001 27188 raft_consensus.cc:595] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Received request to transfer leadership to TS 45ab97b763804767846e9d27a01b561f
I20260924 14:31:46.437497 27188 raft_consensus.cc:603] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260924 14:31:46.382726 32075 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:46.401607 32259 consensus_queue.cc:1046] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260924 14:31:46.439497 27123 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:46.442771 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260924 14:31:46.444360 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:31:46.445050 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:46.335196 32345 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:46.403656 32186 consensus_queue.cc:237] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:46.469962 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:46.470882 32378 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260924 14:31:46.475715 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1386) took 1740 ms. Trace:
I20260924 14:31:46.475845 32200 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:46.476275 26252 rpcz_store.cc:276] 0924 14:31:44.735154 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:44.735352 (+   198us) service_pool.cc:227] Handling call
0924 14:31:44.735673 (+   321us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:45.369951 (+634278us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:46.475415 (+1105464us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b540015efd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:46.475648 (+   233us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1454899968}
I20260924 14:31:46.478796 32085 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260924 14:31:46.478909 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 1.3816e+06us
W20260924 14:31:46.480383 32374 log.cc:924] Time spent T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260924 14:31:46.481033 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:46.477763 27123 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:46.482154 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:46.487334 32200 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:46.492147 32164 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:46.492920 32164 raft_consensus.cc:2741] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260924 14:31:46.493619 32164 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260924 14:31:46.500101 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1384) took 1774 ms. Trace:
I20260924 14:31:46.500383 32372 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:46.500558 26257 rpcz_store.cc:276] 0924 14:31:44.725985 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:44.726465 (+   480us) service_pool.cc:227] Handling call
0924 14:31:44.746777 (+ 20312us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:45.867717 (+1120940us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:46.499864 (+632147us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400382cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:46.500042 (+   178us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2528003456}
I20260924 14:31:46.502077 32198 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.502532 32277 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:46.503402 26257 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:46.519619 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:46.522312 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:46.524645 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1941) took 1509 ms. Trace:
I20260924 14:31:46.524880 26258 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260924 14:31:46.526490 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:46.525105 26397 rpcz_store.cc:276] 0924 14:31:45.015335 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:45.015742 (+   407us) service_pool.cc:227] Handling call
0924 14:31:45.081900 (+ 66158us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:45.111974 (+ 30074us) raft_consensus.cc:1520] Early marking committed up to index 11
0924 14:31:45.169334 (+ 57360us) log.cc:842] Serialized 14 byte log entry
0924 14:31:45.419065 (+249731us) spinlock_profiling.cc:243] Waited 940 us on lock 0x7b180003a6c0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6408345 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb 00007f09b8fd5063 00007f09b8fd557b 00007f09b935888a
0924 14:31:45.675597 (+256532us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:45.681061 (+  5464us) log.cc:842] Serialized 26 byte log entry
0924 14:31:45.681214 (+   153us) raft_consensus.cc:1631] Marking committed up to 12
0924 14:31:45.681465 (+   251us) log.cc:842] Serialized 14 byte log entry
0924 14:31:45.822354 (+140889us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b180003a6c0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d 00007f09bf68a204 00007f09bf68a039 00007f09b6408dc6
0924 14:31:46.322982 (+500628us) spinlock_profiling.cc:243] Waited 37.1 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:46.474497 (+151515us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b180003a6c0. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b639d4ca 00007f09bfa7ef99 00007f09bd1f6bcf 00007f09b64267b6 00007f09b6429b11 00007f09b6429ac9 00007f09b6429a59 00007f09b6429a21 00007f09b6428d1d
0924 14:31:46.513604 (+ 39107us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:46.513710 (+   106us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:46.524455 (+ 10745us) raft_consensus.cc:1671] finished
0924 14:31:46.524476 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:46.524591 (+   115us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":879519,"prepare.queue_time_us":235385,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":13677,"raft.queue_time_us":838558,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":20857,"spinlock_wait_cycles":173236992,"thread_start_us":256371,"threads_started":1,"wal-append.queue_time_us":594877}
I20260924 14:31:46.494220 32313 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:46.503391 32206 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.544596 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:46.550220 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:46.550410 27188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.515388 32312 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:46.457852 32317 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:46.560235 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:46.553210 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:46.562615 26257 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:46.528548 32276 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:46.565400 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.565598 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.566030 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:46.569748 26325 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:46.572290 26257 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
W20260924 14:31:46.572889 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 1.14743e+06us
I20260924 14:31:46.573028 26252 raft_consensus.cc:1226] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:46.574734 26298 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:46.576095 26258 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260924 14:31:46.618063 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52131.4us
I20260924 14:31:46.530625 32198 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260924 14:31:46.567584 32233 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:46.578632 32267 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:46.578986 32166 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260924 14:31:46.626237 32204 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:46.570257 32206 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:46.646916 26398 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:46.649501 26325 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
W20260924 14:31:46.636722 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 152843us
I20260924 14:31:46.560309 32200 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260924 14:31:46.579483 32238 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:46.661495 32238 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:46.662461 32238 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:46.663617 32238 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:46.666863 32190 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:46.627434 32384 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:46.666991 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 2181) took 1700 ms. Trace:
I20260924 14:31:46.659740 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:46.673203 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:46.655187 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:46.672978 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:46.686245 26405 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:46.672462 26404 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:46.679775 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:46.675617 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1265) took 1141 ms. Trace:
I20260924 14:31:46.691761 26324 rpcz_store.cc:276] 0924 14:31:45.533883 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:45.533990 (+   107us) service_pool.cc:227] Handling call
0924 14:31:45.534236 (+   246us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:45.777955 (+243719us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:46.675371 (+897416us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400237654. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:46.675563 (+   192us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":556585344}
I20260924 14:31:46.638146 32019 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260924 14:31:46.667516 32190 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:46.640594 32267 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:46.650148 32085 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:46.698668 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:46.637650 32075 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:46.577665 32276 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:46.675215 32277 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:46.650547 32380 raft_consensus.cc:3032] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:46.674485 32208 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:46.650435 31924 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:46.683348 26249 rpcz_store.cc:276] 0924 14:31:44.966925 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:44.968318 (+  1393us) service_pool.cc:227] Handling call
0924 14:31:45.078194 (+109876us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:45.080027 (+  1833us) raft_consensus.cc:1520] Early marking committed up to index 12
0924 14:31:45.156712 (+ 76685us) log.cc:842] Serialized 14 byte log entry
0924 14:31:45.200977 (+ 44265us) log.cc:842] Serialized 14 byte log entry
0924 14:31:46.666537 (+1465560us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b1800245c80. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b8fdd82c 00007f09b64089dc 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6423155 00007f09b93463d0 00007f09b9358576 00007f09b93690d7
0924 14:31:46.666634 (+    97us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:46.666681 (+    47us) raft_consensus.cc:1631] Marking committed up to 12
0924 14:31:46.666724 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:46.666809 (+    85us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:46.666936 (+   127us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14088,"prepare.queue_time_us":146726,"prepare.run_cpu_time_us":685,"prepare.run_wall_time_us":15812,"raft.queue_time_us":131182,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":41436,"spinlock_wait_cycles":16582912,"thread_start_us":46583,"threads_started":1}
I20260924 14:31:46.695596 32085 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:46.724341 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:46.674947 32316 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:46.675460 32189 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:46.644506 32233 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:46.733798 26398 raft_consensus.cc:2454] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:46.736613 31924 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:46.742106 32233 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:46.742210 31924 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:46.743630 26251 raft_consensus.cc:2379] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:46.744895 32189 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:46.736873 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 56500.6us
I20260924 14:31:46.748327 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:46.751736 32389 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.204s	user 0.002s	sys 0.003s
I20260924 14:31:46.752816 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:46.753294 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1959) took 1394 ms. Trace:
W20260924 14:31:46.754396 32350 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.929s	user 0.005s	sys 0.000s
I20260924 14:31:46.753706 26401 rpcz_store.cc:276] 0924 14:31:45.359016 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:45.359175 (+   159us) service_pool.cc:227] Handling call
0924 14:31:45.359678 (+   503us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:46.494287 (+1134609us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:46.494318 (+    31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:46.506116 (+ 11798us) log.cc:842] Serialized 26 byte log entry
0924 14:31:46.547461 (+ 41345us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:46.547503 (+    42us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:46.547570 (+    67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:46.753074 (+205504us) raft_consensus.cc:1671] finished
0924 14:31:46.753100 (+    26us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:46.753231 (+   131us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1134164,"raft.queue_time_us":1143542,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":104,"thread_start_us":28283,"threads_started":1,"wal-append.queue_time_us":35063}
I20260924 14:31:46.709703 32384 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:46.755893 26401 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:46.757467 26401 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260924 14:31:46.762864 32190 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.766659 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.768709 32085 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.732246 32164 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260924 14:31:46.770133 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.770313 32267 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:46.674166 32178 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260924 14:31:46.771306 32385 consensus_queue.cc:1046] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:46.772488 32287 raft_consensus.cc:3032] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:46.772527 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1949) took 1103 ms. Trace:
I20260924 14:31:46.772990 32287 raft_consensus.cc:729] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:46.737843 32345 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:46.773080 26406 rpcz_store.cc:276] 0924 14:31:45.668999 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:45.669334 (+   335us) service_pool.cc:227] Handling call
0924 14:31:45.669781 (+   447us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:46.512728 (+842947us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:46.512758 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:46.513185 (+   427us) log.cc:842] Serialized 26 byte log entry
0924 14:31:46.632213 (+119028us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:46.632254 (+    41us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:46.632338 (+    84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:46.744069 (+111731us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b4800088d58. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:31:46.772215 (+ 28146us) raft_consensus.cc:1671] finished
0924 14:31:46.772239 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:46.772463 (+   224us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":842497,"raft.queue_time_us":844602,"raft.run_cpu_time_us":805,"raft.run_wall_time_us":228010,"spinlock_wait_cycles":41788160,"thread_start_us":118698,"threads_started":1,"wal-append.queue_time_us":169130}
I20260924 14:31:46.650810 32204 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:46.654332 32290 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:46.718973 32380 raft_consensus.cc:729] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:46.703013 31688 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260924 14:31:46.702749 32312 consensus_queue.cc:1046] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260924 14:31:46.708766 32277 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:46.784170 26405 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 11.
I20260924 14:31:46.781427 32231 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:46.792115 26254 raft_consensus.cc:1226] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
W20260924 14:31:46.792393 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 169621us
I20260924 14:31:46.793193 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:46.793193 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:46.802510 27192 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:46.785355 32166 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:46.804556 26406 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:46.791616 26369 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:46.808830 27191 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:46.812911 32204 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:46.808910 32231 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:46.817991 32231 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.820020 32276 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.823297 32290 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:46.823621 31875 raft_consensus.cc:3032] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:46.739383 32316 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:46.825989 32316 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:46.826329 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1945) took 1188 ms. Trace:
I20260924 14:31:46.826714 26402 rpcz_store.cc:276] 0924 14:31:45.637512 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:45.639578 (+  2066us) service_pool.cc:227] Handling call
0924 14:31:45.665750 (+ 26172us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:46.169965 (+504215us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:46.826089 (+656124us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b54002fbcd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:46.826261 (+   172us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":695335680}
I20260924 14:31:46.794402 32085 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:46.734184 32208 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:46.828641 32085 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:46.829164 26253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1391) took 2029 ms. Trace:
I20260924 14:31:46.829576 26253 rpcz_store.cc:276] 0924 14:31:44.800032 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:44.805790 (+  5758us) service_pool.cc:227] Handling call
0924 14:31:44.806262 (+   472us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:46.275177 (+1468915us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:46.828925 (+553748us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54004fae54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:46.829106 (+   181us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3157040896}
I20260924 14:31:46.830014 32208 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:46.728955 32387 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.087s
I20260924 14:31:46.830971 32166 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:46.831795 32342 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 1.027s	user 0.006s	sys 0.000s
I20260924 14:31:46.833202 32166 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:46.828080 26404 raft_consensus.cc:1261] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:46.838094 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:46.838449 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:46.839318 26256 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:46.823990 31875 raft_consensus.cc:729] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:46.844950 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:46.841159 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 49685.7us
I20260924 14:31:46.857483 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:46.830873 32085 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:46.861182 26254 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:46.844062 32166 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:46.815325 32260 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:46.848167 32306 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.018s	sys 0.013s
I20260924 14:31:46.870579 31984 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:46.872567 32331 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:46.874866 32270 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:46.799036 32287 consensus_queue.cc:260] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:46.880926 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:46.873466 27126 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:46.875043 32215 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:46.885166 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.891358 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:46.892727 26256 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:46.848479 32267 consensus_queue.cc:1046] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
W20260924 14:31:46.905318 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 173261us
W20260924 14:31:46.909937 32306 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.095s	user 0.018s	sys 0.013s
W20260924 14:31:46.936290 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45030.2us
I20260924 14:31:46.915660 31984 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:46.899610 32385 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:46.942941 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:46.931376 26255 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:46.949088 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:46.952315 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:46.904213 32387 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260924 14:31:46.941161 32178 raft_consensus.cc:3032] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:46.887301 31875 consensus_queue.cc:260] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:46.996519 31875 raft_consensus.cc:3037] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
I20260924 14:31:47.004081 26256 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260924 14:31:46.960630 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:47.006598 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:47.007318 26404 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:31:47.007814 32390 raft_consensus.cc:3032] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:47.008157 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:47.008447 32390 raft_consensus.cc:729] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:46.961777 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:46.964716 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:47.009757 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:47.009582 32390 consensus_queue.cc:260] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:47.011019 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:47.013571 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:46.970664 26406 raft_consensus.cc:1226] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260924 14:31:46.951889 32287 raft_consensus.cc:3037] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:47.016191 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:46.909970 32316 consensus_queue.cc:1046] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260924 14:31:46.963238 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:47.017460 32316 raft_consensus.cc:3032] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Stepping down as leader of term 11
I20260924 14:31:47.017822 32316 raft_consensus.cc:729] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:47.018872 26399 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:47.018711 32316 consensus_queue.cc:260] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:47.019976 32316 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:31:47.019788 32208 consensus_queue.cc:1159] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c22052275034c369b7f7dca086fbd1e" responder_term: 12 status { last_received { term: 10 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 11. Current term is 12. Ops: [11.8-11.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:31:47.021884 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.971999 32178 raft_consensus.cc:729] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:47.023149 32178 consensus_queue.cc:260] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:47.024181 26250 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260924 14:31:47.024257 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61445.3us
I20260924 14:31:46.976687 32197 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:31:47.025401 32197 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:47.025792 32197 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:47.026798 26255 raft_consensus.cc:1226] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260924 14:31:46.979738 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:46.983882 27192 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:46.950068 32290 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.078s
I20260924 14:31:46.860558 32380 consensus_queue.cc:260] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:47.028481 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:47.029251 26250 raft_consensus.cc:2379] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 18: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:47.030340 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:47.032106 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:46.986007 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 580099us
I20260924 14:31:47.032742 26253 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:31:47.033546 32380 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:47.034142 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:46.988516 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 469000us
I20260924 14:31:46.940523 32166 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260924 14:31:46.989296 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 414628us
I20260924 14:31:47.036296 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:47.038087 32261 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:47.038740 32261 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:47.040799 32261 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:47.041312 32261 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:47.041785 32261 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:47.042425 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:46.947871 32143 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260924 14:31:46.947324 32204 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260924 14:31:46.989573 32231 consensus_queue.cc:1046] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:46.990330 26400 raft_consensus.cc:2396] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260924 14:31:46.994287 32395 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:47.046691 32395 raft_consensus.cc:3032] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:47.047178 32395 raft_consensus.cc:729] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:47.047493 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:47.050082 27186 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:46.991940 32260 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:46.957222 31924 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
I20260924 14:31:46.947042 31984 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:47.052031 26406 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260924 14:31:47.052498 26406 pending_rounds.cc:76] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 8
I20260924 14:31:47.052772 26406 pending_rounds.cc:97] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260924 14:31:46.989477 31945 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:47.055357 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:47.055905 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:47.008837 26404 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:46.988698 32146 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:46.989840 32075 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:47.044852 27188 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:47.052187 31924 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:47.009760 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:47.053027 26406 raft_consensus.cc:2866] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:46.974038 32317 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:47.035425 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:47.055526 26324 raft_consensus.cc:1261] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:47.057377 26250 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260924 14:31:47.064785 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.065305 26331 raft_consensus.cc:1226] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260924 14:31:47.024448 32178 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:46.948778 32338 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.122s	user 0.020s	sys 0.015s
I20260924 14:31:47.078315 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:47.070072 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:47.078231 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:47.078593 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:47.139995 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:47.140503 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:47.141419 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 103446us
W20260924 14:31:47.143353 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 78887.5us
I20260924 14:31:47.144805 27190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:47.145602 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:47.146487 26330 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:47.147637 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:47.145174 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 72405us
I20260924 14:31:47.142597 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:46.963191 32385 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:47.153286 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.098986 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:47.159075 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.160503 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:47.163383 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335"
I20260924 14:31:47.174288 27182 raft_consensus.cc:2354] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17.
W20260924 14:31:47.150926 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 94612.1us
I20260924 14:31:47.170363 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:47.056519 32235 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:47.217125 32208 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:47.177686 32143 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260924 14:31:47.218741 32143 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:47.219098 32143 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:47.219998 32143 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:47.221256 32143 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:47.223564 32326 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:47.224396 32236 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:47.225822 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.226477 32236 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:31:47.227523 31882 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:47.227025 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.230082 31688 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:47.232939 32368 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:47.233424 32368 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:47.233857 32368 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:47.079226 32146 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:47.237277 32146 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:47.238869 32190 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:47.239526 32190 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:47.240477 32190 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:47.211823 32238 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260924 14:31:47.245690 32261 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:47.249347 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:47.250470 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:47.251437 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:47.079960 32075 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:47.178553 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 90643.3us
I20260924 14:31:47.181531 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:47.149501 32338 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.323s	user 0.021s	sys 0.015s
I20260924 14:31:47.209973 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:46.970834 32277 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:47.196693 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:47.206700 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:47.202498 26221 leader_election.cc:398] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17.
W20260924 14:31:47.211594 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 52629.4us
I20260924 14:31:47.291970 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.293354 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.294691 26221 leader_election.cc:401] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260924 14:31:47.212669 32322 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:47.183197 26404 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:47.226691 32208 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:47.132947 31924 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:47.188810 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:47.196053 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.267939 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:47.269068 27191 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:47.178098 32259 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260924 14:31:47.303046 26327 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: [15.10-16.11]
I20260924 14:31:47.281138 32204 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:47.283089 32326 consensus_queue.cc:1225] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 7e09fbebe29a49d291d23f9d395fe335 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260924 14:31:47.283957 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 57009.1us
W20260924 14:31:47.284864 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52185.8us
W20260924 14:31:47.354830 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 158864us
W20260924 14:31:47.354981 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 61734.2us
I20260924 14:31:47.323875 26324 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:47.324828 26401 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:47.274205 32395 consensus_queue.cc:260] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:47.303601 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 101306us
I20260924 14:31:47.261056 32236 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
W20260924 14:31:47.343827 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 470403us
I20260924 14:31:47.287343 32075 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:47.355640 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.296658 32238 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:47.356858 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.327214 32182 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:47.299873 32166 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:47.296008 31945 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:47.372346 26224 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.370307 27126 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:47.063675 31979 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:47.264360 32235 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:47.365432 26298 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:47.385016 27191 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:47.388454 26256 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:47.350073 32186 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:47.275338 32197 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:47.369838 32236 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:47.376827 32322 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260924 14:31:47.417315 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 51969.1us
I20260924 14:31:47.419411 32188 raft_consensus.cc:2781] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Leader election won for term 11
I20260924 14:31:47.420141 32188 raft_consensus.cc:686] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:47.420326 32153 consensus_queue.cc:1035] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:47.421306 32188 consensus_queue.cc:237] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:47.390662 32166 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:47.394603 32238 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:47.429296 26256 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:47.438766 26326 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:47.444509 27185 raft_consensus.cc:1226] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260924 14:31:47.415706 32290 raft_consensus.cc:2781] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260924 14:31:47.453508 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 56201.9us
I20260924 14:31:47.436880 32166 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:47.468327 26326 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:47.445529 32259 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:47.486676 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:47.414319 32200 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260924 14:31:47.409880 31979 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:47.524466 32235 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:47.402015 31924 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:47.537271 32403 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260924 14:31:47.537060 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 65477.5us
I20260924 14:31:47.537066 31924 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:47.482355 32236 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:47.482096 32390 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:47.452257 32322 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260924 14:31:47.561254 27192 raft_consensus.cc:1226] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260924 14:31:47.472401 32326 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:47.585031 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:47.597313 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:47.598835 26398 raft_consensus.cc:2379] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:31:47.599797 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 227534us
I20260924 14:31:47.601110 26224 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:47.551322 32316 consensus_meta.cc:287] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.531s	user 0.006s	sys 0.001s
I20260924 14:31:47.560920 32236 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:47.705035 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:31:47.565697 32197 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:47.714068 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:47.714839 26403 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:47.717015 26298 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:47.671464 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:47.671778 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 133053us
I20260924 14:31:47.557161 32334 raft_consensus.cc:3032] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:47.672134 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:47.561695 32395 raft_consensus.cc:3037] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:47.723124 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 51086.2us
I20260924 14:31:47.730046 26224 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:47.733609 26224 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:47.670740 32259 raft_consensus.cc:2726] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:47.586058 32146 consensus_queue.cc:1046] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.135s
I20260924 14:31:47.746802 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:47.746806 26331 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260924 14:31:47.705142 32277 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:47.739988 32399 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260924 14:31:47.750993 26331 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:31:47.595620 32326 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:47.756189 32277 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:47.739969 32335 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260924 14:31:47.757323 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
W20260924 14:31:47.757651 32401 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.429s	user 0.002s	sys 0.002s
W20260924 14:31:47.758874 32409 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260924 14:31:47.761289 32424 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
W20260924 14:31:47.763386 32287 consensus_meta.cc:287] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.747s	user 0.003s	sys 0.007s
I20260924 14:31:47.741703 32382 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:47.769604 32166 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:47.720809 32334 raft_consensus.cc:729] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:47.755468 26331 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260924 14:31:47.781533 26324 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:47.754395 31979 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:47.799320 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 47004.6us
I20260924 14:31:47.713336 32271 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260924 14:31:47.781458 32259 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:47.817070 26406 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260924 14:31:47.875427 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 67177.1us
W20260924 14:31:47.876122 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 339470us
W20260924 14:31:47.860960 32178 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.711s	user 0.011s	sys 0.000s
I20260924 14:31:47.886250 27192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:47.893657 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 78235.6us
I20260924 14:31:47.861054 32259 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:47.914378 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:47.918000 27181 raft_consensus.cc:1226] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:47.919433 26397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1964) took 1249 ms. Trace:
W20260924 14:31:48.020267 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 143324us
I20260924 14:31:47.929358 26397 rpcz_store.cc:276] 0924 14:31:46.669937 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:46.677294 (+  7357us) service_pool.cc:227] Handling call
0924 14:31:46.705989 (+ 28695us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:46.829252 (+123263us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:46.829284 (+    32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:46.829758 (+   474us) log.cc:842] Serialized 26 byte log entry
0924 14:31:46.948564 (+118806us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:46.948619 (+    55us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:46.988826 (+ 40207us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540018ced4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:46.988878 (+    52us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:47.676375 (+687497us) spinlock_profiling.cc:243] Waited 87.2 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:47.919105 (+242730us) raft_consensus.cc:1671] finished
0924 14:31:47.919144 (+    39us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:47.919350 (+   206us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8,"raft.queue_time_us":1209,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":483186432,"thread_start_us":118420,"threads_started":1,"wal-append.queue_time_us":353313}
W20260924 14:31:48.037592 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 128033us
W20260924 14:31:48.041416 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 135161us
I20260924 14:31:48.039533 26295 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:48.036173 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:48.024375 31677 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:48.043099 32402 log.cc:924] Time spent T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
W20260924 14:31:48.053049 32368 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.819s	user 0.005s	sys 0.007s
W20260924 14:31:48.069828 26328 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.019s	sys 0.032s
W20260924 14:31:48.070881 26328 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.019s	sys 0.032s
I20260924 14:31:47.898756 32334 consensus_queue.cc:260] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:48.088584 26323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1766) took 1033 ms. Trace:
I20260924 14:31:48.091812 26323 rpcz_store.cc:276] 0924 14:31:47.054962 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.055072 (+   110us) service_pool.cc:227] Handling call
0924 14:31:47.078996 (+ 23924us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:47.079224 (+   228us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:47.453695 (+374471us) log.cc:842] Serialized 14 byte log entry
0924 14:31:48.088211 (+634516us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:48.088276 (+    65us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:48.088327 (+    51us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.088394 (+    67us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:48.088523 (+   129us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1892,"prepare.queue_time_us":429623,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":296426,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":17333,"thread_start_us":904540,"threads_started":3,"wal-append.queue_time_us":313909}
I20260924 14:31:48.077960 26328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1767) took 1022 ms. Trace:
W20260924 14:31:48.085466 31875 consensus_meta.cc:287] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.088s	user 0.014s	sys 0.000s
I20260924 14:31:48.098229 26328 rpcz_store.cc:276] 0924 14:31:47.055351 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.056249 (+   898us) service_pool.cc:227] Handling call
0924 14:31:47.056569 (+   320us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.056964 (+   395us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:47.056994 (+    30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:47.057442 (+   448us) log.cc:842] Serialized 26 byte log entry
0924 14:31:47.452951 (+395509us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:48.077276 (+624325us) log.cc:842] Serialized 14 byte log entry
0924 14:31:48.077555 (+   279us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.077673 (+   118us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:48.077736 (+    63us) raft_consensus.cc:1671] finished
0924 14:31:48.077748 (+    12us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:48.077901 (+   153us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":76566,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":789,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":14356,"spinlock_wait_cycles":1189760,"thread_start_us":983210,"threads_started":2,"wal-append.queue_time_us":652133}
I20260924 14:31:48.064874 31677 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:48.078573 31924 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:48.098465 32334 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:48.067407 32368 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:48.112377 32277 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:48.158761 32426 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.361s	user 0.001s	sys 0.001s
W20260924 14:31:48.155315 32415 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260924 14:31:48.170665 32422 log.cc:924] Time spent T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260924 14:31:48.176705 31984 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.124s	user 0.013s	sys 0.000s
I20260924 14:31:48.161275 31979 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260924 14:31:48.236320 32429 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260924 14:31:48.240312 32428 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260924 14:31:48.180543 31924 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:48.161624 32325 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:48.383891 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 337658us
W20260924 14:31:48.357003 32182 consensus_meta.cc:287] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.960s	user 0.009s	sys 0.004s
I20260924 14:31:48.182730 32143 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:48.092684 32259 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:48.336050 32441 consensus_queue.cc:1159] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e09fbebe29a49d291d23f9d395fe335" responder_term: 14 status { last_received { term: 14 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260924 14:31:48.337421 26404 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.008s	sys 0.040s
W20260924 14:31:48.398274 26404 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.008s	sys 0.040s
I20260924 14:31:48.183065 32394 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:48.343080 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.934s	user 0.020s	sys 0.042s
W20260924 14:31:48.399361 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.020s	sys 0.042s
I20260924 14:31:48.137246 32326 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:48.372355 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 327791us
W20260924 14:31:48.340552 32404 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 1.098s	user 0.003s	sys 0.009s
W20260924 14:31:48.381254 26298 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.024s	sys 0.033s
W20260924 14:31:48.417318 26298 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.024s	sys 0.033s
W20260924 14:31:48.418072 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 701100us
W20260924 14:31:48.329840 26369 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.151.195:45507 timed out after 1.549s (SENT) [suppressed 6 similar messages]
W20260924 14:31:48.386686 26398 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.029s	sys 0.019s
W20260924 14:31:48.421056 26398 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.029s	sys 0.019s
W20260924 14:31:48.374387 32426 log.cc:924] Time spent T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260924 14:31:48.390046 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:48.399797 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 990869us
I20260924 14:31:48.412789 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:48.378154 32271 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.328227 31984 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:48.393537 32182 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943) with higher term. Message: Invalid argument: T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17.
I20260924 14:31:48.462716 27194 raft_consensus.cc:1226] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260924 14:31:48.417419 32442 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:48.399729 32143 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:48.472929 26399 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.035s	sys 0.046s
W20260924 14:31:48.473387 26399 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.035s	sys 0.046s
W20260924 14:31:48.475227 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 439110us
I20260924 14:31:48.476459 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:48.478186 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:48.416520 32394 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:48.482635 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:48.498430 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:48.442685 32271 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:48.508399 26249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 2230) took 1255 ms. Trace:
I20260924 14:31:48.483564 32442 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:48.496181 32326 raft_consensus.cc:2741] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260924 14:31:48.528900 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1979) took 1033 ms. Trace:
I20260924 14:31:48.514635 26249 rpcz_store.cc:276] 0924 14:31:47.253306 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.254038 (+   732us) service_pool.cc:227] Handling call
0924 14:31:47.289887 (+ 35849us) raft_consensus.cc:1457] Updating replica for 2 ops
0924 14:31:47.290265 (+   378us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:47.290294 (+    29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0924 14:31:47.371750 (+ 81456us) log.cc:842] Serialized 52 byte log entry
0924 14:31:47.747101 (+375351us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:47.773398 (+ 26297us) log.cc:842] Serialized 14 byte log entry
0924 14:31:47.773651 (+   253us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:47.773722 (+    71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:48.508120 (+734398us) raft_consensus.cc:1671] finished
0924 14:31:48.508150 (+    30us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:48.508322 (+   172us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":950,"prepare.queue_time_us":34008,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":6331,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":21577,"spinlock_wait_cycles":3136768,"thread_start_us":334906,"threads_started":1,"wal-append.queue_time_us":698030}
W20260924 14:31:48.520150 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 473105us
I20260924 14:31:48.529774 26406 rpcz_store.cc:276] 0924 14:31:47.495475 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.499037 (+  3562us) service_pool.cc:227] Handling call
0924 14:31:47.499416 (+   379us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.860519 (+361103us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.528507 (+667988us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b5400450454. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:48.528815 (+   308us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":725496576}
W20260924 14:31:48.511483 32423 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.334s	user 0.000s	sys 0.001s
W20260924 14:31:48.516650 32380 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 1.482s	user 0.017s	sys 0.000s
W20260924 14:31:48.510864 32075 consensus_meta.cc:287] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.127s	user 0.011s	sys 0.002s
I20260924 14:31:48.534054 26325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1260) took 1480 ms. Trace:
I20260924 14:31:48.540802 26325 rpcz_store.cc:276] 0924 14:31:47.054027 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.054229 (+   202us) service_pool.cc:227] Handling call
0924 14:31:47.054608 (+   379us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.054952 (+   344us) raft_consensus.cc:1520] Early marking committed up to index 8
0924 14:31:47.369760 (+314808us) log.cc:842] Serialized 14 byte log entry
0924 14:31:48.089495 (+719735us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:48.096131 (+  6636us) log.cc:842] Serialized 26 byte log entry
0924 14:31:48.096321 (+   190us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:48.096447 (+   126us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.096531 (+    84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:48.533741 (+437210us) raft_consensus.cc:1671] finished
0924 14:31:48.533773 (+    32us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:48.533968 (+   195us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":66959,"prepare.queue_time_us":381549,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":4067,"raft.queue_time_us":298631,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":20235,"spinlock_wait_cycles":12094848,"thread_start_us":977606,"threads_started":3,"wal-append.queue_time_us":684456}
I20260924 14:31:48.519865 31677 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:48.552964 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:48.562130 32443 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260924 14:31:48.564764 32271 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:48.565634 31924 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:48.566262 31924 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.566747 31924 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.569219 31924 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:48.569840 31924 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.571969 32332 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:48.572695 32332 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:48.574815 26252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1413) took 1187 ms. Trace:
I20260924 14:31:48.575248 26252 rpcz_store.cc:276] 0924 14:31:47.387060 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.387193 (+   133us) service_pool.cc:227] Handling call
0924 14:31:47.391342 (+  4149us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.591193 (+199851us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:47.591222 (+    29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:47.671460 (+ 80238us) log.cc:842] Serialized 26 byte log entry
0924 14:31:48.088759 (+417299us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:48.088815 (+    56us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.088899 (+    84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:48.574621 (+485722us) raft_consensus.cc:1671] finished
0924 14:31:48.574641 (+    20us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:48.574759 (+   118us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":33651,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":8982,"thread_start_us":544785,"threads_started":2,"wal-append.queue_time_us":679934}
I20260924 14:31:48.578244 32143 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:48.580044 32439 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260924 14:31:48.581377 26400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1974) took 1095 ms. Trace:
W20260924 14:31:48.582664 32448 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260924 14:31:48.586174 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:48.586678 31875 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.587205 31875 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.581720 26400 rpcz_store.cc:276] 0924 14:31:47.485867 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.488774 (+  2907us) service_pool.cc:227] Handling call
0924 14:31:47.505836 (+ 17062us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.568357 (+ 62521us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:47.568403 (+    46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:47.582079 (+ 13676us) log.cc:842] Serialized 26 byte log entry
0924 14:31:48.156825 (+574746us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:48.156885 (+    60us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.156982 (+    97us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:48.581173 (+424191us) raft_consensus.cc:1671] finished
0924 14:31:48.581194 (+    21us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:48.581318 (+   124us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10551,"raft.queue_time_us":72324,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":3352,"thread_start_us":442800,"threads_started":1,"wal-append.queue_time_us":842110}
W20260924 14:31:48.540868 32434 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.281s	user 0.001s	sys 0.001s
I20260924 14:31:48.519867 32440 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:48.527918 32453 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:48.589382 31875 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.535885 32368 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:48.541893 32326 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:48.601769 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 271979us
W20260924 14:31:48.608505 32423 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260924 14:31:48.548199 31980 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.574900 32332 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:48.615507 26296 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.035s	sys 0.059s
W20260924 14:31:48.644433 26296 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.035s	sys 0.061s
I20260924 14:31:48.626559 26369 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:48.646454 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:48.647349 26399 raft_consensus.cc:2396] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260924 14:31:48.621227 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:48.646335 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:48.649247 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:48.652079 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:48.651280 26369 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:48.661042 26249 raft_consensus.cc:1226] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:48.661203 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:48.549639 32075 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.570324 31924 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.619359 32190 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.670449 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:48.614236 32311 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:48.667877 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:48.604707 32433 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:48.678359 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 169300us
W20260924 14:31:48.620105 32430 log.cc:924] Time spent T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260924 14:31:48.688661 26327 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:48.652215 32447 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:48.647806 32453 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:48.695976 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:48.696287 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:48.681187 32437 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.259s	user 0.001s	sys 0.001s
I20260924 14:31:48.682236 26369 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:48.686517 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:48.644450 32184 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:48.663569 32192 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.681836 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:48.645937 31980 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:48.677652 31984 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:48.676661 32457 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.620157 32326 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:48.681131 32458 raft_consensus.cc:2726] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260924 14:31:48.699867 32317 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.549s	user 0.016s	sys 0.000s
W20260924 14:31:48.711508 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 177171us
I20260924 14:31:48.725930 32454 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:48.726616 32454 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:48.688540 32271 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260924 14:31:48.727634 26329 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:48.729044 26398 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260924 14:31:48.729523 26398 pending_rounds.cc:76] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 10
I20260924 14:31:48.729789 26398 pending_rounds.cc:97] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260924 14:31:48.705760 32380 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:48.730029 26398 raft_consensus.cc:2866] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:31:48.730283 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 66197.3us
I20260924 14:31:48.682776 32190 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.723805 32259 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:48.725461 32458 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:48.734705 32458 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:48.735178 32458 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:48.736907 32462 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.737478 32462 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:48.709168 32184 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:48.739704 32435 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.740357 32435 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:48.741123 32184 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.732213 26399 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:48.714105 32192 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:48.742754 32259 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.743993 32192 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:48.744571 31924 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:48.745229 31924 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:48.745445 32259 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.746419 26323 raft_consensus.cc:2454] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:48.746768 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:48.687855 32311 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:48.728739 31677 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:48.721179 32382 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.785558 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:48.786338 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:48.782305 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 104442us
I20260924 14:31:48.745841 31924 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.749239 32192 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.805187 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:48.700049 32447 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.810174 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:48.817061 26402 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 23.
I20260924 14:31:48.752902 32394 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:48.846776 32469 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:48.847479 32182 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:48.756976 32259 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:48.853384 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
I20260924 14:31:48.747176 32380 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:467: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260924 14:31:48.704905 32453 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.791735 32368 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:48.796073 32464 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:48.847641 32466 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:48.889286 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:48.869127 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 145296us
I20260924 14:31:48.722960 32457 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.845722 32462 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:48.874845 32390 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 1.309s	user 0.005s	sys 0.016s
I20260924 14:31:48.902240 32473 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:48.903054 32486 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260924 14:31:48.891703 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 86621.3us
I20260924 14:31:48.932811 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:48.845347 31924 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:48.897073 32475 consensus_queue.cc:1046] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:48.803336 31677 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:48.908181 32470 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:48.907779 27185 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1633) took 1863 ms. Trace:
I20260924 14:31:48.843678 32192 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:48.906365 32317 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:48.808610 32382 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.873520 32182 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:48.941421 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 79706.1us
I20260924 14:31:48.942932 27185 rpcz_store.cc:276] 0924 14:31:47.043958 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.046731 (+  2773us) service_pool.cc:227] Handling call
0924 14:31:47.047028 (+   297us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.445107 (+398079us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.907280 (+1462173us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400260a54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:48.907693 (+   413us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":895078272}
I20260924 14:31:48.924510 32075 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.943909 27182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:48.968109 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 65394.7us
I20260924 14:31:48.973192 26399 raft_consensus.cc:2396] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260924 14:31:48.892710 32259 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.907734 32368 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.968302 23135 tablet_server.cc:178] TabletServer@127.22.151.193:38943 shutting down...
I20260924 14:31:48.940464 32466 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:48.980036 27186 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:31:48.968918 32236 consensus_meta.cc:287] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.259s	user 0.012s	sys 0.004s
I20260924 14:31:48.976086 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e09fbebe29a49d291d23f9d395fe335" is_pre_election: true
W20260924 14:31:48.976439 32455 log.cc:924] Time spent T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
W20260924 14:31:48.982707 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 91318.3us
I20260924 14:31:48.941797 32461 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:48.979772 32395 consensus_meta.cc:287] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 1.239s	user 0.000s	sys 0.015s
I20260924 14:31:48.934880 32473 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.868891 32469 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:48.995551 32236 consensus_queue.cc:1046] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:48.996007 32311 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.974059 32317 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:48.997334 26403 raft_consensus.cc:1261] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260924 14:31:49.003750 26403 pending_rounds.cc:76] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 9
I20260924 14:31:49.004109 26403 pending_rounds.cc:97] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260924 14:31:49.004420 26403 raft_consensus.cc:2866] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:49.010828 32472 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:49.011720 32472 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:48.999897 32456 log.cc:924] Time spent T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260924 14:31:48.990489 32326 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:48.985565 32454 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.992066 31980 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.005928 32277 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.034677 32192 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.036108 32259 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.036787 32479 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260924 14:31:49.038231 32236 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:48.996908 32461 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:49.009733 32182 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:48.988034 32484 consensus_queue.cc:1035] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260924 14:31:49.044904 32453 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.045131 32473 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:49.045832 32446 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.047760 26324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1300) took 1514 ms. Trace:
I20260924 14:31:49.050874 32469 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:48.962409 32470 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.008291 32190 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:49.120211 32401 log.cc:924] Time spent T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260924 14:31:49.047801 32380 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.161288 32447 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:49.173708 32334 consensus_meta.cc:287] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.993s	user 0.008s	sys 0.005s
I20260924 14:31:49.122555 32277 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.175194 32472 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.176451 32382 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:49.177631 32467 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260924 14:31:49.223176 32473 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.184866 31688 raft_consensus.cc:3032] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:49.222779 32446 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.168717 26324 rpcz_store.cc:276] 0924 14:31:47.533173 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.819023 (+285850us) service_pool.cc:227] Handling call
0924 14:31:47.819431 (+   408us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.859844 (+ 40413us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:47.859894 (+    50us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:47.898601 (+ 38707us) log.cc:842] Serialized 26 byte log entry
0924 14:31:48.546869 (+648268us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:48.606882 (+ 60013us) log.cc:842] Serialized 14 byte log entry
0924 14:31:48.611644 (+  4762us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.611720 (+    76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:49.047521 (+435801us) raft_consensus.cc:1671] finished
0924 14:31:49.047545 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:49.047690 (+   145us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44307,"prepare.queue_time_us":59310,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":4500,"raft.queue_time_us":283690,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":18677,"spinlock_wait_cycles":1075456,"thread_start_us":489245,"threads_started":1,"wal-append.queue_time_us":654030}
I20260924 14:31:49.214460 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46104 (request call id 1670) took 1478 ms. Trace:
I20260924 14:31:49.253927 26403 rpcz_store.cc:276] 0924 14:31:47.735981 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.886015 (+150034us) service_pool.cc:227] Handling call
0924 14:31:47.917623 (+ 31608us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.005004 (+1087381us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.214137 (+209133us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001c1e54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:49.214378 (+   241us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2481948032}
I20260924 14:31:49.130510 32238 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.235819 26332 raft_consensus.cc:1261] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260924 14:31:49.180593 26330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1769) took 2107 ms. Trace:
I20260924 14:31:49.262115 26330 rpcz_store.cc:276] 0924 14:31:47.073450 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.165447 (+ 91997us) service_pool.cc:227] Handling call
0924 14:31:47.165778 (+   331us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:47.259940 (+ 94162us) raft_consensus.cc:1520] Early marking committed up to index 10
0924 14:31:47.591628 (+331688us) log.cc:842] Serialized 14 byte log entry
0924 14:31:48.106157 (+514529us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:48.147757 (+ 41600us) raft_consensus.cc:1631] Marking committed up to 10
0924 14:31:48.148031 (+   274us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.180160 (+1032129us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b5400558cd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:49.180234 (+    74us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:49.180501 (+   267us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":47780,"prepare.queue_time_us":269748,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":371684,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":8326,"spinlock_wait_cycles":215935360,"thread_start_us":728668,"threads_started":2,"wal-append.queue_time_us":789325}
I20260924 14:31:49.247392 32457 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:49.251056 31688 raft_consensus.cc:729] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:49.276191 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 302990us
I20260924 14:31:49.250469 31924 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.270107 31677 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.268107 32166 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.350392 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:49.250099 32473 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.274035 32368 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.363022 32289 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260924 14:31:49.364279 32289 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:49.364691 32289 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 23 FOLLOWER]: Advancing to term 24
I20260924 14:31:49.364751 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.365957 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.366173 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.366926 26403 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:49.367854 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.369609 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.372249 32461 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.313097 32259 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.373483 32259 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.373495 32289 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.376327 32470 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:49.349651 32317 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.376363 32289 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 24 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.378589 32473 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.380268 32446 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.267529 32238 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.381174 26404 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:49.382397 26400 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
W20260924 14:31:49.362118 32458 consensus_meta.cc:287] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: Time spent flushing consensus metadata: real 0.626s	user 0.004s	sys 0.008s
I20260924 14:31:49.362551 31688 consensus_queue.cc:260] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:49.363024 32488 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260924 14:31:49.382025 32326 raft_consensus.cc:3032] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Stepping down as leader of term 20
I20260924 14:31:49.406190 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:49.383023 32368 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:49.357529 26251 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.000s	sys 0.014s
W20260924 14:31:49.354836 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 331573us
W20260924 14:31:49.410153 26251 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.000s	sys 0.015s
W20260924 14:31:49.384615 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 449549us
W20260924 14:31:49.356724 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 446830us
I20260924 14:31:49.414621 27126 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:49.405656 32470 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:49.406843 32271 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.011s	sys 0.004s
I20260924 14:31:49.417698 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:49.420549 26397 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 23 FOLLOWER]: Advancing to term 24
I20260924 14:31:49.417361 32470 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.423858 32470 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.406570 32326 raft_consensus.cc:729] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:49.413358 32424 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.414852 31688 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:49.412760 32259 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.416719 32391 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:49.428325 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:49.390784 32458 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.438001 26254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55802 (request call id 2232) took 2184 ms. Trace:
I20260924 14:31:49.443336 32322 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.443534 26401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1977) took 2332 ms. Trace:
I20260924 14:31:49.444082 32192 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:49.445727 26332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1292) took 2090 ms. Trace:
I20260924 14:31:49.457703 26332 rpcz_store.cc:276] 0924 14:31:47.355708 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.761576 (+405868us) service_pool.cc:227] Handling call
0924 14:31:47.761841 (+   265us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.261997 (+1500156us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.445502 (+183505us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54005bb8d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:49.445667 (+   165us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3388535168}
I20260924 14:31:49.444772 31979 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:49.445185 26326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1765) took 2390 ms (client timeout 2157 ms). Trace:
W20260924 14:31:49.462234 26326 rpcz_store.cc:269] 0924 14:31:47.054571 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.054684 (+   113us) service_pool.cc:227] Handling call
0924 14:31:49.444978 (+2390294us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b5400505854. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:49.445092 (+   114us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":700235648}
I20260924 14:31:49.445727 26331 raft_consensus.cc:1226] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:49.454094 26401 rpcz_store.cc:276] 0924 14:31:47.111004 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.111149 (+   145us) service_pool.cc:227] Handling call
0924 14:31:47.324639 (+213490us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:47.356904 (+ 32265us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.443328 (+2086424us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400290750. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:49.443478 (+   150us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":333028608}
I20260924 14:31:49.453868 26254 rpcz_store.cc:276] 0924 14:31:47.253904 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.254123 (+   219us) service_pool.cc:227] Handling call
0924 14:31:47.289476 (+ 35353us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:47.327814 (+ 38338us) raft_consensus.cc:1520] Early marking committed up to index 12
0924 14:31:47.578699 (+250885us) log.cc:842] Serialized 14 byte log entry
0924 14:31:47.585516 (+  6817us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:47.585571 (+    55us) raft_consensus.cc:1631] Marking committed up to 12
0924 14:31:47.585614 (+    43us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.558415 (+972801us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b54006c53a8. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bcc95361 00007f09bcd2cebb 00007f09bcd2ce09 00007f09bcd2cd99 00007f09bcd2cd61 00007f09bcd2c05d 00007f09bf68a204 00007f09bf68a039 00007f09b64266b6 00007f09b6429b11 00007f09b6429ac9
0924 14:31:49.437721 (+879306us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b54004c9d54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:49.437775 (+    54us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:49.437915 (+   140us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":199912,"prepare.run_cpu_time_us":23695,"prepare.run_wall_time_us":1030604,"raft.queue_time_us":4123,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":84304,"spinlock_wait_cycles":101265024,"thread_start_us":250591,"threads_started":1}
I20260924 14:31:49.425629 32481 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.460340 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:49.426829 32306 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260924 14:31:49.426468 32271 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.011s	sys 0.004s
I20260924 14:31:49.416882 32368 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.451346 32277 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.452863 32391 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:49.469695 31979 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:49.452059 32290 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:49.468997 26397 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 24.
W20260924 14:31:49.480728 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 466471us
I20260924 14:31:49.481894 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:49.485713 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:49.430326 32332 consensus_queue.cc:1046] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260924 14:31:49.470718 32306 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:49.470434 32494 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260924 14:31:49.483157 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.011s	sys 0.007s
I20260924 14:31:49.484387 27126 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:49.490279 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:49.510571 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.495008 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.507822 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:49.480347 32496 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:49.511184 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.011s	sys 0.007s
I20260924 14:31:49.511462 26257 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:49.511693 26332 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:49.512135 26251 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:49.514479 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:49.456907 32424 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.484259 32290 raft_consensus.cc:3037] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:49.461932 32322 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.484009 32391 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:49.515027 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 539569us
I20260924 14:31:49.518388 26330 raft_consensus.cc:2396] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260924 14:31:49.520804 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:49.557132 26397 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:49.481948 32238 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.508929 32306 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:49.514588 32486 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:49.541252 32186 consensus_queue.cc:1035] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260924 14:31:49.522900 26221 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.006s	sys 0.013s
W20260924 14:31:49.583021 26221 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.006s	sys 0.018s
W20260924 14:31:49.583464 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 779168us
W20260924 14:31:49.488425 32441 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.040s	sys 0.093s
I20260924 14:31:49.555059 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.558296 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.556659 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:49.514804 31984 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 24 FOLLOWER]: Leader election won for term 24
I20260924 14:31:49.469949 32481 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:49.522130 32345 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260924 14:31:49.585579 32497 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:49.587169 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.593964 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.594604 26402 raft_consensus.cc:2379] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:49.588161 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:49.594616 26401 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 13.
I20260924 14:31:49.482237 32326 consensus_queue.cc:260] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:49.585088 32442 consensus_meta.cc:287] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.042s	user 0.001s	sys 0.015s
I20260924 14:31:49.565161 32153 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:49.640607 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 84040.6us
I20260924 14:31:49.648001 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.641407 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:49.587432 32441 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.041s	sys 0.098s
I20260924 14:31:49.650547 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.643257 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.652465 26249 raft_consensus.cc:2396] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260924 14:31:49.651603 26250 raft_consensus.cc:2396] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260924 14:31:49.657528 32479 raft_consensus.cc:3032] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:49.657998 32479 raft_consensus.cc:729] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:49.658782 32153 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.661147 32153 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.650712 31984 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 24 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:49.662684 31984 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.667539 27194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50086 (request call id 1065) took 2582 ms. Trace:
I20260924 14:31:49.592756 32497 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:49.669539 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 84661.3us
I20260924 14:31:49.667963 27194 rpcz_store.cc:276] 0924 14:31:47.084906 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.086924 (+  2018us) service_pool.cc:227] Handling call
0924 14:31:47.087369 (+   445us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:48.465864 (+1378495us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.667238 (+1201374us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54002c51d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:49.667477 (+   239us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3162191104}
I20260924 14:31:49.670248 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.680169 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:49.678175 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.678404 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:49.678833 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:49.636430 32433 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.641605 32440 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.693281 32440 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.694300 32306 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.695060 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:49.673727 32485 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.044s	sys 0.087s
I20260924 14:31:49.658810 32479 consensus_queue.cc:260] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.607597 32447 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:49.695881 32485 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.044s	sys 0.088s
I20260924 14:31:49.698509 32497 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:49.698840 32330 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:31:49.699311 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:49.700186 32440 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:49.701241 32447 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.676813 26402 raft_consensus.cc:2396] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260924 14:31:49.684379 26295 leader_election.cc:334] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:49.702695 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:49.681429 26252 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
W20260924 14:31:49.703294 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:49.703495 32447 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.639284 32470 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:49.704236 32271 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:49.704838 32481 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.706367 32271 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:49.706959 32271 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:49.687016 26298 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:49.712917 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:49.517581 32496 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.689824 32334 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.689980 32441 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:49.690606 32322 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.691987 32290 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.692678 32345 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:49.640141 31980 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:49.701187 32433 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.671835 26397 raft_consensus.cc:2454] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:49.719959 31335 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:49.651588 32442 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:49.652659 32331 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:49.717737 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:49.720635 32493 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:49.695811 32458 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.679934 32382 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.725642 32345 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:49.593550 32486 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:49.699249 32330 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:49.726359 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:49.642845 32449 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.020s	sys 0.055s
I20260924 14:31:49.826066 26399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35444 (request call id 1998) took 1040 ms. Trace:
I20260924 14:31:49.826629 26399 rpcz_store.cc:276] 0924 14:31:48.785380 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:48.786089 (+   709us) service_pool.cc:227] Handling call
0924 14:31:49.760311 (+974222us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54001e41d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:49.825987 (+ 65676us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":399833984}
I20260924 14:31:49.746845 31335 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:49.723424 32334 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.723645 32441 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:49.749027 32331 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:49.824419 32449 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.021s	sys 0.055s
I20260924 14:31:49.752674 32368 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:49.737556 31980 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.843086 32330 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:49.915450 32166 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.933298 31335 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:49.939638 32331 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:49.770731 32382 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:49.825201 32271 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:49.963814 32345 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.050899 32382 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.042217 32290 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:50.042464 32496 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:50.052757 32433 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.063015 32441 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:50.072113 32486 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:50.068881 32345 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:50.123184 32306 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:50.142829 32391 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.002s
I20260924 14:31:50.150374 32391 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.153853 32391 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:49.682147 26257 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:50.232299 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.234042 26250 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:50.234814 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.232345 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.236083 26252 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:50.231069 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:49.752066 26298 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:50.247488 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 495326us
W20260924 14:31:50.249424 26295 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:50.251736 26326 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.005s	sys 0.001s
W20260924 14:31:50.252383 26326 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.005s	sys 0.001s
W20260924 14:31:50.256551 26369 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.011s	sys 0.028s
W20260924 14:31:50.257027 26369 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.011s	sys 0.028s
W20260924 14:31:50.257547 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 702386us
I20260924 14:31:50.258618 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.266358 26330 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:50.269712 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.272936 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.274319 26332 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:31:50.274972 26399 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:50.275755 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.276273 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.277102 26399 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:50.276897 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.277503 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.278148 26401 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:49.799417 26402 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:50.279410 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:50.280685 26404 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.002s	sys 0.019s
I20260924 14:31:50.278158 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:50.279721 26398 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.000s	sys 0.017s
W20260924 14:31:50.298906 27123 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.001s	sys 0.010s
W20260924 14:31:50.300387 26404 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.003s	sys 0.019s
W20260924 14:31:50.300800 31984 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.068s	sys 0.116s
W20260924 14:31:50.301025 27125 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.009s	sys 0.004s
W20260924 14:31:50.301286 31984 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.068s	sys 0.117s
W20260924 14:31:50.301558 27125 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.009s	sys 0.004s
W20260924 14:31:50.280424 26403 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.002s	sys 0.019s
W20260924 14:31:50.302218 26403 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.003s	sys 0.020s
W20260924 14:31:50.302156 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 605398us
I20260924 14:31:50.279876 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:50.363101 26398 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.000s	sys 0.018s
W20260924 14:31:50.300760 27123 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.002s	sys 0.010s
W20260924 14:31:50.380261 26400 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.004s	sys 0.003s
W20260924 14:31:50.380887 26400 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.005s	sys 0.004s
I20260924 14:31:50.387236 26399 raft_consensus.cc:2379] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:50.396642 32238 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.397964 32238 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:50.399997 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 692404us
I20260924 14:31:50.405381 32380 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.408001 32368 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.408634 32380 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.409071 32368 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.411641 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.413697 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.414705 26403 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:50.386626 31882 raft_consensus.cc:2781] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
W20260924 14:31:50.419665 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 116208us
I20260924 14:31:50.425453 31980 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:50.426365 31980 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:49.976538 32166 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.438331 32166 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.438485 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.439456 26330 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:50.440196 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.441428 26324 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:50.443073 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.444926 32382 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.445981 32382 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.447811 31945 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:50.448963 31945 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.449788 32425 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.452311 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.467715 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.468858 32466 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.468981 26323 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:50.469597 32466 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:50.471068 32166 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.471674 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.471920 32166 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.472563 26401 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:50.473564 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.475008 32368 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:50.459308 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.479583 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.479804 32290 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:50.482201 32290 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.485301 32153 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.486279 32153 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.487579 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.488672 26332 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:50.489476 31882 raft_consensus.cc:686] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:50.492013 32290 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:50.468047 32380 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.492883 32382 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.493323 32380 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.494156 32380 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.496254 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.496585 32466 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:50.496822 32475 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:50.491511 31882 consensus_queue.cc:237] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.492297 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:50.498399 26403 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:50.450786 32425 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:50.587085 26397 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:50.497733 32475 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.591248 32153 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.596979 26401 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260924 14:31:50.611788 26401 pending_rounds.cc:76] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 10
I20260924 14:31:50.599766 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:50.600167 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.603987 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.604727 26403 raft_consensus.cc:1226] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260924 14:31:50.605713 32425 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.427151 31980 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.607124 32433 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.588610 32166 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.625857 26401 pending_rounds.cc:97] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260924 14:31:50.638895 32235 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.639904 32235 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.641270 26326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50658 (request call id 1784) took 1129 ms. Trace:
I20260924 14:31:50.641798 26326 rpcz_store.cc:276] 0924 14:31:49.511762 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:49.516757 (+  4995us) service_pool.cc:227] Handling call
0924 14:31:49.551910 (+ 35153us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.552172 (+   262us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:49.580709 (+ 28537us) log.cc:842] Serialized 14 byte log entry
0924 14:31:50.631666 (+1050957us) log.cc:842] Serialized 14 byte log entry
0924 14:31:50.640681 (+  9015us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:50.640744 (+    63us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:50.640834 (+    90us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.640946 (+   112us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:50.641161 (+   215us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":33485,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":163045,"raft.run_cpu_time_us":486,"raft.run_wall_time_us":490,"thread_start_us":1035093,"threads_started":2,"wal-append.queue_time_us":561048}
I20260924 14:31:50.627149 31945 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:50.643640 32331 consensus_meta.cc:287] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.621s	user 0.008s	sys 0.003s
I20260924 14:31:50.669783 32331 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:50.671691 32433 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:50.642530 32235 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.652927 32153 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:50.655920 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:50.674312 32433 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.673707 32153 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.656375 26401 raft_consensus.cc:2866] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:31:50.658524 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:50.677263 32238 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.677824 32238 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:50.678345 26295 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:50.678438 32238 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:50.679657 32290 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260924 14:31:50.661924 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 169846us
I20260924 14:31:50.681151 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:50.682152 32166 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:50.682497 32331 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:50.682940 32166 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.648389 32380 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.663439 32475 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:50.683916 32331 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:50.685477 32166 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:50.685551 26295 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:50.686371 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:50.707384 26332 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:50.687345 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:50.712051 26327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50660 (request call id 1264) took 3537 ms. Trace:
I20260924 14:31:50.712669 26327 rpcz_store.cc:276] 0924 14:31:47.175459 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.342557 (+167098us) service_pool.cc:227] Handling call
0924 14:31:47.342941 (+   384us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:48.689111 (+1346170us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.708866 (+2019755us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540001e0d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:50.711826 (+  2960us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3094374784}
I20260924 14:31:50.723937 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:50.725507 32235 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:50.726241 32235 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:50.726819 32235 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:50.688598 32153 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:50.763770 32306 raft_consensus.cc:2781] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:50.764461 32306 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:50.769259 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 83759.5us
I20260924 14:31:50.746162 32433 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:50.764954 32306 raft_consensus.cc:3037] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:50.773864 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 108370us
I20260924 14:31:50.781605 26404 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1992) took 1218 ms. Trace:
W20260924 14:31:50.785279 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:50.798007 26298 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:50.799981 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 116709us
W20260924 14:31:50.801642 32438 log.cc:924] Time spent T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 2.362s	user 0.030s	sys 0.043s
I20260924 14:31:50.804060 26405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 1975) took 3317 ms. Trace:
I20260924 14:31:50.805497 26398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 1998) took 1153 ms. Trace:
I20260924 14:31:50.804464 26405 rpcz_store.cc:276] 0924 14:31:47.486169 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.488880 (+  2711us) service_pool.cc:227] Handling call
0924 14:31:47.505201 (+ 16321us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:47.516172 (+ 10971us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:47.516216 (+    44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:47.569449 (+ 53233us) log.cc:842] Serialized 26 byte log entry
0924 14:31:48.074026 (+504577us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:48.074064 (+    38us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:48.074143 (+    79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:49.043109 (+968966us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b4800088d58. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09be1f57f1 00007f09be1ec8e7 00007f09be1ec77a 00007f09be2e095f 00007f09be2e0627 00007f09be2f4b2c 00007f09be2f4ad8 00007f09be2f4a58 00007f09be2f4a13 00007f09be2f3cbc 00007f09bd275be6
0924 14:31:50.749885 (+1706776us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:50.803628 (+ 53743us) raft_consensus.cc:1671] finished
0924 14:31:50.803652 (+    24us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:50.803840 (+   188us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31627,"raft.queue_time_us":62833,"raft.run_cpu_time_us":3389,"raft.run_wall_time_us":1463703,"spinlock_wait_cycles":22954240,"thread_start_us":403147,"threads_started":1,"wal-append.queue_time_us":799935}
I20260924 14:31:50.805904 26398 rpcz_store.cc:276] 0924 14:31:49.652737 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:49.653024 (+   287us) service_pool.cc:227] Handling call
0924 14:31:49.653366 (+   342us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.653594 (+   228us) raft_consensus.cc:1520] Early marking committed up to index 11
0924 14:31:50.431687 (+778093us) log.cc:842] Serialized 14 byte log entry
0924 14:31:50.742716 (+311029us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:50.747564 (+  4848us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:50.775824 (+ 28260us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.786709 (+ 10885us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:50.805435 (+ 18726us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":29260,"prepare.queue_time_us":483032,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":39,"raft.queue_time_us":59560,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":19785,"thread_start_us":967678,"threads_started":2,"wal-append.queue_time_us":314889}
I20260924 14:31:50.812340 32238 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:50.814597 32507 log.cc:924] Time spent T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 1.115s	user 0.012s	sys 0.030s
I20260924 14:31:50.818121 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.822423 31984 consensus_queue.cc:1046] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:50.828686 26331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54090 (request call id 1293) took 3452 ms. Trace:
I20260924 14:31:50.829229 26331 rpcz_store.cc:276] 0924 14:31:47.375891 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.800664 (+424773us) service_pool.cc:227] Handling call
0924 14:31:49.445557 (+1644893us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.467296 (+ 21739us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.828238 (+1360942us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400505850. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934da09 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:50.828492 (+   254us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3774703360}
W20260924 14:31:50.844182 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47558.2us
W20260924 14:31:50.852473 31335 consensus_meta.cc:287] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.846s	user 0.008s	sys 0.008s
I20260924 14:31:50.852993 31335 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.787001 26404 rpcz_store.cc:276] 0924 14:31:49.555935 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:49.556051 (+   116us) service_pool.cc:227] Handling call
0924 14:31:49.597602 (+ 41551us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.597852 (+   250us) raft_consensus.cc:1520] Early marking committed up to index 9
0924 14:31:50.301306 (+703454us) log.cc:842] Serialized 14 byte log entry
0924 14:31:50.709753 (+408447us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:50.736596 (+ 26843us) raft_consensus.cc:1631] Marking committed up to 9
0924 14:31:50.745437 (+  8841us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.753244 (+  7807us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:50.773929 (+ 20685us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11821,"prepare.queue_time_us":535809,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":315,"raft.queue_time_us":243503,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":48208,"thread_start_us":1025091,"threads_started":3,"wal-append.queue_time_us":161013}
I20260924 14:31:50.860703 26369 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:50.861477 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.862308 26404 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:50.863478 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.865078 32424 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:50.815279 32330 consensus_meta.cc:287] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.840s	user 0.012s	sys 0.006s
I20260924 14:31:50.868669 32330 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:50.850111 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 49194.1us
I20260924 14:31:50.855198 31335 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:50.900089 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48718.5us
I20260924 14:31:50.892478 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:50.914256 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:50.863222 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:50.930219 26332 raft_consensus.cc:2454] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:50.931798 32530 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:50.936940 32526 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.104s	user 0.002s	sys 0.000s
W20260924 14:31:50.952389 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 89355.1us
I20260924 14:31:50.958339 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.958989 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:50.960511 26250 raft_consensus.cc:1261] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:50.961971 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.962108 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.962863 26251 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:50.963161 26256 raft_consensus.cc:2396] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:50.963965 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.964591 32442 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:50.965342 32442 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:50.967085 26252 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:50.965725 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:50.967664 26258 raft_consensus.cc:1261] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260924 14:31:50.969143 26258 pending_rounds.cc:76] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 5
I20260924 14:31:50.948704 32153 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:50.940546 32235 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:50.975152 32334 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:50.977563 26399 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260924 14:31:50.977982 26258 pending_rounds.cc:97] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260924 14:31:50.978346 26258 raft_consensus.cc:2866] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:50.990557 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.993194 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:50.999123 31677 consensus_queue.cc:1035] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260924 14:31:50.997769 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.002094 26404 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:50.988826 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.002071 32184 raft_consensus.cc:3032] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:50.972206 32306 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:51.013810 26399 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:51.003669 32442 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:51.015938 26329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50658 (request call id 1771) took 3938 ms (client timeout 1657 ms). Trace:
I20260924 14:31:51.016076 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:51.016474 26329 rpcz_store.cc:269] 0924 14:31:47.077534 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:47.139725 (+ 62191us) service_pool.cc:227] Handling call
0924 14:31:51.015485 (+3875760us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b540001dbd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:51.015659 (+   174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3649623424}
I20260924 14:31:51.017042 26251 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:51.026142 32192 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260924 14:31:51.028515 32517 consensus_queue.cc:1046] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260924 14:31:51.031608 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.032421 26254 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:31:51.008425 32460 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.231s	user 0.026s	sys 0.030s
W20260924 14:31:51.036005 32460 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.027s	sys 0.030s
W20260924 14:31:51.037529 31979 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 1.484s	user 0.008s	sys 0.023s
I20260924 14:31:51.038178 31979 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.040728 31979 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.041898 31980 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.042192 26255 raft_consensus.cc:1226] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 18. Current term is 19. Ops: []
I20260924 14:31:51.044746 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.044948 32435 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.045459 26402 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 17
I20260924 14:31:51.047626 32424 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:51.048421 32424 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:51.062765 32325 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:51.074137 32325 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.077179 32330 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:51.079945 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 72722.4us
I20260924 14:31:51.066069 32306 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.072405 27181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1050) took 2636 ms. Trace:
I20260924 14:31:51.062835 27192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50480 (request call id 1650) took 2406 ms. Trace:
I20260924 14:31:51.083612 27192 rpcz_store.cc:276] 0924 14:31:48.655806 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:48.656103 (+   297us) service_pool.cc:227] Handling call
0924 14:31:48.656454 (+   351us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:48.795680 (+139226us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:48.795713 (+    33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:48.795756 (+    43us) raft_consensus.cc:1631] Marking committed up to 12
0924 14:31:48.795788 (+    32us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:51.062182 (+2266394us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400453b54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:51.062269 (+    87us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:51.062584 (+   315us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":319728384}
I20260924 14:31:51.007390 32184 raft_consensus.cc:729] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:51.094527 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.094854 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:51.090273 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 99878us
I20260924 14:31:51.056080 32424 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.098815 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.050647 31335 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:51.092043 32235 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.096508 32330 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.097875 32535 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:51.120340 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:51.120427 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 56701.1us
I20260924 14:31:51.122638 26399 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:51.123356 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:51.130364 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:51.132314 26295 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.136807 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.140162 26253 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:31:51.121104 32535 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.083302 27181 rpcz_store.cc:276] 0924 14:31:48.435842 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:48.435980 (+   138us) service_pool.cc:227] Handling call
0924 14:31:48.457531 (+ 21551us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:48.543855 (+ 86324us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:48.543905 (+    50us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:48.544531 (+   626us) log.cc:842] Serialized 26 byte log entry
0924 14:31:48.694434 (+149903us) raft_consensus.cc:1631] Marking committed up to 8
0924 14:31:48.694471 (+    37us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:51.071604 (+2377133us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b540000ee54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:51.071692 (+    88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:51.071799 (+   107us) raft_consensus.cc:1671] finished
0924 14:31:51.071816 (+    17us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:51.072178 (+   362us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6755,"raft.queue_time_us":10604,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":38479,"spinlock_wait_cycles":181891456,"thread_start_us":149551,"threads_started":1,"wal-append.queue_time_us":183829,"wal-append.run_cpu_time_us":1853,"wal-append.run_wall_time_us":1006740}
I20260924 14:31:51.111658 32306 consensus_queue.cc:1046] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260924 14:31:51.155303 26406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35444 (request call id 2004) took 1791 ms. Trace:
I20260924 14:31:51.158179 26406 rpcz_store.cc:276] 0924 14:31:49.363757 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:49.366474 (+  2717us) service_pool.cc:227] Handling call
0924 14:31:49.366754 (+   280us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:49.389393 (+ 22639us) raft_consensus.cc:1520] Early marking committed up to index 11
0924 14:31:49.413784 (+ 24391us) log.cc:842] Serialized 14 byte log entry
0924 14:31:49.414072 (+   288us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0924 14:31:49.414139 (+    67us) raft_consensus.cc:1631] Marking committed up to 11
0924 14:31:49.414201 (+    62us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:49.742362 (+328161us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b1800243160. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b6426638 00007f09b6429b11
0924 14:31:50.743964 (+1001602us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b1800243160. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09bf2c58b1 00007f09bf2d2661 00007f09b63f3f96 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9
0924 14:31:51.154915 (+410951us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b540023fd54. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:51.154963 (+    48us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:51.155127 (+   164us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":24123,"raft.queue_time_us":103,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3835,"spinlock_wait_cycles":63040128}
I20260924 14:31:51.157670 32184 consensus_queue.cc:260] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.160569 32325 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.163548 32330 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.164897 26402 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:51.165596 32473 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:51.143392 32535 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.171329 32184 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:51.156378 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.155118 32479 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.127942 31335 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:51.184630 32380 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:51.185336 32380 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:51.160699 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:51.186328 32326 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:31:51.143724 26295 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.187435 32380 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.185627 31335 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:51.187834 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:51.188983 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 50387.1us
W20260924 14:31:51.189642 26295 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:51.190866 26295 leader_election.cc:334] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.192144 32153 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:51.200966 32153 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:51.200626 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.201905 26397 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:31:51.203018 32529 log.cc:924] Time spent T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.371s	user 0.001s	sys 0.001s
I20260924 14:31:51.193958 32440 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:51.205860 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:51.207237 32473 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:51.213402 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.212901 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:51.218210 32538 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260924 14:31:51.205937 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.211243 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.218432 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.213614 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:51.191414 26295 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:51.237370 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45973us
I20260924 14:31:51.237944 26258 raft_consensus.cc:2396] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260924 14:31:51.227927 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 87786.5us
I20260924 14:31:51.218815 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.233736 26251 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 12.
W20260924 14:31:51.238631 26295 leader_election.cc:334] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.240756 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.238862 26254 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:51.246817 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:51.244660 26295 leader_election.cc:334] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:51.248970 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.253908 26255 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:51.250101 31945 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:51.259807 31945 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.260154 31945 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.266637 26253 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
I20260924 14:31:51.269685 27126 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:51.272104 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 157366us
I20260924 14:31:51.298692 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.307498 26397 raft_consensus.cc:2396] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260924 14:31:51.306190 26249 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:51.282498 31335 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:51.314976 32450 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:51.315718 32330 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.315745 32450 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:51.317306 32450 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.317708 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:51.319777 32424 consensus_queue.cc:1046] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:51.321601 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.322487 26332 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:51.323614 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.323921 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.322384 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.322288 32517 raft_consensus.cc:3032] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Stepping down as leader of term 18
I20260924 14:31:51.333993 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.351938 26397 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260924 14:31:51.355993 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.357893 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.360239 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.360924 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:51.361778 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.362594 26401 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 7e09fbebe29a49d291d23f9d395fe335 in this term.
I20260924 14:31:51.365592 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.367873 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.361033 26404 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:51.369992 26403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46098 (request call id 2005) took 1098 ms. Trace:
I20260924 14:31:51.373836 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.375140 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.334072 32166 raft_consensus.cc:2741] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260924 14:31:51.381827 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:51.352401 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:51.356487 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:51.374521 26332 raft_consensus.cc:2454] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:51.374315 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.380321 26400 raft_consensus.cc:2379] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:51.322510 32259 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:51.385443 26331 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260924 14:31:51.370566 26403 rpcz_store.cc:276] 0924 14:31:50.271187 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:50.728993 (+457806us) service_pool.cc:227] Handling call
0924 14:31:50.729544 (+   551us) raft_consensus.cc:1457] Updating replica for 1 ops
0924 14:31:50.730088 (+   544us) raft_consensus.cc:1520] Early marking committed up to index 0
0924 14:31:50.730135 (+    47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0924 14:31:50.731003 (+   868us) log.cc:842] Serialized 26 byte log entry
0924 14:31:50.890723 (+159720us) raft_consensus.cc:1631] Marking committed up to 5
0924 14:31:50.890847 (+   124us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:50.890950 (+   103us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0924 14:31:51.369695 (+478745us) raft_consensus.cc:1671] finished
0924 14:31:51.369728 (+    33us) raft_consensus.cc:1677] UpdateReplicas() finished
0924 14:31:51.369919 (+   191us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":159129,"threads_started":1,"wal-append.queue_time_us":558842}
I20260924 14:31:51.340313 32517 raft_consensus.cc:729] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:51.388294 26295 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.404533 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.382737 32530 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:51.406013 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:51.406281 32530 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.407986 32527 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:51.408658 32530 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.408583 32527 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.409360 32530 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:51.409776 32530 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:51.410197 32530 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:51.410673 32527 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.393541 32466 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:51.411916 32235 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.412640 32235 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.413657 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.412631 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.414714 26397 raft_consensus.cc:2379] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 14: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:51.415946 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.417178 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.417598 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.418174 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:51.421937 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:51.422966 26295 leader_election.cc:334] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:51.421857 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 147536us
W20260924 14:31:51.398543 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 73439us
I20260924 14:31:51.418268 26403 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 10.
I20260924 14:31:51.415205 32235 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.398093 32517 consensus_queue.cc:260] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.436349 32494 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.436961 32494 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.437418 32517 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:51.440757 32494 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:51.433573 32153 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.445209 32153 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.411250 32527 consensus_queue.cc:1046] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260924 14:31:51.446781 32527 raft_consensus.cc:3032] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:51.447176 32527 raft_consensus.cc:729] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:51.447019 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.448484 32527 consensus_queue.cc:260] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.449220 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.450428 32166 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:51.451061 32166 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.453503 32166 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:51.427583 26398 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
W20260924 14:31:51.423609 26295 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.456125 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.384543 32259 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:51.457845 32259 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
W20260924 14:31:51.458613 26295 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.461553 32330 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:51.462324 32330 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:51.463944 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.465258 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.469264 26249 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:51.437457 27125 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:51.481452 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:51.456491 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:51.423291 32497 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.018s	sys 0.041s
I20260924 14:31:51.493702 26256 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:31:51.499934 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.506871 26298 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:51.513264 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 75968.1us
W20260924 14:31:51.500800 32497 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.019s	sys 0.041s
I20260924 14:31:51.468104 32494 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.524825 26369 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:51.504945 32527 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:51.531826 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 169767us
W20260924 14:31:51.534209 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 82748.4us
I20260924 14:31:51.513091 32153 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:51.541915 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.520990 32331 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:31:51.526248 32442 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:51.558748 26298 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:51.559592 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.567755 26404 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:51.560223 26405 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:51.572908 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:51.496024 32330 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.554665 32424 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:51.554189 32325 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:51.587210 32458 raft_consensus.cc:2781] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:51.587955 32458 raft_consensus.cc:686] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:51.589100 32458 consensus_queue.cc:237] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.562145 32331 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:51.592566 32331 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.583668 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.571614 32473 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.551795 32494 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:51.606115 32494 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.599964 26255 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:51.579514 32424 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.607308 32424 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:51.602381 32535 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.610306 32535 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.574833 32440 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:31:51.633488 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 183835us
I20260924 14:31:51.643994 32390 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:51.644547 32390 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.644924 32390 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.647923 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.648764 26254 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:51.650306 31945 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.614233 32473 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.642259 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.659713 26255 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 16
I20260924 14:31:51.662323 26400 raft_consensus.cc:1226] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 23. Current term is 24. Ops: []
I20260924 14:31:51.665565 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.664454 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.668308 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.666568 26404 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:51.672627 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.673288 26324 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:51.674207 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.617426 32325 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.676527 26252 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:51.679184 26298 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:51.683303 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.684162 26328 raft_consensus.cc:2354] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18.
I20260924 14:31:51.685387 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.688282 31945 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.688891 32473 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.686244 26331 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:51.698709 26298 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:51.703994 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.704879 26328 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.624716 32546 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260924 14:31:51.707434 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:51.712908 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:51.642640 32461 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.182s
I20260924 14:31:51.718211 32503 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:51.717324 32530 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.714243 31335 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:51.709404 31945 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:51.661270 32547 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.078s
I20260924 14:31:51.736289 26369 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:51.718487 32153 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:51.740502 32535 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:51.741014 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:51.730968 32462 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.014s	sys 0.017s
I20260924 14:31:51.768397 26399 raft_consensus.cc:1226] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:51.745785 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.726027 32503 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.752131 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.786031 26257 raft_consensus.cc:2396] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260924 14:31:51.759001 32153 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:51.768826 32462 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.016s	sys 0.019s
I20260924 14:31:51.743733 32325 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.763026 31335 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:51.784186 31945 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:51.788197 32503 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:51.767371 32535 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:51.798944 32153 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:51.843257 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 261217us
I20260924 14:31:51.833592 32535 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:51.847949 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:51.848892 32530 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:51.850831 32326 consensus_meta.cc:287] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.664s	user 0.008s	sys 0.012s
I20260924 14:31:51.859540 26329 raft_consensus.cc:2396] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260924 14:31:51.861217 32259 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.863540 31980 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:51.877014 31980 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.878639 32380 raft_consensus.cc:3032] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:51.879143 32380 raft_consensus.cc:729] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:51.864384 32550 log.cc:924] Time spent T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
W20260924 14:31:51.872566 32184 consensus_meta.cc:287] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.000s
I20260924 14:31:51.874683 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:51.889082 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.887887 26255 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:51.892210 26328 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:51.895988 26252 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:51.902287 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.910192 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.909857 26369 leader_election.cc:398] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18.
I20260924 14:31:51.912016 26369 leader_election.cc:401] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260924 14:31:51.913872 26256 raft_consensus.cc:1226] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:51.915767 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.915849 31979 consensus_queue.cc:1046] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260924 14:31:51.916790 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.917615 32535 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:51.919018 32259 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.921362 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.921162 26331 raft_consensus.cc:3032] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 23 LEADER]: Stepping down as leader of term 23
I20260924 14:31:51.921906 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:51.909969 26295 leader_election.cc:334] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.922246 26331 raft_consensus.cc:729] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:51.924062 26327 raft_consensus.cc:2454] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:51.930701 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:51.933703 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:51.934957 26295 leader_election.cc:334] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:51.857474 32424 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:51.936640 32390 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:51.938568 32424 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.939513 32390 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:51.944216 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.944496 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.945458 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.946022 26253 raft_consensus.cc:2362] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16. Re-sending same reply.
I20260924 14:31:51.946887 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.947836 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.949105 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:51.950704 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:51.933952 26252 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:51.952576 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.955224 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.928681 32473 raft_consensus.cc:3037] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:51.966485 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:51.968885 26327 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260924 14:31:51.964476 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:51.958622 32535 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:51.966588 26255 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:31:51.975067 26249 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:51.982177 26298 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:51.989794 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:51.990736 26404 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
I20260924 14:31:51.997581 26398 raft_consensus.cc:2396] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260924 14:31:52.001816 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 111674us
I20260924 14:31:51.911372 31335 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.007148 26400 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:51.972741 32075 consensus_queue.cc:1046] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
W20260924 14:31:52.025260 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 150675us
I20260924 14:31:51.993268 32326 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:52.003687 32462 raft_consensus.cc:3032] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:52.005106 32442 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:51.994841 31980 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:51.930104 31945 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.030694 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:52.016215 32527 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:52.040948 26369 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:52.037285 26224 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:52.047415 32497 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:52.047904 32497 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:52.047807 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:52.048321 32497 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:52.048533 32332 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:52.049528 32326 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:52.052232 32535 raft_consensus.cc:2781] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:52.052801 32535 raft_consensus.cc:686] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:52.053608 32535 consensus_queue.cc:237] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.032116 32462 raft_consensus.cc:729] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:52.036988 32563 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:52.081399 26249 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260924 14:31:52.068086 32424 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260924 14:31:52.078478 26295 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:52.095713 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 98494.6us
I20260924 14:31:52.042964 32527 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.065063 32153 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:52.110328 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 81401.7us
I20260924 14:31:52.091013 32424 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:52.115408 26324 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 18: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:52.110351 32171 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
I20260924 14:31:52.126031 32462 consensus_queue.cc:260] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.127298 32462 raft_consensus.cc:3037] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:52.116421 32424 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:52.135972 32557 log.cc:924] Time spent T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.272s	user 0.003s	sys 0.000s
I20260924 14:31:52.113031 32326 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:52.143678 32534 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:52.144304 26331 consensus_queue.cc:260] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.145915 26331 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 23 FOLLOWER]: Advancing to term 24
I20260924 14:31:52.148821 32527 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:52.154994 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:52.157902 26331 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260924 14:31:52.098584 32450 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260924 14:31:52.159865 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 117493us
I20260924 14:31:52.165899 26325 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:52.106096 32497 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:52.160219 32325 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 24 current term: 24 is higher.
I20260924 14:31:52.177527 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.178395 26398 raft_consensus.cc:1226] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260924 14:31:52.178717 26331 raft_consensus.cc:2379] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:52.182740 32497 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:52.184312 27126 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:52.186067 32502 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:52.186717 32502 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:52.176628 32517 consensus_meta.cc:287] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.739s	user 0.011s	sys 0.014s
I20260924 14:31:52.187991 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:52.189293 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:52.191239 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:52.197059 26295 leader_election.cc:334] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:52.198933 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:52.191957 26406 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
W20260924 14:31:52.207842 26371 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.865s (SENT)
I20260924 14:31:52.208534 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.189235 32424 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.212496 32424 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:52.225754 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:52.227630 26331 raft_consensus.cc:1261] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260924 14:31:52.227784 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:52.232801 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:52.233537 26399 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 19
I20260924 14:31:52.235737 32497 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:52.237885 26255 raft_consensus.cc:1226] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 10. Current term is 11. Ops: []
I20260924 14:31:52.201828 32473 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 17. Current term is 18.
I20260924 14:31:52.240743 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:52.241755 32558 consensus_queue.cc:1046] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:52.242779 32558 raft_consensus.cc:3032] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:52.243116 32558 raft_consensus.cc:729] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:52.244007 32558 consensus_queue.cc:260] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.244768 32534 consensus_queue.cc:1046] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:52.245586 32558 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:52.246260 32534 raft_consensus.cc:3032] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Stepping down as leader of term 10
I20260924 14:31:52.246753 32534 raft_consensus.cc:729] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:52.251907 32547 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:52.252408 32547 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:52.252769 32547 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:52.259868 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 148631us
I20260924 14:31:52.261140 32153 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:52.262382 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:52.263759 32330 raft_consensus.cc:2781] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:52.264281 32330 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:52.264739 32330 raft_consensus.cc:3037] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:52.267403 32442 raft_consensus.cc:3032] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:52.267904 32442 raft_consensus.cc:729] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:52.268694 32442 consensus_queue.cc:260] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.270123 32442 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:52.270179 32534 consensus_queue.cc:260] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.272338 32534 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Advancing to term 11
W20260924 14:31:52.275218 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 146328us
I20260924 14:31:52.236054 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:52.284981 26399 raft_consensus.cc:2454] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:52.290397 26403 raft_consensus.cc:2454] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:31:52.291146 26405 raft_consensus.cc:1226] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:52.291640 26398 raft_consensus.cc:1226] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260924 14:31:52.295112 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 64655.2us
I20260924 14:31:52.297609 32458 consensus_queue.cc:1035] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 14:31:52.298691 26224 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:52.300496 32424 raft_consensus.cc:2781] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:52.301232 32424 raft_consensus.cc:686] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 19 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:52.303936 32568 log.cc:924] Time spent T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.151s	user 0.002s	sys 0.000s
I20260924 14:31:52.305686 26406 raft_consensus.cc:1226] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:52.306209 32380 consensus_queue.cc:260] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.318351 32380 raft_consensus.cc:3037] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:31:52.312089 32565 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260924 14:31:52.283803 32503 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:52.326342 32503 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:52.293217 32332 consensus_queue.cc:1046] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
W20260924 14:31:52.333884 32562 log.cc:924] Time spent T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260924 14:31:52.315672 32204 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:52.302453 32424 consensus_queue.cc:237] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:52.339372 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:52.343058 26256 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:31:52.343551 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46635.7us
I20260924 14:31:52.352046 32204 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:52.351471 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 85612.2us
I20260924 14:31:52.380795 26257 raft_consensus.cc:1261] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260924 14:31:52.383919 32204 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:52.389550 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:52.392059 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:52.400482 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 53158.2us
I20260924 14:31:52.396224 26256 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
W20260924 14:31:52.410696 26297 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.724s (SENT)
I20260924 14:31:52.411123 26297 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:52.417820 27125 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:52.416689 32325 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:52.419400 32506 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:52.421131 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:52.423038 32506 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:52.424945 32458 raft_consensus.cc:2741] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260924 14:31:52.424314 32506 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:52.447762 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 58349.8us
I20260924 14:31:52.448585 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:52.452453 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47099.7us
I20260924 14:31:52.453928 32330 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:52.420382 32450 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:52.458377 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:52.459306 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 362604us
W20260924 14:31:52.468044 32580 log.cc:924] Time spent T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260924 14:31:52.469444 32559 log.cc:924] Time spent T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.427s	user 0.000s	sys 0.001s
I20260924 14:31:52.471333 32184 raft_consensus.cc:3032] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Stepping down as leader of term 19
I20260924 14:31:52.471753 32184 raft_consensus.cc:729] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:52.472620 32184 consensus_queue.cc:260] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:52.473387 32200 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:52.474275 32184 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:52.463249 32325 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260924 14:31:52.477206 32325 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:52.477540 32325 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Advancing to term 22
W20260924 14:31:52.494415 32569 log.cc:924] Time spent T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.351s	user 0.002s	sys 0.000s
I20260924 14:31:52.489197 26298 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.501771 31945 consensus_queue.cc:1035] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260924 14:31:52.504709 26327 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:52.505085 32326 consensus_queue.cc:1035] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 14:31:52.506075 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.506733 26327 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:52.506830 32473 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:52.507386 32473 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:52.507941 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:52.508639 26398 raft_consensus.cc:2379] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:52.508520 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.509552 32473 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:52.509949 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.510470 32330 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:52.510561 26327 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:52.511919 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.513236 26327 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:52.513275 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:52.502858 32573 raft_consensus.cc:2781] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:52.514990 32573 raft_consensus.cc:686] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:52.515877 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.515946 32573 consensus_queue.cc:237] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.517328 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:52.519059 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:52.520907 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 72374.4us
I20260924 14:31:52.524626 32547 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.528579 32547 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:52.532023 26297 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.533057 32153 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:52.533732 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:52.537415 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.538357 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:52.543618 26402 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:52.548763 31335 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:52.442281 32380 consensus_queue.cc:1046] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.055s
I20260924 14:31:52.550556 32380 consensus_queue.cc:1225] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260924 14:31:52.550642 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.551146 32380 raft_consensus.cc:3032] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:52.551497 32380 raft_consensus.cc:729] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:52.552374 32380 consensus_queue.cc:260] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:52.553521 32380 raft_consensus.cc:3037] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:52.554054 31335 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:52.555593 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:52.556408 31335 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260924 14:31:52.558265 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.561064 32204 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:52.563794 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:52.569803 32204 raft_consensus.cc:2781] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:52.570319 32204 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:52.570699 32204 raft_consensus.cc:3037] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:52.572333 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:52.573179 31335 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:52.583638 31335 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:52.584091 31335 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:52.560366 26402 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:52.582064 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.586145 32153 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:52.586539 26224 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:52.590847 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:52.583648 31984 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:52.590936 32424 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:52.586549 32153 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:52.600819 32543 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:52.632337 32462 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260924 14:31:52.636931 32543 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:52.640056 31984 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:52.642356 31984 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:52.643496 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:52.644174 26257 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:52.604497 32442 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:52.646878 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:52.592157 32517 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:52.616228 32534 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260924 14:31:52.647567 26406 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:52.645619 27125 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:52.650882 26297 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:52.604594 26402 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260924 14:31:52.648437 31335 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:52.636066 32153 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:52.639277 32543 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:52.655133 32424 raft_consensus.cc:686] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:52.631099 32325 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:52.703016 31984 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:52.753634 31984 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:52.756948 31984 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:52.656816 26402 pending_rounds.cc:76] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 7
I20260924 14:31:52.823648 26402 pending_rounds.cc:97] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260924 14:31:52.824333 26402 raft_consensus.cc:2866] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260924 14:31:52.988426 26371 leader_election.cc:334] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.680s (SENT)
I20260924 14:31:52.994102 26371 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:52.996181 31335 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:52.997082 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 298773us
I20260924 14:31:53.003644 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:52.999940 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 395626us
I20260924 14:31:52.986439 32380 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:52.994764 32075 raft_consensus.cc:2781] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:53.020236 32075 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:53.029249 32075 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:53.019434 32153 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.008416 31945 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:53.049000 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:53.004690 32543 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.060014 31945 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.021174 32380 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.062407 31945 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:53.063131 32380 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:53.064092 32380 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:53.067809 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:53.068552 31984 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.069576 26297 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:53.070919 31984 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.071412 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:53.073141 32204 consensus_meta.cc:287] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.013s
W20260924 14:31:53.115362 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 125207us
I20260924 14:31:53.112514 32200 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.119395 32200 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.121632 32200 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.076859 32458 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:53.118623 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.123533 32547 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:53.125473 26221 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.126345 32547 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:53.115123 31945 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:53.127692 26221 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:53.071627 31979 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:53.071817 32326 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:53.071923 32573 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:31:53.136708 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 65376.6us
I20260924 14:31:53.138464 26297 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:53.141387 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:53.144403 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.087411 32330 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.077189 32543 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.136168 32547 raft_consensus.cc:2741] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260924 14:31:53.155747 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:53.139672 32326 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:53.145670 32558 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:53.162235 32547 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:53.162701 32547 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:53.163051 32547 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.166015 32075 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.152043 32506 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.150300 31979 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:53.179883 32153 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.189881 26073 catalog_manager.cc:5495] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:53.190513 32506 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.110523 32424 consensus_queue.cc:237] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.140332 32204 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:53.197633 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 59286.1us
I20260924 14:31:53.204541 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.153299 32330 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.153211 32380 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:53.158761 32326 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:53.168326 32558 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:53.168069 32325 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:53.222988 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 67351.9us
I20260924 14:31:53.224927 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:53.184214 32473 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.191335 31979 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:53.218511 32153 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:53.193851 32506 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.243952 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:53.248334 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.249212 26406 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.226982 32558 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:53.250931 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:53.240164 32461 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:53.256453 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:53.251948 26331 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:53.257198 26330 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:53.257922 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:53.259865 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:53.259919 26325 raft_consensus.cc:1226] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:53.237305 31979 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.264537 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:53.269102 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:53.256394 32380 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260924 14:31:53.278980 26369 leader_election.cc:334] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.279541 26369 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.235947 32473 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.259493 32204 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:53.286692 26326 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260924 14:31:53.286960 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.256634 32506 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260924 14:31:53.289402 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:53.290596 32458 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:31:53.292167 26369 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.293078 26369 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.293445 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.294148 32458 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260924 14:31:53.295141 26369 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.259146 32461 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:53.296152 26369 leader_election.cc:334] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.296926 26330 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
W20260924 14:31:53.297374 26369 leader_election.cc:334] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.298066 26323 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260924 14:31:53.298286 26369 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.281268 32075 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.301066 32473 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:31:53.303604 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:31:53.304409 26369 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.305109 26369 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.305760 26369 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.306391 26369 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.306747 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.297412 32461 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:53.311126 26406 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
W20260924 14:31:53.312691 26369 leader_election.cc:334] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.313241 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 100896us
I20260924 14:31:53.314919 26404 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260924 14:31:53.313144 26369 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.311831 32473 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260924 14:31:53.281968 32380 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
W20260924 14:31:53.321681 26369 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.324366 26369 leader_election.cc:334] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.325845 26369 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.317786 32458 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:53.333889 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.321054 32380 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:53.343916 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.344509 26257 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.345340 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:53.348490 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:53.346020 26402 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
W20260924 14:31:53.365410 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 76813.1us
W20260924 14:31:53.338816 26295 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 121 similar messages]
I20260924 14:31:53.366269 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:53.328912 32543 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:53.369589 26328 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:31:53.339811 32473 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:31:53.376360 26256 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260924 14:31:53.377625 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.345198 32330 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.380882 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.391834 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.401221 26399 raft_consensus.cc:2379] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 19: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.397135 27123 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.385077 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.398339 27126 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:53.403090 26254 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 13.
W20260924 14:31:53.386397 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47619.9us
I20260924 14:31:53.379416 26256 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260924 14:31:53.409842 32596 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:53.413681 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.415938 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:53.416843 26323 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:53.416437 27123 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:53.408656 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:53.420892 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 54707.6us
I20260924 14:31:53.424640 32598 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:53.425694 32598 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:53.426046 32598 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:53.428251 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 439818us
W20260924 14:31:53.429039 26371 leader_election.cc:334] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.721s (SENT)
I20260924 14:31:53.429602 32506 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:53.405462 32332 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260924 14:31:53.431659 32517 consensus_queue.cc:1046] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:53.433318 32479 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:53.430884 32503 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:53.424142 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.434419 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.434634 26257 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:53.421761 26221 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.435693 26404 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 14.
W20260924 14:31:53.439414 26295 leader_election.cc:334] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.394452 32547 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.442005 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:53.443231 26371 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.385510 32594 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260924 14:31:53.421012 32330 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader election won for term 20
W20260924 14:31:53.446696 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 104107us
I20260924 14:31:53.408381 32543 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
W20260924 14:31:53.445425 26295 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.451171 26295 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.451838 27126 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:53.452239 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:53.455788 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:53.457816 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.461352 32204 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:31:53.462910 32599 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:53.463438 32599 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:53.463832 32599 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:31:53.462954 26075 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 142) took 33855 ms (client timeout 15000 ms). Trace:
I20260924 14:31:53.469635 27125 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:53.388162 26073 catalog_manager.cc:5495] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:53.456880 32458 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:53.491776 32458 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:53.492182 32458 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:53.492514 26073 catalog_manager.cc:5495] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 15, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:53.494550 32547 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.465968 32517 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:53.453274 32600 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:53.455193 32332 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:53.497890 31979 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.500804 26329 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:53.503474 32200 consensus_queue.cc:1046] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:53.452885 32424 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:53.504117 32534 raft_consensus.cc:3032] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Stepping down as leader of term 19
I20260924 14:31:53.504586 32534 raft_consensus.cc:729] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:53.453248 32330 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 20 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:53.505446 32534 consensus_queue.cc:260] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.505900 32330 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.506917 32534 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.507268 32171 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260924 14:31:53.476202 26075 rpcz_store.cc:269] 0924 14:31:19.553112 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:19.553408 (+   296us) service_pool.cc:227] Handling call
0924 14:31:20.342433 (+789025us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b540000ab90. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b63ee851 00007f09b63eed92 00007f09bf33f7c9 00007f09bf2d7c36 00007f09bf2a8c28 00007f09bf3a2c3e 00007f09bd6b4fb4 00007f09bd6b4f42 00007f09bd6b4e71 00007f09bd6b4dec
0924 14:31:53.408084 (+33065651us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:46.493288 (+     0us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b64085fe 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6423155 00007f09bcd5c714
0924 14:31:46.736380 (+243092us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:46.749321 (+ 12941us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:46.749379 (+    58us) write_op.cc:435] Acquired schema lock
0924 14:31:46.749397 (+    18us) tablet.cc:660] Decoding operations
0924 14:31:46.750746 (+  1349us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:46.750903 (+   157us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:46.750960 (+    57us) tablet.cc:683] Acquiring locks for 9 operations
0924 14:31:46.883101 (+132141us) tablet.cc:699] Row locks acquired
0924 14:31:46.896184 (+ 13083us) write_op.cc:260] PREPARE: finished
0924 14:31:46.927647 (+ 31463us) write_op.cc:270] Start()
0924 14:31:46.943597 (+ 15950us) write_op.cc:276] Timestamp: P: 1790260306917637 usec, L: 0
0924 14:31:46.950559 (+  6962us) op_driver.cc:331] REPLICATION: starting
0924 14:31:46.992740 (+ 42181us) log.cc:842] Serialized 3672 byte log entry
0924 14:31:48.333028 (+1340288us) op_driver.cc:444] REPLICATION: finished
0924 14:31:48.711889 (+378861us) write_op.cc:301] APPLY: starting
0924 14:31:48.728270 (+ 16381us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:49.021195 (+292925us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:49.031463 (+ 10268us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:49.666458 (+634995us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:49.666747 (+   289us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0924 14:31:49.666808 (+    61us) write_op.cc:312] APPLY: finished
0924 14:31:49.696872 (+ 30064us) log.cc:842] Serialized 88 byte log entry
0924 14:31:50.762287 (+1065415us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b64085fe 00007f09bf23fa39 00007f09b64258bb 00007f09b6423cbf 00007f09b6425e22 00007f09b8fcf5bb
0924 14:31:51.036672 (+274385us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:51.037195 (+   523us) write_op.cc:454] Released schema lock
0924 14:31:51.037325 (+   130us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:51.037802 (+   477us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:51.037845 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":119640,"reported_tablets":43,"spinlock_wait_cycles":5177088,"child_traces":[["op",{"apply.queue_time_us":334072,"apply.run_cpu_time_us":87345,"apply.run_wall_time_us":2389071,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41783,"num_ops":9,"prepare.queue_time_us":339734,"prepare.run_cpu_time_us":22949,"prepare.run_wall_time_us":672295,"raft.queue_time_us":621451,"raft.run_cpu_time_us":15180,"raft.run_wall_time_us":358471,"replication_time_us":1308737,"spinlock_wait_cycles":93513472,"thread_start_us":2369379,"threads_started":5,"wal-append.queue_time_us":1773188,"wal-append.run_cpu_time_us":72539,"wal-append.run_wall_time_us":2584810}]]}
W20260924 14:31:53.523561 26295 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.501619 32326 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.533610 26324 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:53.534732 32200 raft_consensus.cc:3032] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:53.535043 32326 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.535256 32200 raft_consensus.cc:729] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:53.536163 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 66629.8us
I20260924 14:31:53.536558 32596 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:53.556915 27125 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:53.559993 32517 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:53.560177 32200 consensus_queue.cc:260] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.560863 26323 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260924 14:31:53.561352 32200 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:53.524962 32558 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.494880 26073 catalog_manager.cc:5495] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:53.502162 32380 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.565840 26073 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:53.566967 32380 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.543433 31979 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.567725 26249 raft_consensus.cc:1226] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:53.558961 32332 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.569968 32171 raft_consensus.cc:3037] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:53.570196 32332 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.572930 32332 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:53.582193 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.584445 26402 raft_consensus.cc:2396] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260924 14:31:53.559083 26369 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.565071 32558 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:53.551347 32598 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.607547 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:53.600512 26369 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.610527 26369 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:53.611014 26323 raft_consensus.cc:2379] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 22: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:53.583560 32597 consensus_queue.cc:1046] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260924 14:31:53.616779 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 168019us
I20260924 14:31:53.625013 26249 raft_consensus.cc:1226] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260924 14:31:53.611210 26369 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:53.633561 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 187477us
I20260924 14:31:53.634689 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:53.640317 26403 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:53.640399 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:53.641964 26224 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:53.643422 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.644357 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.648468 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.649479 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.645613 26257 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:53.663857 26255 raft_consensus.cc:2379] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.667052 26258 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
W20260924 14:31:53.676082 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61397.8us
I20260924 14:31:53.677666 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.677914 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.677603 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.680976 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.681713 26401 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Advancing to term 22
I20260924 14:31:53.681087 26406 raft_consensus.cc:1261] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:31:53.685431 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.686390 26397 raft_consensus.cc:2354] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16.
I20260924 14:31:53.687307 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.688479 32326 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260924 14:31:53.689247 26371 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.621s (SENT)
I20260924 14:31:53.689718 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.688537 32442 consensus_queue.cc:1035] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260924 14:31:53.691789 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.688457 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.689714 32326 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:53.695763 26401 raft_consensus.cc:2454] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 22.
I20260924 14:31:53.696202 32326 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:53.698279 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.679527 26257 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:53.699301 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.700091 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.700976 26258 raft_consensus.cc:2396] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260924 14:31:53.700560 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:53.698330 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.700452 26251 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:53.701704 26252 raft_consensus.cc:2454] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:53.705188 26254 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 20: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.706142 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.707093 26250 raft_consensus.cc:2454] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:31:53.639798 32380 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:53.716629 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.717913 26399 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260924 14:31:53.720041 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.720325 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.723839 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.723839 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.724848 26253 raft_consensus.cc:2379] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 11: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.725431 26257 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:53.721335 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.727610 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.728334 26255 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:53.734812 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:53.735033 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45815.5us
I20260924 14:31:53.735249 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.735515 26254 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:53.636185 32543 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260924 14:31:53.741254 26252 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:53.741336 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.637768 32458 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.745267 26257 raft_consensus.cc:2354] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20.
I20260924 14:31:53.747388 26298 leader_election.cc:398] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16.
I20260924 14:31:53.747779 26298 leader_election.cc:401] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260924 14:31:53.721480 32330 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:53.725014 32380 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:53.734653 32595 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:53.728421 32473 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:53.732586 32558 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:53.640242 32600 consensus_queue.cc:1046] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260924 14:31:53.724385 32607 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:53.587644 26073 catalog_manager.cc:5495] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:53.758195 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.758649 26404 raft_consensus.cc:1226] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260924 14:31:53.763352 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.697794 32326 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:53.761281 32558 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:53.772840 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.773054 32598 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:53.757447 32599 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:53.757329 32380 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:53.756721 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:53.797781 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.752995 32479 raft_consensus.cc:3037] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:31:53.778129 26254 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:53.790449 26249 raft_consensus.cc:1261] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260924 14:31:53.762208 32442 raft_consensus.cc:3037] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:53.788552 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.801044 26249 pending_rounds.cc:76] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Aborting all ops after (but not including) 10
I20260924 14:31:53.799816 26298 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.804158 26249 pending_rounds.cc:97] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260924 14:31:53.804433 26252 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 20: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.804494 26249 raft_consensus.cc:2866] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:53.806677 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.807497 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.808220 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.808871 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.809516 26252 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:53.809433 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.810662 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.812013 32599 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:53.774987 32607 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:53.823885 26297 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.630s (SENT)
I20260924 14:31:53.824342 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.817499 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.790762 32461 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:53.827091 26254 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:53.826313 26257 raft_consensus.cc:2454] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:53.791849 32558 raft_consensus.cc:3037] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:53.835966 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:53.832741 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.835296 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.838572 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.843014 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.844211 26258 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260924 14:31:53.838927 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:53.839483 26371 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 1.512s (SENT)
I20260924 14:31:53.845952 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:53.845117 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.820689 32326 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:53.761610 32330 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.853302 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:53.856040 26257 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260924 14:31:53.761901 32595 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:53.865175 26254 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:53.871145 32171 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:53.854852 31979 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:53.838070 32440 raft_consensus.cc:2781] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader election won for term 22
I20260924 14:31:53.853649 32473 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:53.875080 32595 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:53.875824 26073 catalog_manager.cc:5495] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 18 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:53.877808 26073 catalog_manager.cc:5495] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 16, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:31:53.880033 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:53.884178 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.829090 32607 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:53.886901 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.894356 26252 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.894146 32380 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:53.902865 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:53.906098 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 106506us
I20260924 14:31:53.929477 26298 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:53.970011 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 45815.5us
I20260924 14:31:53.971771 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:53.972496 26402 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:53.973353 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:53.974222 26257 raft_consensus.cc:2396] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260924 14:31:53.986474 23135 thread.cc:536] Waited for 1000ms trying to join with heartbeat (tid 27223)
W20260924 14:31:54.015793 26297 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Timed out: RequestConsensusVote RPC to 127.22.151.194:46433 timed out after 2.085s (SENT)
W20260924 14:31:54.022351 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 50439.7us
I20260924 14:31:54.023753 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.024580 26252 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 20: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:53.902251 32440 raft_consensus.cc:686] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:54.030866 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.039733 26371 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.040647 26332 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:54.043083 32479 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:31:54.044100 32440 consensus_queue.cc:237] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.017827 32461 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:54.047586 32458 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:54.052194 26403 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:54.055691 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.056483 26257 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 15
I20260924 14:31:54.057449 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.065179 26257 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
W20260924 14:31:54.075902 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 275995us
I20260924 14:31:54.018571 32573 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260924 14:31:54.091334 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.115940 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.116513 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.097544 26329 raft_consensus.cc:1226] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:54.120867 26327 raft_consensus.cc:2379] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 20: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:54.067766 32632 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:54.126499 26252 raft_consensus.cc:1226] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:54.127324 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.128177 26328 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 18: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:54.047339 32593 consensus_queue.cc:1035] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260924 14:31:54.120016 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.158991 32380 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:54.096016 32600 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:54.167794 26324 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Rejecting Update request from peer 6c22052275034c369b7f7dca086fbd1e for earlier term 13. Current term is 14. Ops: []
I20260924 14:31:54.069476 32624 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.063s
I20260924 14:31:54.172185 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:54.171547 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.173079 26402 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:54.179131 32595 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:54.179711 32595 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:54.180099 32595 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:54.164985 32380 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.181811 32380 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.183933 32596 raft_consensus.cc:3032] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:54.184211 32380 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.184384 32596 raft_consensus.cc:729] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:54.185355 32596 consensus_queue.cc:260] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.186592 32596 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:54.189162 26404 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260924 14:31:54.191860 32592 raft_consensus.cc:3032] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:54.192291 32592 raft_consensus.cc:729] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:54.161191 32424 raft_consensus.cc:482] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.131165 32632 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:54.173720 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 116368us
I20260924 14:31:54.191509 32595 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.199245 26371 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:54.193886 26369 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.204763 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.213943 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.214851 26404 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:31:54.224597 26224 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
W20260924 14:31:54.225675 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 182507us
I20260924 14:31:54.224586 27125 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:54.174618 32624 consensus_queue.cc:1225] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer 45ab97b763804767846e9d27a01b561f log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260924 14:31:54.228981 32625 consensus_queue.cc:1046] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:54.233176 32623 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.236176 26326 raft_consensus.cc:1226] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:54.019680 26073 catalog_manager.cc:5495] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 14, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 14 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.238217 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.211050 26402 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 15.
I20260924 14:31:54.193845 32631 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.216463 32458 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:54.264175 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.264050 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.218447 32600 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:54.202100 32632 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:54.222932 32204 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.193066 32592 consensus_queue.cc:260] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.264705 32479 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:54.263389 32380 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260924 14:31:54.300925 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62895.2us
I20260924 14:31:54.279109 32330 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.214787 32424 raft_consensus.cc:504] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.281639 32461 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:54.311053 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.315575 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.283273 32204 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.318457 32204 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.297561 32458 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:54.274173 32473 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260924 14:31:54.309149 32380 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:54.278172 32645 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260924 14:31:54.266738 32623 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:54.347661 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 120659us
W20260924 14:31:54.359153 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 131626us
I20260924 14:31:54.340209 32592 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:54.353127 26402 pending_rounds.cc:76] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 8
I20260924 14:31:54.364092 26402 pending_rounds.cc:97] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260924 14:31:54.364445 26402 raft_consensus.cc:2866] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:54.349933 32380 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:54.273638 32631 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:54.370672 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 55169.8us
W20260924 14:31:54.370051 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 105966us
W20260924 14:31:54.372233 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 136952us
I20260924 14:31:54.348644 32597 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260924 14:31:54.381369 32633 consensus_queue.cc:1046] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:54.381928 26254 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260924 14:31:54.386009 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.386622 26251 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 11.
I20260924 14:31:54.366317 31979 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:54.393430 32326 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260924 14:31:54.394192 32623 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.394088 32632 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.395067 32643 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260924 14:31:54.395568 32442 consensus_meta.cc:287] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260924 14:31:54.383929 26371 leader_election.cc:398] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20.
I20260924 14:31:54.407768 26371 leader_election.cc:401] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:54.410018 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.401927 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.412864 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:54.427006 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48948.3us
I20260924 14:31:54.428336 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.431097 26399 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 10.
I20260924 14:31:54.433148 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:54.437664 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 273329us
I20260924 14:31:54.347289 32642 consensus_queue.cc:1046] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260924 14:31:54.413568 32473 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20.
I20260924 14:31:54.465081 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.430202 32645 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:54.424355 32610 raft_consensus.cc:3032] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 19 LEADER]: Stepping down as leader of term 19
I20260924 14:31:54.436057 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.468123 26371 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.470954 32591 consensus_queue.cc:1035] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:54.468537 32646 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260924 14:31:54.469278 32650 log.cc:924] Time spent T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260924 14:31:54.468730 32607 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.468585 32649 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:54.470508 32648 log.cc:924] Time spent T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260924 14:31:54.424078 32633 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:54.473215 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:54.474982 32517 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260924 14:31:54.475806 32604 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Stepping down as leader of term 13
I20260924 14:31:54.477244 32506 raft_consensus.cc:3032] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:54.479218 32330 raft_consensus.cc:3032] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:54.436030 32651 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:54.497360 32631 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.417826 32442 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 for earlier term 15. Current term is 16.
W20260924 14:31:54.505689 26221 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.507296 32610 raft_consensus.cc:729] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:54.414682 26073 catalog_manager.cc:5495] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 16, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:54.522476 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.575793 26404 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:54.520208 32646 raft_consensus.cc:482] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:54.533653 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.522336 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.579659 32646 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:54.574620 26073 catalog_manager.cc:5495] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:54.437074 32632 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:54.514333 32654 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:31:54.536988 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 63846.4us
I20260924 14:31:54.539774 32649 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.536485 32506 raft_consensus.cc:729] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:54.584721 32506 consensus_queue.cc:260] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.586040 32506 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:54.530495 31979 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:54.587718 31979 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.589792 31979 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:54.536177 32604 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:54.543613 32461 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:54.544301 32652 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260924 14:31:54.544992 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 66604.5us
I20260924 14:31:54.591943 32652 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:54.592350 32652 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:54.594515 26406 raft_consensus.cc:1261] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260924 14:31:54.534139 32424 leader_election.cc:288] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.596886 32649 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.590837 32604 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.597996 32604 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:54.600555 32461 consensus_queue.cc:1035] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:54.601225 26403 raft_consensus.cc:1226] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 14. Current term is 15. Ops: []
W20260924 14:31:54.538471 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 102523us
I20260924 14:31:54.547269 32171 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.549075 32619 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.513151 32479 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:54.549551 32558 consensus_meta.cc:287] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.713s	user 0.012s	sys 0.000s
W20260924 14:31:54.550148 26295 leader_election.cc:334] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.549885 32651 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.539244 32330 raft_consensus.cc:729] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:54.551386 32458 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:54.557090 32607 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:54.568140 26221 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.422348 32595 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:54.583055 32610 consensus_queue.cc:260] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.607082 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.546638 32380 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.581461 26073 catalog_manager.cc:5495] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:54.610445 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.615352 32461 consensus_queue.cc:1046] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260924 14:31:54.614059 26332 raft_consensus.cc:2454] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
W20260924 14:31:54.615123 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 65017.3us
W20260924 14:31:54.639631 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:54.615622 32619 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:54.619509 26404 raft_consensus.cc:2454] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:54.619976 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:54.626719 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 121070us
I20260924 14:31:54.644484 26298 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:54.617211 32458 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:54.625101 32330 consensus_queue.cc:260] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.621637 32442 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:31:54.616338 32558 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:54.648048 32461 raft_consensus.cc:3032] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Stepping down as leader of term 14
I20260924 14:31:54.648375 32621 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:54.649366 31979 raft_consensus.cc:2781] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:54.649240 32621 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.649921 31979 raft_consensus.cc:686] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:54.650929 31979 consensus_queue.cc:237] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.647168 32603 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:31:54.651593 26256 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:54.626346 32651 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.651278 32621 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.652635 32619 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:54.653322 32461 raft_consensus.cc:729] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:54.654270 32442 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:54.616207 32171 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.654253 32461 consensus_queue.cc:260] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:54.643213 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:54.647060 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:54.656028 26221 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:54.656965 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:54.657050 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.657182 32595 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260924 14:31:54.657537 26221 leader_election.cc:334] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.659068 32442 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:54.657176 32461 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:54.659215 26221 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.669698 32458 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.663594 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.670831 32458 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:54.649278 32380 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.671528 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:54.672633 32171 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.673436 32424 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:54.674584 26221 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.674989 26221 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.676828 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.677711 26405 raft_consensus.cc:2379] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e09fbebe29a49d291d23f9d395fe335 in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:54.667657 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.678238 32442 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:54.678140 32652 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.681317 32610 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:31:54.681607 32424 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:54.664379 32558 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:54.663198 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.677515 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.653941 32479 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:54.684161 26398 raft_consensus.cc:2454] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:54.685372 26254 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:54.685596 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.692400 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.694783 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.695319 26403 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:54.698273 27125 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:54.701092 26221 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:54.702020 26221 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.702346 26326 raft_consensus.cc:1226] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 19. Current term is 20. Ops: []
I20260924 14:31:54.704421 32619 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:31:54.704869 32619 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:54.705186 32619 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:54.699007 32664 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:54.708549 32645 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:54.708359 32664 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.709342 26401 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:54.702361 26221 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:54.715476 26295 leader_election.cc:334] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:54.719740 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:31:54.720427 26295 leader_election.cc:334] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.696149 32599 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260924 14:31:54.729857 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 49576.7us
I20260924 14:31:54.732725 32664 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.731783 26073 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.736236 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.735857 26073 catalog_manager.cc:5495] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 19. New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.736650 32646 raft_consensus.cc:504] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.737866 26399 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260924 14:31:54.746079 32442 consensus_queue.cc:1046] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260924 14:31:54.749461 32662 log.cc:924] Time spent T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:31:54.749770 32330 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:54.750597 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48599.9us
W20260924 14:31:54.777557 26221 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.751852 32594 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:54.755946 26224 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:54.757851 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.737315 26073 catalog_manager.cc:5495] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:54.738996 32646 leader_election.cc:288] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:54.779156 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.802110 26256 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:31:54.785032 32380 raft_consensus.cc:2781] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:31:54.803952 32380 raft_consensus.cc:686] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:54.804858 32380 consensus_queue.cc:237] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:31:54.789345 26221 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.790123 32619 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:54.812520 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.813305 32594 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:54.813834 32619 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:54.793015 32595 raft_consensus.cc:3032] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Stepping down as leader of term 16
I20260924 14:31:54.814487 32595 raft_consensus.cc:729] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:31:54.814726 26221 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:54.815346 32595 consensus_queue.cc:260] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.816612 32595 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:54.818820 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.819525 26250 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:54.821153 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.821738 26401 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:54.823319 26256 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:54.825682 27125 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:54.843885 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.844589 26255 raft_consensus.cc:2362] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18. Re-sending same reply.
I20260924 14:31:54.829777 26403 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:54.844467 26073 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.845719 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.846107 26073 catalog_manager.cc:5495] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.846988 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.848728 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.848024 26073 catalog_manager.cc:5495] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.850899 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.851213 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.852545 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.852011 26073 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 15, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 15 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:54.853376 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.854029 32652 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 12 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.854732 26073 catalog_manager.cc:5495] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:54.855611 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:54.857060 26402 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Advancing to term 12
I20260924 14:31:54.857745 26073 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 23, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 23 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.842310 32619 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:31:54.860805 26251 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17. Ops: []
I20260924 14:31:54.870931 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.871344 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.825157 32517 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.862787 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.863402 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:54.872936 26255 raft_consensus.cc:3037] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:54.865185 26401 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 21.
I20260924 14:31:54.867933 26224 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:54.869542 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf849a0475754232bd6bf5cd41d3821c" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.871552 26330 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:54.875309 32380 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:54.875823 32380 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 19 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:54.875962 27126 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:54.876791 32380 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.877811 26326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.883415 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.867199 26073 catalog_manager.cc:5495] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.885627 26332 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 20: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:54.867178 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:54.889137 26330 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
I20260924 14:31:54.885789 26256 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:54.888315 26255 raft_consensus.cc:2454] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
I20260924 14:31:54.890631 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9301429ace424dc58cbf9b57301e674c" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:54.890901 26332 raft_consensus.cc:1261] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:54.892736 26297 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:54.892976 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.893143 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.894752 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:54.895356 26328 raft_consensus.cc:2362] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19. Re-sending same reply.
I20260924 14:31:54.895488 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.895731 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.896195 26256 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:31:54.900027 32171 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:31:54.900137 32631 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:31:54.900791 32462 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:54.901132 32619 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:54.904114 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:54.910569 26402 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 12.
I20260924 14:31:54.913348 27123 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.903041 31979 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:54.885300 26073 catalog_manager.cc:5495] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 5 to 9, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 9 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.911046 32610 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:54.928387 32610 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:54.912838 32171 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:54.929279 32610 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.929155 26073 catalog_manager.cc:5495] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 15, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.930015 32171 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:54.911482 32631 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:54.931175 26073 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.931850 32631 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:54.933063 26073 catalog_manager.cc:5495] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 7 to 10, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 10 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:54.895582 32517 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.935711 32652 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:54.910924 32646 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:54.926832 26327 raft_consensus.cc:2396] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260924 14:31:54.938149 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.939049 26257 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:54.902213 32619 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:54.942961 26406 raft_consensus.cc:1261] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260924 14:31:54.944031 26251 raft_consensus.cc:1226] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260924 14:31:54.944396 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.945603 27123 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.945694 32618 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:54.946341 32649 raft_consensus.cc:3037] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:54.947079 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:54.945744 32631 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:54.952566 27123 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.957974 27125 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:54.958801 32479 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:54.959704 27123 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.956802 26329 raft_consensus.cc:2454] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:31:54.963145 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:54.963882 26257 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
W20260924 14:31:54.987234 23135 thread.cc:536] Waited for 2000ms trying to join with heartbeat (tid 27223)
I20260924 14:31:54.965276 26221 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.007109 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.031788 26328 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 20.
I20260924 14:31:54.960000 32517 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.012950 26369 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:54.962633 32664 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:31:55.036963 32664 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:55.037388 32664 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:31:55.032212 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 67028.1us
I20260924 14:31:55.034266 26250 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 21.
I20260924 14:31:55.035526 32604 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:55.036226 32646 raft_consensus.cc:2781] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:55.050065 32646 raft_consensus.cc:686] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:55.046260 26221 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:54.934752 26073 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:55.048869 32380 consensus_queue.cc:1046] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:54.963866 32599 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:55.078955 32380 raft_consensus.cc:3032] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:55.079427 32380 raft_consensus.cc:729] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:31:55.085873 32440 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:55.086477 32440 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:54.963640 32594 consensus_queue.cc:1035] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
I20260924 14:31:55.051198 32646 consensus_queue.cc:237] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.076819 32604 raft_consensus.cc:2726] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:31:55.104249 32479 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.104961 32479 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:55.107131 32479 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.111716 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.112206 32184 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.112947 32184 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:55.115190 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.122694 26221 leader_election.cc:302] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.129570 26073 catalog_manager.cc:5495] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 15, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:55.132844 32440 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:55.136029 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.140833 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
I20260924 14:31:55.080255 32380 consensus_queue.cc:260] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.146627 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:31:55.147820 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.151024 26371 leader_election.cc:302] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.134832 32649 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:55.137511 32442 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.154214 26371 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:55.156348 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:55.165555 26297 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:55.145669 32517 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:31:55.171545 26295 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.172227 32461 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:31:55.156543 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.144421 32594 raft_consensus.cc:2726] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:55.164992 32458 raft_consensus.cc:2781] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:55.156720 32632 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:31:55.176971 32461 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:55.184499 32458 raft_consensus.cc:686] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:55.192364 32461 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:55.193109 32458 consensus_queue.cc:237] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.182960 32632 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:55.172463 32442 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.200160 32632 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:31:55.200575 32442 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.178537 32595 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:55.147754 32424 consensus_queue.cc:1035] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260924 14:31:55.153044 26221 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.176889 32184 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.175045 32649 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:55.204754 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 51761.4us
W20260924 14:31:55.207325 26221 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.207767 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:55.208940 26221 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:55.209628 26221 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.209941 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.180416 32610 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:55.214080 26221 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.223896 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:55.225528 26221 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.225996 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:55.227056 26221 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.227851 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:55.228842 26221 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:55.229935 26221 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.230463 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:55.219758 32607 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:55.229218 32603 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:31:55.225198 32594 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:55.243929 32424 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260924 14:31:55.214090 32604 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:55.231066 32558 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:31:55.210953 32649 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:55.261451 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:55.268497 32649 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.274878 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.276309 26257 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:55.281646 27125 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:55.275386 32380 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:55.286655 32618 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:55.288571 32618 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:55.289088 32618 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:55.305819 26406 raft_consensus.cc:1261] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:55.307715 32604 consensus_queue.cc:1035] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:55.319000 26257 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:55.322315 26399 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:55.352370 32607 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:55.353330 32607 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.358204 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.361224 32607 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.368410 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.341753 32603 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:55.325417 32424 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260924 14:31:55.387624 26401 raft_consensus.cc:1261] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:31:55.341113 32649 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260924 14:31:55.373601 32594 raft_consensus.cc:482] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:55.405479 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 231103us
I20260924 14:31:55.407941 26297 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:55.411211 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 181287us
W20260924 14:31:55.412132 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:55.413621 32640 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:55.414105 32640 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:55.414389 32640 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:55.395310 32603 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:55.417460 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:31:55.422094 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:31:55.422730 26221 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:55.423730 26221 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.424180 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.416030 32594 raft_consensus.cc:504] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:55.425490 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.425940 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.426505 32594 leader_election.cc:288] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.426784 32604 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:55.428831 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.429705 26255 raft_consensus.cc:2379] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:55.407039 32442 consensus_queue.cc:1035] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260924 14:31:55.432215 26399 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260924 14:31:55.433393 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.427205 32594 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260924 14:31:55.439014 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:55.446810 26221 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.447379 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:55.518014 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 71238.3us
W20260924 14:31:55.518863 26221 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.519332 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:55.520622 26221 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:55.415979 32461 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:55.579483 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 129781us
I20260924 14:31:55.579851 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.580762 26329 raft_consensus.cc:2379] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:55.568929 32424 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260924 14:31:55.586117 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 150727us
I20260924 14:31:55.586671 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad9bf5791f146a4b74218b2dc7e458f" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.600136 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.600481 32603 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.602137 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:55.602627 32664 consensus_meta.cc:287] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Time spent flushing consensus metadata: real 0.565s	user 0.013s	sys 0.000s
I20260924 14:31:55.602732 26323 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:55.617352 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.622725 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.631001 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.633857 26323 raft_consensus.cc:1261] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:55.568621 32685 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:31:55.652143 26221 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.688263 26332 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260924 14:31:55.688578 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:55.696309 26221 leader_election.cc:302] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.637652 32597 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.603359 32664 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:55.637120 32604 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:55.617553 32632 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.641279 32506 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:55.707581 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 83445.6us
I20260924 14:31:55.644290 32649 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:55.698764 32424 raft_consensus.cc:3037] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:55.698474 31979 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:55.724721 32693 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.723058 32688 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.698204 32597 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:55.728371 32594 raft_consensus.cc:2726] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:55.715112 32604 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.760257 32654 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.779484 32597 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:55.754226 32646 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260924 14:31:55.756937 32693 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:55.790264 32693 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.791630 32664 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.766957 32688 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:55.794924 32688 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.754866 32506 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:55.808259 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.809128 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:55.809746 26323 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:31:55.797842 32380 consensus_meta.cc:287] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.000s
I20260924 14:31:55.816845 26257 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:55.772771 32618 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:55.826697 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 130496us
I20260924 14:31:55.827968 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:55.781608 32654 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:55.824811 32632 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:55.850953 32380 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:55.857017 26255 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260924 14:31:55.861364 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:55.862108 26250 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260924 14:31:55.871730 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 84365.9us
I20260924 14:31:55.868019 26256 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 14.
I20260924 14:31:55.873979 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.876412 26371 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:55.877722 26251 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:55.874912 26249 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
W20260924 14:31:55.875978 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 154818us
I20260924 14:31:55.878875 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.888934 26371 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:55.887415 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.894320 26332 raft_consensus.cc:1261] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260924 14:31:55.860543 32685 consensus_queue.cc:1035] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260924 14:31:55.909825 26332 pending_rounds.cc:76] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 8
I20260924 14:31:55.917415 26332 pending_rounds.cc:97] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260924 14:31:55.917172 26323 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 21.
I20260924 14:31:55.917786 26332 raft_consensus.cc:2866] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:55.920035 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:55.920816 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:55.921419 26327 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:55.920876 26399 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:31:55.921715 26323 raft_consensus.cc:2354] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15.
I20260924 14:31:55.925454 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.927313 26402 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:55.933768 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:55.934121 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.930423 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24655c8fcef4b9882a0ab16bfc5bbb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:55.939824 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:55.943073 26256 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:55.880282 32632 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260924 14:31:55.946979 27123 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:55.916605 32604 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.896704 32696 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:31:55.928689 32688 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:55.954422 26404 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 16.
I20260924 14:31:55.893739 32646 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
I20260924 14:31:55.958781 32708 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:31:55.944062 32631 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:55.947676 32506 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:55.948218 32707 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:55.955947 32688 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:55.932446 32664 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260924 14:31:55.970598 32631 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:55.975343 26256 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 21.
I20260924 14:31:56.008317 26297 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:56.024519 23135 thread.cc:536] Waited for 3000ms trying to join with heartbeat (tid 27223)
I20260924 14:31:56.051307 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.121304 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:56.002592 32640 consensus_meta.cc:287] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.588s	user 0.006s	sys 0.006s
W20260924 14:31:56.122833 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 166579us
I20260924 14:31:56.127291 26400 raft_consensus.cc:2379] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 18: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
W20260924 14:31:56.046890 32689 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.236s	user 0.003s	sys 0.000s
W20260924 14:31:56.029790 32699 log.cc:924] Time spent T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260924 14:31:56.150568 32461 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:56.151305 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:31:56.047317 32618 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.053555 32696 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:56.157390 32696 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.023931 31979 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:56.166304 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:56.167109 26256 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:55.978286 32688 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:55.993253 32654 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:56.004822 32631 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:55.990790 32424 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:31:56.184140 26256 raft_consensus.cc:2454] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:31:56.123306 32708 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:56.192495 26297 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:56.128283 32683 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:56.195161 26255 raft_consensus.cc:1261] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260924 14:31:56.196897 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1455727abfd74a9aa50ba77fa9c79771" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.123171 32640 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:56.201399 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:31:55.979254 32707 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:56.217495 32641 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:56.218405 32641 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.218883 32707 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.199915 32631 raft_consensus.cc:2781] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:31:56.228034 32631 raft_consensus.cc:686] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:56.229022 32631 consensus_queue.cc:237] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.229763 32717 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260924 14:31:56.231738 32708 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.232519 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.233974 32506 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:56.236734 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:56.202205 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 260259us
I20260924 14:31:56.237874 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.198554 32683 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:56.238575 26403 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:56.197906 32709 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:56.241732 26224 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:56.239066 26256 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:56.246320 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:56.247076 26330 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 18. Current term is 19. Ops: []
I20260924 14:31:56.255236 32703 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:56.255733 32703 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:56.256420 32703 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:56.260576 32716 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.261127 32716 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.267110 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.268178 26403 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:56.272418 26369 leader_election.cc:398] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15.
I20260924 14:31:56.238996 32641 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.243902 32683 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:56.349036 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 75119us
W20260924 14:31:56.349565 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 97116.7us
I20260924 14:31:56.263266 32716 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.248196 32697 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:56.357316 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 82261.5us
I20260924 14:31:56.350931 31979 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:56.244661 32695 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.275694 32707 raft_consensus.cc:482] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:56.379786 32707 raft_consensus.cc:504] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.379673 32695 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.382897 32707 leader_election.cc:288] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.368894 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:56.384263 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.385005 26256 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:31:56.386080 26371 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:56.274561 32610 consensus_queue.cc:1035] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
W20260924 14:31:56.378376 26295 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:56.392728 32664 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:31:56.378360 32697 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.385442 32695 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.367522 32595 raft_consensus.cc:482] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.402004 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:56.425705 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 47375us
I20260924 14:31:56.378367 31979 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:56.438158 31979 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:56.439333 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:56.440776 32705 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:31:56.441174 32705 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:56.441506 32705 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:56.428588 32595 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:56.434271 32720 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260924 14:31:56.428992 32695 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:31:56.429531 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:56.270478 32709 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.402807 32707 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:56.454908 27125 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:56.457029 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:31:56.457289 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 69229.2us
W20260924 14:31:56.463485 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 89692.6us
I20260924 14:31:56.475409 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:56.476143 26327 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:31:56.455385 32695 raft_consensus.cc:482] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:56.461448 32618 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:56.470760 32707 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:56.498962 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.498483 31979 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.482385 32640 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.485253 32424 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:56.494980 32695 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:31:56.552073 32727 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260924 14:31:56.552963 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.553646 26257 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:31:56.554162 32595 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.495426 32733 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:31:56.542054 32723 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260924 14:31:56.544888 32732 consensus_queue.cc:1046] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:56.523262 32506 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.548408 32709 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.549070 32424 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:56.581574 32732 consensus_queue.cc:1225] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260924 14:31:56.557535 26371 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:56.581741 32506 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:56.582489 32594 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.560760 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:56.526790 32715 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
W20260924 14:31:56.583951 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 83230us
I20260924 14:31:56.590406 26298 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:56.590606 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:56.591523 32461 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:31:56.598536 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.583055 32594 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.589936 32709 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:31:56.592900 32721 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.592628 32558 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:56.609613 32558 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:56.610100 32697 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:56.607936 32709 raft_consensus.cc:482] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:56.611814 32709 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:56.611872 32558 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:56.613161 32558 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:56.613631 32558 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.616178 32705 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:56.621819 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:56.622450 26328 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:56.638101 26249 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:56.642524 26256 raft_consensus.cc:2379] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
W20260924 14:31:56.637465 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 133476us
I20260924 14:31:56.647897 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:56.630605 32708 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:56.648612 26330 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:56.650508 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.651467 26254 raft_consensus.cc:2379] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:56.651137 32688 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.650203 32708 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:56.658430 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:56.653153 26402 raft_consensus.cc:2379] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:31:56.653738 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.663079 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.668094 26328 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:56.673501 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:56.685748 26398 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:56.689797 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:56.638482 31979 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.651563 32705 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:56.715277 26221 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:56.719054 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:56.672079 32646 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260924 14:31:56.732823 32558 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:56.734668 32728 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.255s	user 0.000s	sys 0.000s
I20260924 14:31:56.628173 32721 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:56.739526 32703 raft_consensus.cc:504] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.740701 26257 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:31:56.741390 32721 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:56.741781 32703 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.728077 32683 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.750743 32683 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:56.752570 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 256173us
I20260924 14:31:56.755119 32700 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:56.755796 32700 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:56.760646 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.761309 26249 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:56.761489 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:56.762107 26251 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:56.762976 32708 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.763856 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d85003b754a6e95eaf1d19022c217" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.764443 26329 raft_consensus.cc:2354] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:56.764616 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:56.769264 26258 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
W20260924 14:31:56.775944 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 93478.5us
I20260924 14:31:56.777068 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:56.777963 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 106800us
I20260924 14:31:56.779027 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:56.798609 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 129427us
I20260924 14:31:56.795897 32506 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:56.803598 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.763906 32424 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:56.805119 26398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.811553 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.813093 26401 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:56.808707 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.737278 32695 raft_consensus.cc:504] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.814670 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:56.819123 26405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.819394 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:56.826263 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:56.843606 27125 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
W20260924 14:31:56.847488 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 165743us
I20260924 14:31:56.849044 26401 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:56.815424 32687 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:56.815411 32746 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:31:56.793553 32709 raft_consensus.cc:504] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.845618 32597 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:56.827240 32688 leader_election.cc:288] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:56.919793 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 71415.6us
I20260924 14:31:56.869244 32594 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:56.841490 32700 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:56.873373 32479 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:56.959980 26404 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:56.969230 32479 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:56.975071 32458 raft_consensus.cc:3032] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Stepping down as leader of term 18
I20260924 14:31:56.975785 32458 raft_consensus.cc:729] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:56.976771 32458 consensus_queue.cc:260] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:56.978118 32458 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:31:57.046485 23135 thread.cc:536] Waited for 4000ms trying to join with heartbeat (tid 27223)
I20260924 14:31:56.941872 32597 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:56.969755 32479 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:56.937534 32687 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:57.072459 26404 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
W20260924 14:31:57.063645 32720 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260924 14:31:57.067045 32597 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
W20260924 14:31:57.090034 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 217547us
I20260924 14:31:57.112568 32709 leader_election.cc:288] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:31:57.118736 32739 log.cc:924] Time spent T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
W20260924 14:31:57.120994 32750 log.cc:924] Time spent T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260924 14:31:57.129108 26251 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 19.
I20260924 14:31:57.128626 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:57.140936 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 190428us
I20260924 14:31:57.142063 27123 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:31:57.143348 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 325485us
I20260924 14:31:57.170691 27123 leader_election.cc:302] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.172655 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:57.173533 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.175467 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:57.176115 26328 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:57.174396 26403 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:31:57.179519 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:57.145179 32517 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:57.177907 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 232478us
I20260924 14:31:57.185777 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:57.190778 26325 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:57.182816 26324 raft_consensus.cc:1226] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260924 14:31:57.169684 32695 leader_election.cc:288] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.199357 26328 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 21.
I20260924 14:31:57.206830 26369 leader_election.cc:302] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:57.177927 32709 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.208335 32707 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
W20260924 14:31:57.214466 32742 log.cc:924] Time spent T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260924 14:31:57.215617 32748 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:31:57.216078 32748 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:57.216310 32748 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:57.217139 32597 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:57.219992 32725 log.cc:924] Time spent T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: Append to log took a long time: real 0.618s	user 0.003s	sys 0.004s
I20260924 14:31:57.224124 26325 raft_consensus.cc:2454] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 17.
W20260924 14:31:57.227803 32758 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260924 14:31:57.219249 32597 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.238246 32709 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:57.273789 26298 leader_election.cc:302] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:57.275120 26369 leader_election.cc:398] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:57.274461 27125 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:57.141242 32687 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.276269 26369 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:57.199095 32517 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:57.293977   302 raft_consensus.cc:2726] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:57.295749 32709 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:57.299817 32731 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.301247   300 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.301462 32330 raft_consensus.cc:482] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:31:57.301779 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 62572.3us
W20260924 14:31:57.312856 32766 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260924 14:31:57.313840 26254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.314455 26254 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:57.316910 32517 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:57.319550 32731 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.320600 32330 raft_consensus.cc:504] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.321455 32697 raft_consensus.cc:2781] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:57.321903 32731 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:57.322144 32697 raft_consensus.cc:686] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:57.283608 32707 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:57.305356 26332 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: []
I20260924 14:31:57.323215 32697 consensus_queue.cc:237] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.292639 32184 raft_consensus.cc:482] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:57.322780 32330 leader_election.cc:288] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.235940 32695 raft_consensus.cc:2781] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:31:57.288424 32607 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:57.335602 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.308645 32711 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:57.311239 32632 raft_consensus.cc:2781] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:57.345125 27126 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335; no voters: f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.348408 26254 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:57.345974 32607 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:57.351648 32764 consensus_queue.cc:1046] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260924 14:31:57.352602 32764 consensus_queue.cc:1225] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260924 14:31:57.353551 32607 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:57.355942 31979 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:31:57.356582 31979 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.358924 31979 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:57.350464 32632 raft_consensus.cc:686] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:57.360703 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.361083 32632 consensus_queue.cc:237] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.364283 32607 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:57.364917 32607 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.367180 32607 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.368505 26255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49594 (request call id 1469) took 1510 ms. Trace:
I20260924 14:31:57.368952 26255 rpcz_store.cc:276] 0924 14:31:55.858216 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:55.858327 (+   111us) service_pool.cc:227] Handling call
0924 14:31:55.858577 (+   250us) raft_consensus.cc:1457] Updating replica for 0 ops
0924 14:31:56.197134 (+338557us) raft_consensus.cc:1683] Filling consensus response to leader.
0924 14:31:57.368233 (+1171099us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54002a1054. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b934ef51 00007f09b934d99e 00007f09be28f2e9 00007f09b7975684 00007f09b7975612 00007f09b7975541 00007f09b79754bc 00007f09b7974772 00007f09b777894c 00007f09b7777d86
0924 14:31:57.368415 (+   182us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":773141760}
I20260924 14:31:57.368448 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:57.347102 32733 consensus_meta.cc:287] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Time spent flushing consensus metadata: real 0.760s	user 0.004s	sys 0.009s
I20260924 14:31:57.374910 32733 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.375650 26323 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:57.329244   300 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.377143 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:57.379050 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 208752us
I20260924 14:31:57.382475 26325 raft_consensus.cc:2354] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17.
I20260924 14:31:57.377854 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.384404 26332 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:31:57.386029 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.386663 26256 raft_consensus.cc:3037] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:57.387583 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.389961 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.332407 32184 raft_consensus.cc:504] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.392900 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.393508 32184 leader_election.cc:288] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.382654 32746 raft_consensus.cc:482] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:57.394191 32732 consensus_queue.cc:1035] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:57.392967 32698 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.395187 32698 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:57.396895 26251 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260924 14:31:57.347394 32695 raft_consensus.cc:686] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 21 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:57.405109 32479 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.413005 32695 consensus_queue.cc:237] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.413887 32479 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.382537 32707 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.410043 32746 raft_consensus.cc:504] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.417770 26256 raft_consensus.cc:2454] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:31:57.421646 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.421679 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.425793 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a94b735387741fca7f83059b64c0917" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.426762 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad26252e4284fb18db8d64f502c4a10" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.418357 32746 leader_election.cc:288] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.427484 26255 raft_consensus.cc:3037] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:31:57.437537 26255 raft_consensus.cc:2454] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 21.
I20260924 14:31:57.441331   300 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:57.441874 26400 raft_consensus.cc:3032] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Stepping down as leader of term 14
W20260924 14:31:57.442342 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61898.9us
I20260924 14:31:57.442353 26400 raft_consensus.cc:729] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:57.443562 26400 consensus_queue.cc:260] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.443778 32748 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.446017 32748 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.447204 32698 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:57.449980 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 82111.7us
I20260924 14:31:57.451363 27125 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:57.452792 26400 raft_consensus.cc:3037] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:31:57.452961 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.453677 26332 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:57.455612 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.456903 26328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:57.457562 26328 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:57.456959 26325 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260924 14:31:57.428175 32632 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.462934 32754 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader election won for term 17
I20260924 14:31:57.463434 26254 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260924 14:31:57.465202 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.465824 26249 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:57.463522 32754 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:57.466274 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.462984 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.469563 26250 raft_consensus.cc:1226] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Rejecting Update request from peer 45ab97b763804767846e9d27a01b561f for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260924 14:31:57.453783 26369 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.473819 26255 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260924 14:31:57.476253 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6029c377f94de7b21145dc0ac2f5a7" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:57.477094 26251 raft_consensus.cc:2354] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:57.472815 26369 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.478458 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:31:57.478895 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 133943us
I20260924 14:31:57.479041 26332 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 18
I20260924 14:31:57.479597 26328 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:57.483245 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.481041 32696 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:57.484409 32754 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.492164 32171 raft_consensus.cc:482] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:57.492797 32171 raft_consensus.cc:504] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.493948 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28de2d954e094686bc5d3b931d2924ce" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.462468 32632 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.512181 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:57.532466 26249 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:57.486140 32184 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260924 14:31:57.496838 32748 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.537392 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5581950b69104df485f842c6b78b1047" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.537557 32748 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:57.535780 27123 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.539817 31979 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 14:31:57.501325 32719 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:31:57.541553 32171 leader_election.cc:288] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.542591 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac352e5066864ac6be3f50d39a066493" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.550577 26400 raft_consensus.cc:1261] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260924 14:31:57.564230 26371 leader_election.cc:398] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:57.558081 26332 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 18.
I20260924 14:31:57.571621 26400 pending_rounds.cc:76] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Aborting all ops after (but not including) 8
I20260924 14:31:57.572302 26371 leader_election.cc:401] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260924 14:31:57.573565 26400 pending_rounds.cc:97] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260924 14:31:57.574283 26400 raft_consensus.cc:2866] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:57.542605 32748 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:57.576005 32691 raft_consensus.cc:2726] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433) with higher term. Message: Invalid argument: T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f for earlier term 16. Current term is 17.
I20260924 14:31:57.587800 27125 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:57.588712 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e4a72c4886471c8096037b808a6543" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.570560 32171 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:57.559721 32719 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:57.584285 32747 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260924 14:31:57.554113 32732 raft_consensus.cc:482] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:57.604194 27126 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:57.570293 32761 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260924 14:31:57.612730 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.615183 26402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46098 (request call id 2118) took 1020 ms. Trace:
I20260924 14:31:57.615617 26402 rpcz_store.cc:276] 0924 14:31:56.594404 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:56.594543 (+   139us) service_pool.cc:227] Handling call
0924 14:31:57.615009 (+1020466us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b54001e4bd4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:57.615133 (+   124us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":123667840}
W20260924 14:31:57.616559 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52202.5us
I20260924 14:31:57.583606 32631 consensus_queue.cc:1035] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260924 14:31:57.591003 32619 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260924 14:31:57.621502 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26229a9b02a646ac80993cd410cd7237" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.609411 32711 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:57.626012 27126 leader_election.cc:302] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:57.625353 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 741609us
I20260924 14:31:57.614060 32171 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:57.634253 32711 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Becoming Leader. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
W20260924 14:31:57.636044 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:31:57.634229 26221 leader_election.cc:398] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17.
I20260924 14:31:57.639240 26221 leader_election.cc:401] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260924 14:31:57.639374 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.640443 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:57.641623 32741 raft_consensus.cc:3037] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:57.661199 32632 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:57.666795 26221 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.667689 32748 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.641881 32719 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.668967 26221 leader_election.cc:302] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.669662 32748 raft_consensus.cc:2726] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260924 14:31:57.670831   316 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260924 14:31:57.671522 26257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49618 (request call id 1872) took 1115 ms. Trace:
I20260924 14:31:57.671985 26257 rpcz_store.cc:276] 0924 14:31:56.556231 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:56.560418 (+  4187us) service_pool.cc:227] Handling call
0924 14:31:57.671320 (+1110902us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54000c17d4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3471 00007f09b93522ae 00007f09be28fc27 00007f09b7977c04 00007f09b7977b92 00007f09b7977ac1 00007f09b7977a3c 00007f09b7976cf2 00007f09b777894c 00007f09b7777d86 00007f09b777772f
0924 14:31:57.671421 (+   101us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":360561536}
I20260924 14:31:57.609920 32732 raft_consensus.cc:504] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.677845 32732 leader_election.cc:288] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.678402 32711 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.721120 26297 leader_election.cc:302] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f
W20260924 14:31:57.728906 26221 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260924 14:31:57.730415 26221 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.643369 32767 raft_consensus.cc:2726] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:57.731240 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 92639.2us
I20260924 14:31:57.654371 32746 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260924 14:31:57.737457 26295 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.698519 32506 raft_consensus.cc:482] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
W20260924 14:31:57.742471 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 106610us
I20260924 14:31:57.732785 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.744016 27126 leader_election.cc:302] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.749711   324 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:57.753265 32654 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:57.753890 32654 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.744887 32682 raft_consensus.cc:2741] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260924 14:31:57.740130 26295 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:57.770174 26221 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.770879 26221 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.738504 32746 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:31:57.761719   319 log.cc:924] Time spent T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260924 14:31:57.780318 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 302085us
W20260924 14:31:57.772821   320 log.cc:924] Time spent T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.101s	user 0.002s	sys 0.000s
I20260924 14:31:57.772954 32631 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:31:57.788086 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 44198.8us
I20260924 14:31:57.774840 32698 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.778290 32746 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:57.762686 32506 raft_consensus.cc:504] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.798305 27126 leader_election.cc:302] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.779778 32717 raft_consensus.cc:482] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:57.809141 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.810184 32717 raft_consensus.cc:504] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.797586 32755 raft_consensus.cc:2726] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.821327 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:57.825644 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 52924.9us
I20260924 14:31:57.827507 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:31:57.828285 26401 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:57.828755 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.829938 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.828219 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:57.841733 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:57.826521 26221 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.845816 26221 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:57.846807 26221 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.852697 26401 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:31:57.847862 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67038d5a253c4d3e8d78c9fc7446aac8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:57.804111 32654 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:57.811625 32693 raft_consensus.cc:2726] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:57.864162 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.867555 26329 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:57.869383 26224 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: f59f3234edf84aab852fa989fb754d31
W20260924 14:31:57.871824 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 47901.5us
I20260924 14:31:57.863348 32717 leader_election.cc:288] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.872174 32748 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:57.874589 32698 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:31:57.901660 26255 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:57.902642 26369 leader_election.cc:302] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.911281 32698 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:31:57.921320   329 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260924 14:31:57.920907 32698 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.910892 32506 leader_election.cc:288] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:57.908840 32646 raft_consensus.cc:2726] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.931345 32171 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:57.931959 32171 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.933923 32171 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.934392 32746 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:57.938787 26252 raft_consensus.cc:1226] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Rejecting Update request from peer 7e09fbebe29a49d291d23f9d395fe335 for earlier term 15. Current term is 16. Ops: []
I20260924 14:31:57.939813 26332 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260924 14:31:57.944628 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.946553 32746 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:57.956492 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:31:57.957818 27123 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 61580.7us
I20260924 14:31:57.958241 32741 raft_consensus.cc:2726] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507) with higher term. Message: Invalid argument: T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 16. Current term is 17.
W20260924 14:31:57.962248 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.969188 26295 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.904793 32693 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:57.970161 32693 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:57.972671 32693 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:57.921455   323 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260924 14:31:57.975181 32687 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:31:57.976990 26295 leader_election.cc:334] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.978577 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.980052 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78da914f30dc4c65be5ff03ec099253b" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.943185 32740 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260924 14:31:57.993261 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:57.993942 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:57.994442 26369 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.994951 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:57.994804 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:57.996351 32752 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:31:57.997016 26369 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.996862 32752 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:57.997427 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.915143 32682 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260924 14:31:57.998813 32752 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:58.000802 32740 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:57.962870   305 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:58.004793 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d74021f6fc4625982decf68a590eac" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:58.002787 27126 leader_election.cc:302] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:58.003463 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:57.922860 32747 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260924 14:31:58.019893 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb2dfc0ec214f5497b6887c8f98ad9f" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:57.980289 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:58.028620 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:57.999331 32752 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:57.992640 32687 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:58.036118 26295 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:31:58.036422 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 123898us
W20260924 14:31:58.037062 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 56793.2us
W20260924 14:31:58.037721 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:58.040977 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:58.043246 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00498335d0b847f299d347155be5c437" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:58.047199 23135 thread.cc:536] Waited for 5000ms trying to join with heartbeat (tid 27223)
I20260924 14:31:58.048627 32682 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:58.051879 32171 raft_consensus.cc:2726] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:58.053519 27123 leader_election.cc:302] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:58.054428 32184 raft_consensus.cc:2726] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260924 14:31:58.058434 27123 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:58.063279 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260924 14:31:58.072009 26221 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:58.035646 32696 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:31:58.076128 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:58.075040 26221 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:58.085865 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:58.031039   305 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:58.072965 32754 raft_consensus.cc:2726] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260924 14:31:58.097855 26250 raft_consensus.cc:1261] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260924 14:31:58.098285 27123 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:58.080767 32748 raft_consensus.cc:2726] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:31:58.080039 32685 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:31:58.103261 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:58.141846 26329 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:58.145285 26255 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:58.151635 32597 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260924 14:31:58.151619 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 48457.3us
I20260924 14:31:58.160321 26397 raft_consensus.cc:1226] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Rejecting Update request from peer f59f3234edf84aab852fa989fb754d31 for earlier term 17. Current term is 18. Ops: []
I20260924 14:31:58.161844 32747 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260924 14:31:58.162453 27125 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57949.7us
I20260924 14:31:58.192199 32618 consensus_queue.cc:1046] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260924 14:31:58.184682 32682 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:58.169305 32663 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260924 14:31:58.171362 32654 consensus_queue.cc:1035] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260924 14:31:58.220453   335 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260924 14:31:58.231362 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:58.207031 32687 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260924 14:31:58.260273 26252 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260924 14:31:58.255661 32712 raft_consensus.cc:3032] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Stepping down as leader of term 15
I20260924 14:31:58.256820 26400 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:58.208801 32682 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:31:58.265823 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 119357us
I20260924 14:31:58.270861 32707 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:31:58.272882 32707 raft_consensus.cc:482] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:31:58.273425 32707 raft_consensus.cc:504] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:58.276113 26252 raft_consensus.cc:1261] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:58.280112 32632 consensus_queue.cc:1035] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:31:58.281152   324 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:31:58.281685   324 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:58.283715 26221 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:58.285285 26221 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:58.286134 32703 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260924 14:31:58.293195 32682 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:31:58.264961   305 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:58.296169 32610 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:58.296705 32610 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:58.298799 32610 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:58.262849 32712 raft_consensus.cc:729] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e09fbebe29a49d291d23f9d395fe335, State: Running, Role: LEADER
I20260924 14:31:58.308843 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:58.309685 26253 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 18.
I20260924 14:31:58.311690 26249 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:58.313211 26253 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260924 14:31:58.314131 26371 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:31:58.315884 32440 consensus_queue.cc:1046] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:58.316638 32440 consensus_queue.cc:1225] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Peer 6c22052275034c369b7f7dca086fbd1e log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260924 14:31:58.319626 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:58.320122 26329 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260924 14:31:58.320370 26323 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:31:58.323530 26397 raft_consensus.cc:1261] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260924 14:31:58.335837 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:58.337397 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:58.341630 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:58.342119 26295 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 131 similar messages]
I20260924 14:31:58.342644 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:58.303346 32707 leader_election.cc:288] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:58.343286 26330 raft_consensus.cc:2379] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 19: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:31:58.334924 32654 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:31:58.355556 26403 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:31:58.348780 26327 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:31:58.349020 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:58.349447 32693 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 14:31:58.353488 26323 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260924 14:31:58.364943 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5083097b524eb584b0af59aace9850" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:58.302259 32712 consensus_queue.cc:260] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:58.343223   340 log.cc:924] Time spent T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260924 14:31:58.338348   324 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:58.361038 32654 raft_consensus.cc:482] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:31:58.385577 26295 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:58.391435 32711 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:31:58.393885   325 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:58.396685 27123 leader_election.cc:302] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:58.398183 32710 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:58.398766 27123 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:58.391830   336 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260924 14:31:58.384944 32744 raft_consensus.cc:3032] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Stepping down as leader of term 17
I20260924 14:31:58.398964 32506 raft_consensus.cc:2726] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:58.363559 32687 consensus_queue.cc:1035] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260924 14:31:58.391197 26295 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:58.370347 32700 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260924 14:31:58.388535 32654 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
W20260924 14:31:58.428725 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:31:58.416671 32712 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:58.411759 32744 raft_consensus.cc:729] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:31:58.420873 32767 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:58.427174   309 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260924 14:31:58.438265   341 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260924 14:31:58.444684   345 log.cc:924] Time spent T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260924 14:31:58.512550 32744 consensus_queue.cc:260] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:58.595911 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 81614.8us
I20260924 14:31:58.611356   303 raft_consensus.cc:482] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:58.612047   303 raft_consensus.cc:504] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:58.662441 26332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:31:58.677987 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 75349.8us
W20260924 14:31:58.682637 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 113687us
I20260924 14:31:58.691098 26327 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Refusing update from remote peer 7e09fbebe29a49d291d23f9d395fe335: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260924 14:31:58.719318 32744 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:31:58.709913   303 leader_election.cc:288] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:31:58.758010   349 log.cc:924] Time spent T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260924 14:31:58.719784   310 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260924 14:31:58.760238   353 log.cc:924] Time spent T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260924 14:31:58.787217 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 86007us
W20260924 14:31:58.818954 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:58.823110   361 log.cc:924] Time spent T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260924 14:31:58.849084 32654 raft_consensus.cc:504] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:31:58.909271 32654 leader_election.cc:288] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:58.910293 26404 raft_consensus.cc:2354] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 19. Current term is 20.
I20260924 14:31:58.929777   305 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:58.930601   305 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:58.936978   305 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:59.005045 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:59.008653 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0756687d99e643c094531124c9438376" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:59.013471 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "014ae451ea994b5e9926dc80215382e4" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:31:59.026163 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 80543us
I20260924 14:31:59.033082 26224 leader_election.cc:398] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 19. Current term is 20.
I20260924 14:31:59.034122 26224 leader_election.cc:401] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260924 14:31:59.034785 32700 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 20
I20260924 14:31:59.027875 26324 raft_consensus.cc:1261] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260924 14:31:59.040169 26324 pending_rounds.cc:76] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 10
I20260924 14:31:59.040534 26324 pending_rounds.cc:97] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260924 14:31:59.040885 26324 raft_consensus.cc:2866] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:59.057325 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:31:59.058759 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.060322 23135 raft_consensus.cc:2229] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.060983 23135 pending_rounds.cc:62] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Trying to abort 1 pending ops.
I20260924 14:31:59.061337 23135 pending_rounds.cc:68] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335: Aborting op as it isn't in flight: id { term: 16 index: 10 } timestamp: 7332906214000791552 op_type: NO_OP noop_request { }
I20260924 14:31:59.061930 23135 raft_consensus.cc:2866] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260924 14:31:59.062400 23135 raft_consensus.cc:2258] T c4ddc9746f0a41f08d38d9ab46bac5ba P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.077708   363 consensus_queue.cc:1035] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:59.079202 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:59.080297 26324 raft_consensus.cc:3037] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
W20260924 14:31:59.083976 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 51032.9us
I20260924 14:31:59.092435 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.093722 23135 raft_consensus.cc:2229] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.094515 23135 raft_consensus.cc:2258] T 78da914f30dc4c65be5ff03ec099253b P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.097748 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.098542 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ededbbb82305459a8fbb9dc1f5731ee8" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:59.099118 23135 raft_consensus.cc:2229] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.101601 23135 raft_consensus.cc:2258] T 986260198b71418c831bfc4617da4603 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260924 14:31:59.104847 26073 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55200 (request call id 131) took 15081 ms (client timeout 15000 ms). Trace:
W20260924 14:31:59.105458 27126 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 44177.3us
W20260924 14:31:59.105459 26073 rpcz_store.cc:269] 0924 14:31:44.023865 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:44.027477 (+  3612us) service_pool.cc:227] Handling call
0924 14:31:59.104690 (+15077213us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:55.932325 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:55.999158 (+ 66833us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:55.999219 (+    61us) write_op.cc:435] Acquired schema lock
0924 14:31:55.999240 (+    21us) tablet.cc:660] Decoding operations
0924 14:31:56.001353 (+  2113us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:56.001460 (+   107us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:56.001506 (+    46us) tablet.cc:683] Acquiring locks for 28 operations
0924 14:31:56.203990 (+202484us) tablet.cc:699] Row locks acquired
0924 14:31:56.224377 (+ 20387us) write_op.cc:260] PREPARE: finished
0924 14:31:56.224623 (+   246us) write_op.cc:270] Start()
0924 14:31:56.224841 (+   218us) write_op.cc:276] Timestamp: P: 1790260316224533 usec, L: 0
0924 14:31:56.224890 (+    49us) op_driver.cc:331] REPLICATION: starting
0924 14:31:56.225875 (+   985us) log.cc:842] Serialized 11093 byte log entry
0924 14:31:56.984726 (+758851us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b44000232cc. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b626af11 00007f09b6338a95 00007f09b63394b2 00007f09b8fe1511 00007f09b8fdce1a 00007f09b8fd0bee 00007f09b8fcff36 00007f09b8fcfb58 00007f09b8fd9551 00007f09b8fd9509 00007f09b8fd9499
0924 14:31:57.352025 (+367299us) op_driver.cc:444] REPLICATION: finished
0924 14:31:57.536091 (+184066us) write_op.cc:301] APPLY: starting
0924 14:31:57.536196 (+   105us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:57.977848 (+441652us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:57.977893 (+    45us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:58.477004 (+499111us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:58.477410 (+   406us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=56,mrs_lookups=0
0924 14:31:58.477472 (+    62us) write_op.cc:312] APPLY: finished
0924 14:31:58.478024 (+   552us) log.cc:842] Serialized 243 byte log entry
0924 14:31:58.611287 (+133263us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:58.648480 (+ 37193us) write_op.cc:454] Released schema lock
0924 14:31:58.655269 (+  6789us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:58.683945 (+ 28676us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:58.693445 (+  9500us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":452805,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":183749,"apply.run_cpu_time_us":53728,"apply.run_wall_time_us":1175821,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121279,"num_ops":28,"prepare.queue_time_us":256950,"prepare.run_cpu_time_us":28738,"prepare.run_wall_time_us":388354,"raft.queue_time_us":130460,"raft.run_cpu_time_us":10535,"raft.run_wall_time_us":72648,"replication_time_us":1127016,"spinlock_wait_cycles":19827072,"thread_start_us":558012,"threads_started":5,"wal-append.queue_time_us":389966,"wal-append.run_cpu_time_us":44515,"wal-append.run_wall_time_us":1905397}]]}
I20260924 14:31:59.141741 27126 leader_election.cc:302] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
W20260924 14:31:59.160333 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:31:59.170464 27125 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:59.178865 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.179749   303 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:31:59.184322 26324 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:31:59.162567   305 raft_consensus.cc:2726] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:59.208603 26369 leader_election.cc:302] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:59.210186   364 raft_consensus.cc:2781] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:31:59.210871   364 raft_consensus.cc:686] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:31:59.211993   364 consensus_queue.cc:237] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:31:59.209185 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:59.220263 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:59.220896 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:59.215210 32700 raft_consensus.cc:2726] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583) with higher term. Message: Invalid argument: T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e for earlier term 19. Current term is 20.
I20260924 14:31:59.192691 23135 raft_consensus.cc:2229] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.232964 26252 raft_consensus.cc:2454] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
W20260924 14:31:59.233848 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:59.242806 23135 raft_consensus.cc:2258] T 968ded8aff90408e9baafab16ae17e14 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.227360 26074 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 11 to 13, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.253269 26074 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.266245 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.267284 23135 raft_consensus.cc:2229] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.268407 23135 raft_consensus.cc:2258] T 80f626c203394b4f8b85ab089d6aa044 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.269011 26074 catalog_manager.cc:5495] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.270778 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.273871 23135 raft_consensus.cc:2229] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.275511 23135 raft_consensus.cc:2258] T 8f9812df92004da0ba2c451e7b423e66 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.273383 26074 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.278385 26074 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 9 to 11, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.281302 26074 catalog_manager.cc:5495] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 4 to 10, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 10 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.284955 26074 catalog_manager.cc:5495] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 8 to 10, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 10 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.287211 26074 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
W20260924 14:31:59.299021 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:59.303153 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.317054 23135 raft_consensus.cc:2229] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.318645 23135 raft_consensus.cc:2258] T 00498335d0b847f299d347155be5c437 P 7e09fbebe29a49d291d23f9d395fe335 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.324316 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.325362 23135 raft_consensus.cc:2229] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.326591 23135 raft_consensus.cc:2258] T e81d1761568f40418d9259b74501d677 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.330325 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.331271 23135 raft_consensus.cc:2229] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.332288 23135 raft_consensus.cc:2258] T 26229a9b02a646ac80993cd410cd7237 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.337823 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.338840 23135 raft_consensus.cc:2229] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.340344 23135 raft_consensus.cc:2258] T 612b21159cac4f00b88b7076d49d4fec P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.346447   310 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:59.347251   310 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:31:59.350274 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:59.351539 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ed8dcd463347619717298c81a42998" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:59.353084 27126 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:31:59.353971   310 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:31:59.357777 32479 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:31:59.360085 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.361668 23135 raft_consensus.cc:2229] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.362533 23135 raft_consensus.cc:2258] T 4630fa065ede40338b319f0846fdd511 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.376912 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.378788 23135 raft_consensus.cc:2229] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.393222 23135 raft_consensus.cc:2258] T 7ed216090b8f4d88a827c18c45fd0422 P 7e09fbebe29a49d291d23f9d395fe335 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260924 14:31:59.412871 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260924 14:31:59.421988 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.435914 23135 raft_consensus.cc:2229] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.440164 23135 raft_consensus.cc:2258] T 64e0b4058f18400dab940d3832586746 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.447286 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.454746 23135 raft_consensus.cc:2229] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.456396 23135 raft_consensus.cc:2258] T fbdb392c304c4e15b1b6c10c61a95323 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.473380 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.474622 23135 raft_consensus.cc:2229] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.481525 23135 raft_consensus.cc:2258] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.506628 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.507944 23135 raft_consensus.cc:2229] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.513573 23135 raft_consensus.cc:2258] T d2f9b654ec4542a781bdc872c6ddbdce P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.516485 23135 tablet_replica.cc:330] stopping tablet replica
W20260924 14:31:59.518661 26074 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 142) took 38805 ms (client timeout 15000 ms). Trace:
I20260924 14:31:59.519668 23135 raft_consensus.cc:2229] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.520332 23135 raft_consensus.cc:2258] T 014ae451ea994b5e9926dc80215382e4 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.530756 26078 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 8 to 11, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:59.532796 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.533999 23135 raft_consensus.cc:2229] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.533537 26078 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 10 to 11, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 11 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:31:59.535332 23135 raft_consensus.cc:2258] T bcac7743e33046019152a570ae473ca3 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.537791 26078 catalog_manager.cc:5495] T ac352e5066864ac6be3f50d39a066493 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 8 to 9, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 9 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.547060 26078 catalog_manager.cc:5495] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.551126 26078 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.558844 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.560034 23135 raft_consensus.cc:2229] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.560788 23135 raft_consensus.cc:2258] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.563995 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.565464 23135 raft_consensus.cc:2229] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.567386 23135 raft_consensus.cc:2258] T f272a708f6ca4bd8bc3a3c4925abad6d P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.571818 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.600965   300 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:31:59.601730   300 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:31:59.611683 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:31:59.612727 23135 raft_consensus.cc:2229] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.613169 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:59.614137 26255 raft_consensus.cc:2379] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 16: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:31:59.615674 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:31:59.616555 32641 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:31:59.619837   300 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:31:59.625650 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260924 14:31:59.519168 26074 rpcz_store.cc:269] 0924 14:31:20.713395 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:20.713700 (+   305us) service_pool.cc:227] Handling call
0924 14:31:59.518576 (+38804876us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:59.332221 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:59.332715 (+   494us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:59.332744 (+    29us) write_op.cc:435] Acquired schema lock
0924 14:31:59.332759 (+    15us) tablet.cc:660] Decoding operations
0924 14:31:59.333636 (+   877us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:59.333720 (+    84us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:59.333746 (+    26us) tablet.cc:683] Acquiring locks for 8 operations
0924 14:31:59.334403 (+   657us) tablet.cc:699] Row locks acquired
0924 14:31:59.334425 (+    22us) write_op.cc:260] PREPARE: finished
0924 14:31:59.334581 (+   156us) write_op.cc:270] Start()
0924 14:31:59.334696 (+   115us) write_op.cc:276] Timestamp: P: 1790260319334536 usec, L: 0
0924 14:31:59.334718 (+    22us) op_driver.cc:331] REPLICATION: starting
0924 14:31:59.359608 (+ 24890us) log.cc:842] Serialized 3281 byte log entry
0924 14:31:59.375038 (+ 15430us) op_driver.cc:444] REPLICATION: finished
0924 14:31:59.413512 (+ 38474us) write_op.cc:301] APPLY: starting
0924 14:31:59.413607 (+    95us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:59.418385 (+  4778us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:59.418496 (+   111us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:59.423216 (+  4720us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:59.423487 (+   271us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0924 14:31:59.423541 (+    54us) write_op.cc:312] APPLY: finished
0924 14:31:59.423813 (+   272us) log.cc:842] Serialized 80 byte log entry
0924 14:31:59.424600 (+   787us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:59.424756 (+   156us) write_op.cc:454] Released schema lock
0924 14:31:59.424808 (+    52us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:59.425089 (+   281us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:59.425119 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":64253,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":38171,"apply.run_cpu_time_us":11184,"apply.run_wall_time_us":12430,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37599,"num_ops":8,"prepare.queue_time_us":38676,"prepare.run_cpu_time_us":3822,"prepare.run_wall_time_us":31800,"replication_time_us":40208,"thread_start_us":59963,"threads_started":2}]]}
I20260924 14:31:59.614161 23135 raft_consensus.cc:2258] T a2799f93a20d4fd58d617adf2eda3070 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.630756 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.631718 23135 raft_consensus.cc:2229] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.644155 32641 raft_consensus.cc:2726] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:31:59.644996 32479 raft_consensus.cc:482] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:59.645663 32479 raft_consensus.cc:504] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:31:59.649425 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:59.652915 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f81913531e84a379f3bc3886ea64e95" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:31:59.654452 27126 leader_election.cc:302] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:31:59.655315 32705 raft_consensus.cc:2726] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:31:59.656903 32479 leader_election.cc:288] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:31:59.663971 26249 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260924 14:31:59.665537   364 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260924 14:31:59.674373 23135 raft_consensus.cc:2258] T a83a2b98291f41d7beed3c1329ba4bc9 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.675689 26073 catalog_manager.cc:5495] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 21, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 21 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.677354 26073 catalog_manager.cc:5495] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:59.678861 26073 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 18, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 18 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
W20260924 14:31:59.682614 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260924 14:31:59.682251 26073 catalog_manager.cc:5495] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 22, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 22 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:59.684309 26073 catalog_manager.cc:5495] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 6 to 19, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:31:59.675190 26078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 8) took 41216 ms (client timeout 15000 ms). Trace:
I20260924 14:31:59.686568 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.685400 26073 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.688144 26073 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 15, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.689040 26073 catalog_manager.cc:5495] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.690066 26073 catalog_manager.cc:5495] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 19, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:59.687502 23135 raft_consensus.cc:2229] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.691159 26073 catalog_manager.cc:5495] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 16 to 17, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 17 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
W20260924 14:31:59.686269 26078 rpcz_store.cc:269] 0924 14:31:18.458565 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:18.463681 (+  5116us) service_pool.cc:227] Handling call
0924 14:31:59.675104 (+41211423us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:31:59.555881 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:31:59.556402 (+   521us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:31:59.556442 (+    40us) write_op.cc:435] Acquired schema lock
0924 14:31:59.556459 (+    17us) tablet.cc:660] Decoding operations
0924 14:31:59.557170 (+   711us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:31:59.557228 (+    58us) write_op.cc:641] Partition lock acquired for write op
0924 14:31:59.557256 (+    28us) tablet.cc:683] Acquiring locks for 5 operations
0924 14:31:59.557840 (+   584us) tablet.cc:699] Row locks acquired
0924 14:31:59.557860 (+    20us) write_op.cc:260] PREPARE: finished
0924 14:31:59.558026 (+   166us) write_op.cc:270] Start()
0924 14:31:59.558143 (+   117us) write_op.cc:276] Timestamp: P: 1790260319557976 usec, L: 0
0924 14:31:59.558165 (+    22us) op_driver.cc:331] REPLICATION: starting
0924 14:31:59.559045 (+   880us) log.cc:842] Serialized 2108 byte log entry
0924 14:31:59.562262 (+  3217us) op_driver.cc:444] REPLICATION: finished
0924 14:31:59.563516 (+  1254us) write_op.cc:301] APPLY: starting
0924 14:31:59.563592 (+    76us) tablet.cc:1365] starting BulkCheckPresence
0924 14:31:59.566222 (+  2630us) tablet.cc:1368] finished BulkCheckPresence
0924 14:31:59.566318 (+    96us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:31:59.571734 (+  5416us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:31:59.571918 (+   184us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0924 14:31:59.571964 (+    46us) write_op.cc:312] APPLY: finished
0924 14:31:59.572182 (+   218us) log.cc:842] Serialized 56 byte log entry
0924 14:31:59.572346 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:59.572452 (+   106us) write_op.cc:454] Released schema lock
0924 14:31:59.572495 (+    43us) write_op.cc:341] FINISH: Updating metrics
0924 14:31:59.572776 (+   281us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:31:59.572805 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":634269,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":1005,"apply.run_cpu_time_us":7238,"apply.run_wall_time_us":9599,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20920,"num_ops":5,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":3385,"prepare.run_wall_time_us":3873,"replication_time_us":4003}]]}
I20260924 14:31:59.696018 26073 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 18, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 18 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.699728 23135 raft_consensus.cc:2258] T 4a92b51c57b441158371bdfdf2d193c1 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.699499 26073 catalog_manager.cc:5495] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 16 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 18 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:59.701205 26073 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 19, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.702493 26073 catalog_manager.cc:5495] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 17, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 17 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.703709 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.703881 26073 catalog_manager.cc:5495] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:31:59.704694 23135 raft_consensus.cc:2229] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.705698 26073 catalog_manager.cc:5495] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 15, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 15 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.706549 23135 raft_consensus.cc:2258] T 5581950b69104df485f842c6b78b1047 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.707319 26073 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.708638 26073 catalog_manager.cc:5495] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 18, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 18 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.709764 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.709913 26073 catalog_manager.cc:5495] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.710749 23135 raft_consensus.cc:2229] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.711148 26073 catalog_manager.cc:5495] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:59.712411 26073 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.713518 26073 catalog_manager.cc:5495] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 13 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 16 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.714707 26073 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.715991 26073 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 18, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 18 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:31:59.717268 26073 catalog_manager.cc:5495] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 17, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 17 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.718726 26073 catalog_manager.cc:5495] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 11 to 16, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 16 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.720268 26073 catalog_manager.cc:5495] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 18 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:59.721501 26073 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 23 to 24, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 24 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:59.722718 26073 catalog_manager.cc:5495] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 15, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 15 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.724009 26073 catalog_manager.cc:5495] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.725561 26073 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 16 to 19, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 19 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.727075 23135 raft_consensus.cc:2258] T 898441fbc17a42e09c7639e00843680b P 7e09fbebe29a49d291d23f9d395fe335 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.727152 26073 catalog_manager.cc:5495] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 10 to 17. New cstate: current_term: 17 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:31:59.738744 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.736095 26073 catalog_manager.cc:5495] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 16 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:31:59.740633 26073 catalog_manager.cc:5495] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } }
I20260924 14:31:59.740072 23135 raft_consensus.cc:2229] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 24 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.745393 23135 raft_consensus.cc:2258] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 7e09fbebe29a49d291d23f9d395fe335 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.751292 32618 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:31:59.756558 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.757580 23135 raft_consensus.cc:2229] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.761626 23135 raft_consensus.cc:2258] T 70ed8dcd463347619717298c81a42998 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.757104 32618 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:59.777320 32618 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:59.778070 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:31:59.778673 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:31:59.778860 26324 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:59.779394 26255 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 16.
I20260924 14:31:59.780254 27123 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:31:59.781261 32705 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:31:59.781652 32705 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:31:59.781993 32705 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:59.789549 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.790551 23135 raft_consensus.cc:2229] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.792140 23135 raft_consensus.cc:2258] T dda86ff6977641a8af47dad8d1d1a049 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.798003 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.798908 23135 raft_consensus.cc:2229] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.799978 23135 raft_consensus.cc:2258] T 16dc088d2abf460eb278e44fc4f60620 P 7e09fbebe29a49d291d23f9d395fe335 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.802495 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.803280 23135 raft_consensus.cc:2229] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.804292 23135 raft_consensus.cc:2258] T f59f0af5dd6048918c6579b295ac5510 P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.806829 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.807421 23135 raft_consensus.cc:2229] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.808455 23135 raft_consensus.cc:2258] T 97ce0c6b37e04bf89405065d71329eb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.809237 26324 raft_consensus.cc:1261] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260924 14:31:59.811113   364 consensus_queue.cc:1035] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:31:59.811832   364 consensus_queue.cc:1225] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [LEADER]: Peer f59f3234edf84aab852fa989fb754d31 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260924 14:31:59.814563 26324 pending_rounds.cc:76] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Aborting all ops after (but not including) 10
I20260924 14:31:59.809711 32705 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:31:59.814895 26324 pending_rounds.cc:97] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260924 14:31:59.815196 26324 raft_consensus.cc:2866] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260924 14:31:59.817230 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:31:59.817929 26255 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:59.826028 23135 tablet_replica.cc:330] stopping tablet replica
W20260924 14:31:59.826566 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260924 14:31:59.829109 23135 raft_consensus.cc:2229] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.835069 26255 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:59.837249 27125 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335; no voters: 
I20260924 14:31:59.835090 23135 raft_consensus.cc:2258] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.838701 32618 raft_consensus.cc:2684] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Received election callback for term 17 while not running: Illegal state: RaftConsensus is not running: State = Stopped
I20260924 14:31:59.849648 32705 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P 7e09fbebe29a49d291d23f9d395fe335 [CANDIDATE]: Term 17 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:31:59.850653 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "7e09fbebe29a49d291d23f9d395fe335" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:31:59.851356 26330 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:31:59.852945 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.853885 23135 raft_consensus.cc:2229] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.855054 23135 raft_consensus.cc:2258] T 62575580f9ac4403b5f531f616fe41fd P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.860517 26330 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e09fbebe29a49d291d23f9d395fe335 in term 17.
I20260924 14:31:59.871650 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.889585 23135 raft_consensus.cc:2229] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 LEADER]: Raft consensus shutting down.
I20260924 14:31:59.892211 23135 raft_consensus.cc:2258] T 0eec7b6dbbd94479aa179fa23fc1f403 P 7e09fbebe29a49d291d23f9d395fe335 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.897037 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.897876 23135 raft_consensus.cc:2229] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.898499 23135 raft_consensus.cc:2258] T 051d85003b754a6e95eaf1d19022c217 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.921303 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.922066 23135 raft_consensus.cc:2229] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.922658 23135 raft_consensus.cc:2258] T c6e4a72c4886471c8096037b808a6543 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.936365 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.937196 23135 raft_consensus.cc:2229] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.952353 23135 raft_consensus.cc:2258] T d4d74021f6fc4625982decf68a590eac P 7e09fbebe29a49d291d23f9d395fe335 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.967664 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:31:59.968497 23135 raft_consensus.cc:2229] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:31:59.969058 23135 raft_consensus.cc:2258] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 7e09fbebe29a49d291d23f9d395fe335 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:31:59.993840 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.004220 23135 raft_consensus.cc:2229] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:00.005406 23135 raft_consensus.cc:2258] T e188e3a6a42340858d5f3ada3f5e867d P 7e09fbebe29a49d291d23f9d395fe335 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.009266 26079 catalog_manager.cc:5495] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.013952 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.014938 23135 raft_consensus.cc:2229] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:00.023595 23135 raft_consensus.cc:2258] T 1c5083097b524eb584b0af59aace9850 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.039083 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.040066 23135 raft_consensus.cc:2229] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:00.041229 23135 raft_consensus.cc:2258] T e912296c0f6f4c4d89bbec1581a46cf9 P 7e09fbebe29a49d291d23f9d395fe335 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.044410 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.045253 23135 raft_consensus.cc:2229] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:00.045846 23135 raft_consensus.cc:2258] T 0756687d99e643c094531124c9438376 P 7e09fbebe29a49d291d23f9d395fe335 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.064944 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.064328 26073 catalog_manager.cc:5495] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 20, leader changed from f59f3234edf84aab852fa989fb754d31 (127.22.151.195) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 20 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } }
I20260924 14:32:00.065811 23135 raft_consensus.cc:2229] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:32:00.067098 23135 raft_consensus.cc:2258] T 168958697ba24715a127c9c7b7c520ff P 7e09fbebe29a49d291d23f9d395fe335 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.078900 26077 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 13 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 15 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:00.078959 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.092299 23135 raft_consensus.cc:2229] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:00.092974 23135 raft_consensus.cc:2258] T 9f81913531e84a379f3bc3886ea64e95 P 7e09fbebe29a49d291d23f9d395fe335 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:00.093605 26077 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 15 to 17, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 17 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.095130 26077 catalog_manager.cc:5495] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.097079 26077 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 6 to 11, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 11 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:00.099123 23135 tablet_replica.cc:330] stopping tablet replica
I20260924 14:32:00.099840 23135 raft_consensus.cc:2229] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 LEADER]: Raft consensus shutting down.
I20260924 14:32:00.101162 23135 raft_consensus.cc:2258] T 00361946476c4fecbd244a86b9cfb752 P 7e09fbebe29a49d291d23f9d395fe335 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:00.102612 26079 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 141) took 67038 ms (client timeout 15000 ms). Trace:
W20260924 14:32:00.103137 26079 rpcz_store.cc:269] 0924 14:30:53.063866 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:53.064054 (+   188us) service_pool.cc:227] Handling call
0924 14:32:00.102536 (+67038482us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:32:00.027722 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:32:00.028216 (+   494us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:32:00.028263 (+    47us) write_op.cc:435] Acquired schema lock
0924 14:32:00.028284 (+    21us) tablet.cc:660] Decoding operations
0924 14:32:00.028630 (+   346us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:32:00.028681 (+    51us) write_op.cc:641] Partition lock acquired for write op
0924 14:32:00.028705 (+    24us) tablet.cc:683] Acquiring locks for 1 operations
0924 14:32:00.028886 (+   181us) tablet.cc:699] Row locks acquired
0924 14:32:00.028906 (+    20us) write_op.cc:260] PREPARE: finished
0924 14:32:00.029043 (+   137us) write_op.cc:270] Start()
0924 14:32:00.029153 (+   110us) write_op.cc:276] Timestamp: P: 1790260320029002 usec, L: 0
0924 14:32:00.029171 (+    18us) op_driver.cc:331] REPLICATION: starting
0924 14:32:00.029756 (+   585us) log.cc:842] Serialized 543 byte log entry
0924 14:32:00.032215 (+  2459us) op_driver.cc:444] REPLICATION: finished
0924 14:32:00.032709 (+   494us) write_op.cc:301] APPLY: starting
0924 14:32:00.032797 (+    88us) tablet.cc:1365] starting BulkCheckPresence
0924 14:32:00.033660 (+   863us) tablet.cc:1368] finished BulkCheckPresence
0924 14:32:00.033767 (+   107us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:32:00.034483 (+   716us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:32:00.034652 (+   169us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0924 14:32:00.034696 (+    44us) write_op.cc:312] APPLY: finished
0924 14:32:00.034887 (+   191us) log.cc:842] Serialized 24 byte log entry
0924 14:32:00.035034 (+   147us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.035116 (+    82us) write_op.cc:454] Released schema lock
0924 14:32:00.035153 (+    37us) write_op.cc:341] FINISH: Updating metrics
0924 14:32:00.036578 (+  1425us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.036609 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2567379,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":264,"apply.run_cpu_time_us":3055,"apply.run_wall_time_us":4132,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":1432,"prepare.run_cpu_time_us":2318,"prepare.run_wall_time_us":2318,"replication_time_us":2950}]]}
I20260924 14:32:00.103688 26077 catalog_manager.cc:5495] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.122366 26077 catalog_manager.cc:5495] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 11 to 15, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 15 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:00.141039 26077 catalog_manager.cc:5495] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:32:00.151891 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:32:00.151197 26077 catalog_manager.cc:5495] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 10 to 14, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 14 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.155625 26077 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.157032 26077 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 10 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 15 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
W20260924 14:32:00.200532 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260924 14:32:00.309703   385 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.310503   385 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.317582   385 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:00.319186 26221 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:32:00.325271 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.328053 26224 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.328984 32700 raft_consensus.cc:2726] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:00.336370   385 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.337077   385 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.341188   385 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:00.347587 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:32:00.354223 26221 leader_election.cc:334] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:32:00.354833 26221 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:00.357515   385 raft_consensus.cc:2726] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:00.372154   364 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.372917   364 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.376665 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:32:00.382437 26369 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:32:00.383553   364 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.382963 26369 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.384934   364 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260924 14:32:00.386842 26076 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 14 to 18, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 18 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
W20260924 14:32:00.402971 26077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 143) took 13443 ms (client timeout 15000 ms). Trace:
W20260924 14:32:00.406057 26077 rpcz_store.cc:269] 0924 14:31:46.959951 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:46.969425 (+  9474us) service_pool.cc:227] Handling call
0924 14:31:47.550136 (+580711us) spinlock_profiling.cc:243] Waited 89.3 ms on lock 0x7b540015fed4. stack: 00007f09b63f4033 00007f09b63f3f30 00007f09b5d52534 00007f09bfa5e8b4 00007f09bfa5e849 00007f09b92d3af0 00007f09b9356f2a 00007f09bf2a8d6c 00007f09bf3a2c3e 00007f09bd6b4fb4 00007f09bd6b4f42 00007f09bd6b4e71 00007f09bd6b4dec 00007f09bd6b40a2 00007f09b777894c 00007f09b7777d86
0924 14:32:00.402887 (+12852751us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:32:00.187293 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:32:00.187802 (+   509us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:32:00.187837 (+    35us) write_op.cc:435] Acquired schema lock
0924 14:32:00.187856 (+    19us) tablet.cc:660] Decoding operations
0924 14:32:00.188884 (+  1028us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:32:00.188951 (+    67us) write_op.cc:641] Partition lock acquired for write op
0924 14:32:00.188986 (+    35us) tablet.cc:683] Acquiring locks for 10 operations
0924 14:32:00.189783 (+   797us) tablet.cc:699] Row locks acquired
0924 14:32:00.189800 (+    17us) write_op.cc:260] PREPARE: finished
0924 14:32:00.189946 (+   146us) write_op.cc:270] Start()
0924 14:32:00.190059 (+   113us) write_op.cc:276] Timestamp: P: 1790260320189900 usec, L: 0
0924 14:32:00.190081 (+    22us) op_driver.cc:331] REPLICATION: starting
0924 14:32:00.190698 (+   617us) log.cc:842] Serialized 4063 byte log entry
0924 14:32:00.194731 (+  4033us) op_driver.cc:444] REPLICATION: finished
0924 14:32:00.197042 (+  2311us) write_op.cc:301] APPLY: starting
0924 14:32:00.197131 (+    89us) tablet.cc:1365] starting BulkCheckPresence
0924 14:32:00.204681 (+  7550us) tablet.cc:1368] finished BulkCheckPresence
0924 14:32:00.204823 (+   142us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:32:00.220577 (+ 15754us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:32:00.220802 (+   225us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0924 14:32:00.220846 (+    44us) write_op.cc:312] APPLY: finished
0924 14:32:00.221163 (+   317us) log.cc:842] Serialized 96 byte log entry
0924 14:32:00.296601 (+ 75438us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.311607 (+ 15006us) write_op.cc:454] Released schema lock
0924 14:32:00.311654 (+    47us) write_op.cc:341] FINISH: Updating metrics
0924 14:32:00.311928 (+   274us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.311961 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21244,"reported_tablets":44,"spinlock_wait_cycles":205277952,"child_traces":[["op",{"apply.queue_time_us":2087,"apply.run_cpu_time_us":14111,"apply.run_wall_time_us":115225,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41840,"num_ops":10,"prepare.queue_time_us":104,"prepare.run_cpu_time_us":3695,"prepare.run_wall_time_us":15888,"replication_time_us":4549}]]}
W20260924 14:32:00.473238 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:00.563427   385 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.594177   385 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.606133   385 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.607424 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.616017   364 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.617326   364 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.621049   300 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.621464   386 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.622458   300 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.622754   386 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.625401   364 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.625420   385 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.626516   385 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.630779   385 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.632103   300 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.632818 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.634330 26400 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 11.
I20260924 14:32:00.635586 26224 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:00.636597   385 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:32:00.637058   385 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.637511   385 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
W20260924 14:32:00.640125 26295 leader_election.cc:334] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:00.640949 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.636626   386 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:00.643726 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:00.644438 26369 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.648967   390 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.650087   364 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.650034   390 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.650906   364 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.652971   390 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:00.653414   364 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.660694   364 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.661514   364 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:00.668143 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.678534   364 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.679764   394 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.680645   394 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.681453   385 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.684497 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:32:00.686323 26369 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.686671   385 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.687983 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:00.688789 26400 raft_consensus.cc:3037] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:32:00.690411   394 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:00.697921 26295 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.698781 26295 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.707645   392 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.708180 32610 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.708663   392 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.709192 32610 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.712410   392 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.712989 32610 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.718570   386 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.721904   386 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:00.724280 26369 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:00.725023 26369 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.725266   386 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.725786   300 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.726611   300 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.727627 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.728366 26406 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:32:00.729044   300 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.730083 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:00.730769 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.731289   394 raft_consensus.cc:2726] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:00.731674 26324 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:32:00.732543   394 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260924 14:32:00.733547 26076 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 143) took 18375 ms (client timeout 15000 ms). Trace:
I20260924 14:32:00.733734   300 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.733191   394 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.736112   394 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:32:00.734776   300 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.741004   385 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.741916   385 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.752948   385 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:00.754818   394 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.756402 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:00.742974 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 45103.5us
I20260924 14:32:00.757704 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:00.757973   355 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260924 14:32:00.758466   355 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.758823   355 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Advancing to term 17
I20260924 14:32:00.755635   300 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.762903   394 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.764436 32641 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.765007   364 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.765326 32641 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.765838   364 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.768194 32641 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.770341   364 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:00.774118 26369 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.775144 26075 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.780128 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.780977 26255 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:32:00.782169 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.782898 26256 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:32:00.784761 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:32:00.785552 26080 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 7) took 71569 ms (client timeout 15000 ms). Trace:
I20260924 14:32:00.771183   355 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:00.786020 26080 rpcz_store.cc:269] 0924 14:30:49.215997 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:30:49.216201 (+   204us) service_pool.cc:227] Handling call
0924 14:32:00.785300 (+71569099us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":50133,"reported_tablets":45}
W20260924 14:32:00.787794 26067 connection.cc:441] server connection from 127.0.0.1:34948 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 7) could send its response
I20260924 14:32:00.789819   355 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.793597 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.794430 26253 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
W20260924 14:32:00.795403 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.795975 26075 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.796615 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.797523 26324 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:32:00.797729 26075 catalog_manager.cc:5495] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 15 to 16, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 16 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:00.799065 26406 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 17: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:32:00.795356 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.800446 26255 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 11.
I20260924 14:32:00.802129 26075 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 13, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 13 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
W20260924 14:32:00.808756 26295 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:00.809875 26295 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:00.810510 26295 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:00.811038 26295 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.811473 26295 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.812335   394 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260924 14:32:00.734316 26076 rpcz_store.cc:269] 0924 14:31:42.312225 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:42.404867 (+ 92642us) service_pool.cc:227] Handling call
0924 14:32:00.687351 (+18282484us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:32:00.416961 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:32:00.423072 (+  6111us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:32:00.423474 (+   402us) write_op.cc:435] Acquired schema lock
0924 14:32:00.423870 (+   396us) tablet.cc:660] Decoding operations
0924 14:32:00.424765 (+   895us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:32:00.424823 (+    58us) write_op.cc:641] Partition lock acquired for write op
0924 14:32:00.424848 (+    25us) tablet.cc:683] Acquiring locks for 1 operations
0924 14:32:00.425045 (+   197us) tablet.cc:699] Row locks acquired
0924 14:32:00.425059 (+    14us) write_op.cc:260] PREPARE: finished
0924 14:32:00.425206 (+   147us) write_op.cc:270] Start()
0924 14:32:00.425318 (+   112us) write_op.cc:276] Timestamp: P: 1790260320425162 usec, L: 0
0924 14:32:00.425334 (+    16us) op_driver.cc:331] REPLICATION: starting
0924 14:32:00.425935 (+   601us) log.cc:842] Serialized 543 byte log entry
0924 14:32:00.427609 (+  1674us) op_driver.cc:444] REPLICATION: finished
0924 14:32:00.427933 (+   324us) write_op.cc:301] APPLY: starting
0924 14:32:00.428011 (+    78us) tablet.cc:1365] starting BulkCheckPresence
0924 14:32:00.429257 (+  1246us) tablet.cc:1368] finished BulkCheckPresence
0924 14:32:00.429353 (+    96us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:32:00.430003 (+   650us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:32:00.430153 (+   150us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0924 14:32:00.430190 (+    37us) write_op.cc:312] APPLY: finished
0924 14:32:00.430361 (+   171us) log.cc:842] Serialized 24 byte log entry
0924 14:32:00.430499 (+   138us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.430578 (+    79us) write_op.cc:454] Released schema lock
0924 14:32:00.430613 (+    35us) write_op.cc:341] FINISH: Updating metrics
0924 14:32:00.430847 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:00.430871 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":417490,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":100,"apply.run_cpu_time_us":2663,"apply.run_wall_time_us":3139,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":1985,"prepare.run_cpu_time_us":2753,"prepare.run_wall_time_us":9841,"replication_time_us":2159}]]}
I20260924 14:32:00.816911   355 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.817086   364 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:00.817512 26295 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.817770   355 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.817891   364 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.819798   394 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.820492   364 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.820495   394 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.823416   394 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.824404 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.825204 26401 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:32:00.826431 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:32:00.826858 26369 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.827484   394 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:32:00.828037   394 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:00.827564 26369 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.827942 26075 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.832237   364 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.832902   364 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.834122   385 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.834980   364 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.834862   385 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.829007   355 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.837510   385 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:00.827564 26295 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.829720 26400 raft_consensus.cc:2454] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 12.
I20260924 14:32:00.828665   394 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:00.837968 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:32:00.841578 26369 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.842355 26224 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:00.843559   385 raft_consensus.cc:2781] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:32:00.844108   385 raft_consensus.cc:686] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:00.845345   385 consensus_queue.cc:237] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.846621 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.847370 26406 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 21.
I20260924 14:32:00.848383 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:00.849450   390 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260924 14:32:00.849844   390 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.850216   390 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:32:00.858985   394 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.849721 26075 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 18 to 19, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 19 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.870237 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.869908 26075 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.870910 26323 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:32:00.872519 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.873818   355 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:00.873375 26075 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 15 to 17, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193). New cstate: current_term: 17 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.874298   355 raft_consensus.cc:482] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.863251 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.864941   389 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.875138   355 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:00.875478   389 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.877977   389 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:00.865764 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:00.878877 26324 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 17: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:00.867110 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.880148 26330 raft_consensus.cc:2379] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:00.867825 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f272a708f6ca4bd8bc3a3c4925abad6d" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:00.881026 26400 raft_consensus.cc:3037] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:00.881361 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:00.882330   364 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260924 14:32:00.884033   364 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:00.884439   364 raft_consensus.cc:482] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.884853   364 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:00.885804 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.890177 32610 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:32:00.890552 32610 raft_consensus.cc:482] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.890941 32610 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:00.893735 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:00.899187   300 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.875049 26075 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:00.899854   300 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:00.890702 26371 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.901957   300 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.901414 26075 catalog_manager.cc:5495] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 15 to 16, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 16 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:00.903419 26075 catalog_manager.cc:5495] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 19, leader changed from 45ab97b763804767846e9d27a01b561f (127.22.151.196) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 19 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:00.905385 26075 catalog_manager.cc:5495] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "7e09fbebe29a49d291d23f9d395fe335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:32:00.908340   397 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260924 14:32:00.909485 26295 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.915171 26400 raft_consensus.cc:2454] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:32:00.860668   390 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.903864 32610 raft_consensus.cc:504] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.874365 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:00.920557   364 raft_consensus.cc:504] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:00.922979   392 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:00.924898 26298 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:00.898394   355 raft_consensus.cc:504] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:00.933688 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 59537.9us
I20260924 14:32:00.935784 32700 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:32:00.936223 32700 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.936654 32700 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:32:00.936805 32641 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.937698 32641 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.938313   364 leader_election.cc:288] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.932350   300 raft_consensus.cc:2781] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:32:00.931998   389 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.945022 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.945449 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.946468 26253 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:00.947433   386 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.948120   386 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.948607 32610 leader_election.cc:288] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:00.950467 26369 leader_election.cc:334] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.950855   386 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.951548   389 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:00.953728 26369 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.957041 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:00.957317 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.958272 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.958971 26252 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 14.
I20260924 14:32:00.958811 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:32:00.950774 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:00.952162 32641 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.961297 26256 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:32:00.963486 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:00.965224 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:32:00.969455 26295 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.943481   300 raft_consensus.cc:686] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:00.978129   389 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:00.969300   364 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.979858 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.980029   364 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.981036 32610 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:00.981308 26400 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 24.
I20260924 14:32:00.976917   300 consensus_queue.cc:237] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.983058 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.983517 32610 raft_consensus.cc:482] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.983762 26330 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 24.
I20260924 14:32:00.983973 32610 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
W20260924 14:32:00.984992 26295 leader_election.cc:334] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.985510 26295 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.985622 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:00.986402 32641 raft_consensus.cc:2726] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:00.987069   392 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260924 14:32:00.987372 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:00.987510   392 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.987926   398 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.988024   392 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 24 FOLLOWER]: Advancing to term 25
W20260924 14:32:00.988530 26369 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.988745   397 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260924 14:32:00.989190   397 raft_consensus.cc:482] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:00.989233   364 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.989625   397 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:32:00.988576   398 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:00.992600   398 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:00.988745 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:00.995697 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:00.996352 26330 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 19.
I20260924 14:32:00.998457   397 raft_consensus.cc:504] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.000686   397 leader_election.cc:288] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:00.999372 32700 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.001880 26256 raft_consensus.cc:2379] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 13: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.002162 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.002883   397 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:00.941313   394 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.003240   397 raft_consensus.cc:482] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:00.987018 26221 leader_election.cc:334] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.011042   398 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.012794 26221 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.005302 26295 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:00.997942 32641 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260924 14:32:01.013401   398 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.014073 32641 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.014711   397 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.014827 32641 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 24 FOLLOWER]: Advancing to term 25
I20260924 14:32:01.015671   398 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:01.004858 26369 leader_election.cc:334] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.017493 26369 leader_election.cc:334] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.019013 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbdb392c304c4e15b1b6c10c61a95323" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.019730 26330 raft_consensus.cc:3037] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.019902   389 raft_consensus.cc:2726] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:01.005905   355 leader_election.cc:288] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.020977 32610 raft_consensus.cc:504] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.004379 32700 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.026307   392 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.033257 32700 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.033555 26330 raft_consensus.cc:2454] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
W20260924 14:32:01.019868 26221 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.038640 26221 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.036161 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.023520 32610 leader_election.cc:288] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.039672 32700 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:01.040995 26221 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.041819 32700 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.043004 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.044282 26076 catalog_manager.cc:5495] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:01.045742 26369 leader_election.cc:302] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.046010 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.046694 26323 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:32:01.047947 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.048056 32610 raft_consensus.cc:2781] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.062250   300 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.075590   300 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.078019   300 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.066745   389 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.037197   392 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.080065   389 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.082227   389 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.068037 32700 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.068822 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2799f93a20d4fd58d617adf2eda3070" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.094916 26256 raft_consensus.cc:2379] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 22: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.094717 32700 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.069674 26406 raft_consensus.cc:3037] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:32:01.095644 32641 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.096992 32700 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.097381 26324 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 25: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:01.075276   397 raft_consensus.cc:504] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.071655 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.099756   397 leader_election.cc:288] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Requested vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.100472 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.072320   385 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.102869   386 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260924 14:32:01.074108 26369 leader_election.cc:334] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.109416 26369 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.109845 26369 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:32:01.110901 26369 leader_election.cc:334] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.063210   390 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.102984 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986260198b71418c831bfc4617da4603" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.111831 26256 raft_consensus.cc:3037] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.113665 32641 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.113750   392 raft_consensus.cc:2726] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:01.114400 32641 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:32:01.114586   392 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.115013 32610 raft_consensus.cc:686] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.115180   392 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.116256 32610 consensus_queue.cc:237] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.118011   392 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.072531   394 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.102449   385 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.106139 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630fa065ede40338b319f0846fdd511" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.126394 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.107049 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed216090b8f4d88a827c18c45fd0422" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.126662 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.127406 26255 raft_consensus.cc:3037] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Advancing to term 12
I20260924 14:32:01.127943 26324 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:32:01.128656 26253 raft_consensus.cc:3037] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.127331 26400 raft_consensus.cc:2379] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 25: Already voted for candidate 45ab97b763804767846e9d27a01b561f in this term.
I20260924 14:32:01.130841   385 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.108362 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80f626c203394b4f8b85ab089d6aa044" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.119935   386 raft_consensus.cc:482] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.133299   386 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.134212   397 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.134864   397 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:01.136523 26295 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.132831 26249 raft_consensus.cc:3037] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:01.137907   385 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:01.138967 26295 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.138749   385 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.142184   385 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.143097 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.143404 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.143967 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.144047 26400 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:32:01.144143 26401 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
W20260924 14:32:01.144415 26369 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.137034   397 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.145018 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.139689   394 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.148200   394 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.152096 26255 raft_consensus.cc:2454] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:32:01.126670 26256 raft_consensus.cc:2454] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:32:01.161181 26253 raft_consensus.cc:2454] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:32:01.160522 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.163096 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.164038   390 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:32:01.164424   390 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.164798   390 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:01.164881   385 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:32:01.165457   385 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.165829   385 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:32:01.169926   386 raft_consensus.cc:504] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.158324 26406 raft_consensus.cc:2454] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 22.
I20260924 14:32:01.171350 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.172039 26324 raft_consensus.cc:3037] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:32:01.174870 26249 raft_consensus.cc:2454] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
W20260924 14:32:01.177150 26221 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.177898 26369 leader_election.cc:334] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.178100 26221 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
W20260924 14:32:01.178514 26369 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.177320 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 76948.6us
I20260924 14:32:01.179723 26224 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.180531 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.184195 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.179965 32700 raft_consensus.cc:2726] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:32:01.176496   386 leader_election.cc:288] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:01.191803 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.193681 26295 leader_election.cc:334] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.189350   389 raft_consensus.cc:2781] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Leader election won for term 22
I20260924 14:32:01.198057 26324 raft_consensus.cc:2454] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:32:01.200631 32641 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:32:01.207551 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.204927   386 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260924 14:32:01.214720 32700 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.217284   392 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:32:01.217468 32700 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.222828 32700 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.224015 32700 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.224591 32700 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.213068 32641 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:01.225412 26369 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.225843 32641 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:32:01.226805 32700 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.233090   390 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.235281   390 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.263571   364 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.264364   364 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:01.268541 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 88097.8us
I20260924 14:32:01.271602 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.272215 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.272655 26324 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:32:01.272957 26406 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 13.
I20260924 14:32:01.268110   364 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.269924   394 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.274056   390 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.274389   394 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.274773   390 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.276650   394 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.240641   389 raft_consensus.cc:686] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:32:01.279574 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 95432.7us
I20260924 14:32:01.280283 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "612b21159cac4f00b88b7076d49d4fec" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.280429 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.280910 26406 raft_consensus.cc:3037] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:01.287591   364 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.287976   389 consensus_queue.cc:237] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.288221   364 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.284307   390 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.290287   364 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.291316   385 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.284437   394 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:32:01.296371 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.298702 26406 raft_consensus.cc:2454] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:32:01.302136 26401 raft_consensus.cc:1261] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:32:01.303862   385 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:01.305557 26369 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.301010 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.300349 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.307170 26406 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:01.308745 32641 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:01.310227 26369 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.311684 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.312554   409 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:32:01.314384   409 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.314939   409 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:32:01.313899 32641 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.312862 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.318683 26221 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.319731 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.299728   394 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.320329   390 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:32:01.321976   385 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260924 14:32:01.323148   390 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.323746   385 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.324164   390 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:32:01.324266   385 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:32:01.322958 32700 raft_consensus.cc:2781] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:32:01.322754   394 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:32:01.322693 26371 leader_election.cc:302] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.321899 26224 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.326841 26371 leader_election.cc:302] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.328069 26371 leader_election.cc:302] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.329857 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.330418   397 raft_consensus.cc:2781] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.330821 26401 raft_consensus.cc:3037] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:32:01.322398 32641 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260924 14:32:01.329155 26295 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.329485   364 raft_consensus.cc:2781] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader election won for term 12
I20260924 14:32:01.341997   410 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260924 14:32:01.344457   397 raft_consensus.cc:686] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.345052 32610 raft_consensus.cc:2781] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.348969   412 raft_consensus.cc:2781] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:32:01.349223 26401 raft_consensus.cc:2454] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
W20260924 14:32:01.360293 26295 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.328485 26221 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.349321 26371 leader_election.cc:302] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.333346 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.362370 26221 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.363165 26221 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.364081 26221 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.365063 26221 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.368654 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 46855.6us
I20260924 14:32:01.369699 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.357582   364 raft_consensus.cc:686] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.372401 26224 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.373693 26369 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:01.368726   355 raft_consensus.cc:2781] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:32:01.372227   386 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:32:01.391474 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.369141   410 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.392479 26324 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 21.
I20260924 14:32:01.396225 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.397186 26324 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 17.
I20260924 14:32:01.399550   418 raft_consensus.cc:2726] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260924 14:32:01.373893 32700 raft_consensus.cc:686] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:01.385341   300 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.372017   389 consensus_queue.cc:1035] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260924 14:32:01.412094   364 consensus_queue.cc:237] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.414002   390 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.423544   355 raft_consensus.cc:686] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.426988   402 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:01.427536   402 raft_consensus.cc:482] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.427886   402 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.428653   385 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.401904   410 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Advancing to term 15
I20260924 14:32:01.431735 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.433691 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898441fbc17a42e09c7639e00843680b" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.436566 26397 raft_consensus.cc:3037] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Advancing to term 14
I20260924 14:32:01.437558 26253 raft_consensus.cc:2379] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 14: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.403614 32610 raft_consensus.cc:686] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.440673   411 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.440932 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.441666 26252 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 15.
I20260924 14:32:01.441412   411 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.442504   300 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.443473 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.444963 32700 consensus_queue.cc:237] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.376078   397 consensus_queue.cc:237] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.450726 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.453092 26371 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.408449   394 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.457998   417 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.458601   417 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.460819   417 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:01.463053 23135 thread.cc:536] Waited for 1000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:01.467762 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:01.468110 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83a2b98291f41d7beed3c1329ba4bc9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.468782 26330 raft_consensus.cc:3037] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.431915   409 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.468672 32610 consensus_queue.cc:237] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.473069   414 raft_consensus.cc:2781] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
I20260924 14:32:01.473296   417 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.473860   414 raft_consensus.cc:686] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:01.473944   417 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.478574 26330 raft_consensus.cc:2454] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 20.
I20260924 14:32:01.479650 26397 raft_consensus.cc:2454] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 14.
I20260924 14:32:01.485493   300 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.426801   355 consensus_queue.cc:237] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:01.486655   413 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260924 14:32:01.475972   390 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.485575   420 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:32:01.455466   412 raft_consensus.cc:686] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:01.488591   402 raft_consensus.cc:504] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.500771   402 leader_election.cc:288] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.484524   385 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:01.509964 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 53258.6us
I20260924 14:32:01.510942   412 consensus_queue.cc:237] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.511432 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:01.517057 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:32:01.534658 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 57806.6us
I20260924 14:32:01.538759 26369 leader_election.cc:302] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.528335   424 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260924 14:32:01.535549 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 57827.1us
I20260924 14:32:01.475102   414 consensus_queue.cc:237] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.544821   398 raft_consensus.cc:2781] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.554733 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.555881 26250 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
I20260924 14:32:01.557744 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.558058 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9812df92004da0ba2c451e7b423e66" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.558957 26256 raft_consensus.cc:3037] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.559772 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.560860 26252 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 15: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.566293 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.567255 26253 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:32:01.569412 26256 raft_consensus.cc:2454] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
I20260924 14:32:01.545419   398 raft_consensus.cc:686] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:01.539558   385 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:32:01.529186   417 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:01.583387 26221 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.547075   424 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:01.584827 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:01.587855 26221 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.589784 26221 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.575358   385 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
W20260924 14:32:01.591310 26221 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.592039 26221 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.596292   415 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.596886   415 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.597057   410 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.599107   415 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.591156   385 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:01.599349   410 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:01.623049 26295 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.604303   398 consensus_queue.cc:237] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:01.626360 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:01.628916 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.607369 26221 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.612394   394 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.614166   411 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.587826   424 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Advancing to term 22
I20260924 14:32:01.584931   409 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:01.632300 26221 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.644552 26221 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.645190 26221 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.646124 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 103337us
W20260924 14:32:01.648065 26221 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.648628 26221 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.649220 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59f0af5dd6048918c6579b295ac5510" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.649847 26406 raft_consensus.cc:3037] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.650377 26397 raft_consensus.cc:1261] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:32:01.651396   409 raft_consensus.cc:2726] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:01.653224 26224 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.655225 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:01.655958 26402 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 20.
I20260924 14:32:01.659488 26406 raft_consensus.cc:2454] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
I20260924 14:32:01.662627 26324 raft_consensus.cc:1261] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260924 14:32:01.664011   364 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.664610   364 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.666741   364 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.667117   415 raft_consensus.cc:2726] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:32:01.668816   410 raft_consensus.cc:2781] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Leader election won for term 14
I20260924 14:32:01.669520   410 raft_consensus.cc:686] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:32:01.670600 26295 leader_election.cc:334] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.671860 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:01.676159   410 consensus_queue.cc:237] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.679589 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.680912 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.691607 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.692371 26256 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:01.693045 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.693892 26253 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 19: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.695822 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.696878 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.697619 26330 raft_consensus.cc:2379] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 19: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:01.696527 26323 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 19.
W20260924 14:32:01.698347 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:01.698917 26221 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.699412 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:01.700100   385 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:01.702314 26221 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.704516   398 consensus_queue.cc:1035] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260924 14:32:01.705647 26295 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.706233 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:01.714967   424 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.719146 26250 raft_consensus.cc:2379] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 22: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:01.719595   424 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.722683   385 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:01.687469   411 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:32:01.724334   411 raft_consensus.cc:482] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.724711   411 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:01.731149 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.689603   417 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:32:01.703650   394 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:32:01.734416 26297 leader_election.cc:302] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.736569 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.738353 26324 raft_consensus.cc:2379] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 15: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:01.738499   416 raft_consensus.cc:2781] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.732851   417 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.740063   416 raft_consensus.cc:686] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:01.741070   416 consensus_queue.cc:237] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.739203   417 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:32:01.737182   300 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:32:01.743916   300 raft_consensus.cc:482] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.744424   300 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:32:01.747885 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:01.750790 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16dc088d2abf460eb278e44fc4f60620" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:01.752182 26327 raft_consensus.cc:3037] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Advancing to term 22
W20260924 14:32:01.751646 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.753007 26221 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.753710 26221 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.754400 26221 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.751036 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce0c6b37e04bf89405065d71329eb8" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:32:01.754851 26221 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.755148 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
W20260924 14:32:01.755332 26221 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.755820 26330 raft_consensus.cc:3037] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 14 FOLLOWER]: Advancing to term 15
W20260924 14:32:01.756575   429 log.cc:924] Time spent T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.002s
I20260924 14:32:01.760723 26327 raft_consensus.cc:2454] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 22.
I20260924 14:32:01.763106 26330 raft_consensus.cc:2454] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 15.
I20260924 14:32:01.755148 26325 raft_consensus.cc:3037] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:32:01.754552   427 log.cc:924] Time spent T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.117s	user 0.000s	sys 0.002s
I20260924 14:32:01.766152   411 raft_consensus.cc:504] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:01.770229 26221 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.821354 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 51217.4us
W20260924 14:32:01.822006 26221 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.772378 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.823498 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51234.9us
I20260924 14:32:01.824512 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.784081 26369 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.786038 26402 raft_consensus.cc:1261] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260924 14:32:01.826316 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:01.789161   300 raft_consensus.cc:504] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.793896 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.797931   402 consensus_queue.cc:1035] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:32:01.798803 26298 leader_election.cc:302] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.839823 26221 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:01.811407   417 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.829803 26297 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:01.839069 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 45265.5us
I20260924 14:32:01.844334 26371 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.845232 32610 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260924 14:32:01.845912 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.846635   418 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260924 14:32:01.841104 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.852320 26323 raft_consensus.cc:2379] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 16: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:01.842021 32641 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:32:01.839824   394 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:01.853137   405 raft_consensus.cc:2781] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:01.853322 32641 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:32:01.853382   394 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.853770   405 raft_consensus.cc:686] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:01.854007   394 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.854404   417 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.856659   424 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:01.857138   424 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.857079 26221 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:01.857666   424 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:01.858359 26221 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.860273   385 raft_consensus.cc:2726] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260924 14:32:01.860725   389 raft_consensus.cc:2781] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Leader election won for term 22
I20260924 14:32:01.854768   405 consensus_queue.cc:237] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:01.861299   389 raft_consensus.cc:686] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:01.862221   389 consensus_queue.cc:237] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.863535   432 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:32:01.863991   432 raft_consensus.cc:482] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.864362   432 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:01.867851 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:01.867898   410 consensus_queue.cc:1035] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260924 14:32:01.868736 26249 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 12.
I20260924 14:32:01.872260 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.872294 26256 raft_consensus.cc:1261] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260924 14:32:01.874250 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188e3a6a42340858d5f3ada3f5e867d" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:01.874840 26402 raft_consensus.cc:3037] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Advancing to term 21
I20260924 14:32:01.883654   423 consensus_queue.cc:1035] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260924 14:32:01.889541   300 leader_election.cc:288] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.849936   409 raft_consensus.cc:2726] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260924 14:32:01.904062 26256 raft_consensus.cc:1261] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260924 14:32:01.868482   411 leader_election.cc:288] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.899158 26325 raft_consensus.cc:2454] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 18.
I20260924 14:32:01.879388 32610 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260924 14:32:01.922544 32610 raft_consensus.cc:482] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e09fbebe29a49d291d23f9d395fe335)
I20260924 14:32:01.923015 32610 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Advancing to term 13
W20260924 14:32:01.915375 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:01.924742 26295 leader_election.cc:334] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.926271 26221 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:01.926689 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 85844.2us
W20260924 14:32:01.927145 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:32:01.930403 26250 raft_consensus.cc:1261] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260924 14:32:01.951166 26401 raft_consensus.cc:1261] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260924 14:32:01.956528 26221 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.952899 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda86ff6977641a8af47dad8d1d1a049" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.957232 26253 raft_consensus.cc:3037] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:32:01.958325 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e912296c0f6f4c4d89bbec1581a46cf9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:01.958624   419 consensus_queue.cc:1035] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260924 14:32:01.959019 26250 raft_consensus.cc:3037] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:01.957835 26221 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:01.966897 26221 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:01.967823   419 raft_consensus.cc:2781] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election won for term 18
W20260924 14:32:01.969823 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:01.977713   419 raft_consensus.cc:686] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
W20260924 14:32:01.967865 26295 leader_election.cc:334] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:01.980724   414 raft_consensus.cc:2781] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader election won for term 15
W20260924 14:32:01.980908 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:01.981389   414 raft_consensus.cc:686] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:01.978775   419 consensus_queue.cc:237] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.982370   414 consensus_queue.cc:237] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.977435   432 raft_consensus.cc:504] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:01.973618   411 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:01.984776 26076 catalog_manager.cc:5495] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 20 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:01.990886 26402 raft_consensus.cc:2454] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 21.
I20260924 14:32:01.951038   392 consensus_queue.cc:1035] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260924 14:32:01.993477   432 leader_election.cc:288] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:01.994017 26255 raft_consensus.cc:1261] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260924 14:32:01.994107   411 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:01.995572 26400 raft_consensus.cc:1261] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260924 14:32:01.996492   411 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:01.990734 26250 raft_consensus.cc:2454] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:32:02.000946   390 raft_consensus.cc:2726] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:32:02.002291 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.003480 26297 leader_election.cc:302] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:02.008024 26256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575580f9ac4403b5f531f616fe41fd" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:02.008687 26256 raft_consensus.cc:3037] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:02.009546 26224 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:02.010737   409 raft_consensus.cc:2781] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:32:02.011267   409 raft_consensus.cc:686] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 21 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:02.011981 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:02.016705 26253 raft_consensus.cc:2454] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:02.025307 26297 leader_election.cc:302] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:02.017272 26256 raft_consensus.cc:2454] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 16.
I20260924 14:32:01.939575   423 consensus_queue.cc:1035] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260924 14:32:02.029778 26255 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
W20260924 14:32:02.107524 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 76569.8us
W20260924 14:32:02.107916   439 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260924 14:32:02.021855   416 raft_consensus.cc:2781] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:32:01.990442   424 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:02.108915   300 raft_consensus.cc:2781] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:32:02.151516 26075 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 144) took 3428 ms. Trace:
I20260924 14:32:02.105764   364 consensus_queue.cc:1035] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260924 14:32:02.012212   409 consensus_queue.cc:237] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:02.110936   410 consensus_queue.cc:1035] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260924 14:32:02.164647 26371 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 51023.4us
I20260924 14:32:02.177604 26371 leader_election.cc:302] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:02.158726   300 raft_consensus.cc:686] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 19 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:02.152206 26075 rpcz_store.cc:276] 0924 14:31:58.717069 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:58.717319 (+   250us) service_pool.cc:227] Handling call
0924 14:32:02.144845 (+3427526us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0924 14:32:00.977064 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0924 14:32:00.977713 (+   649us) write_op.cc:432] Acquiring schema lock in shared mode
0924 14:32:00.977770 (+    57us) write_op.cc:435] Acquired schema lock
0924 14:32:00.977785 (+    15us) tablet.cc:660] Decoding operations
0924 14:32:00.978913 (+  1128us) write_op.cc:620] Acquiring the partition lock for write op
0924 14:32:00.979016 (+   103us) write_op.cc:641] Partition lock acquired for write op
0924 14:32:00.979054 (+    38us) tablet.cc:683] Acquiring locks for 12 operations
0924 14:32:01.008892 (+ 29838us) tablet.cc:699] Row locks acquired
0924 14:32:01.008942 (+    50us) write_op.cc:260] PREPARE: finished
0924 14:32:01.009162 (+   220us) write_op.cc:270] Start()
0924 14:32:01.009362 (+   200us) write_op.cc:276] Timestamp: P: 1790260321009082 usec, L: 0
0924 14:32:01.009410 (+    48us) op_driver.cc:331] REPLICATION: starting
0924 14:32:01.010175 (+   765us) log.cc:842] Serialized 4845 byte log entry
0924 14:32:01.071123 (+ 60948us) op_driver.cc:444] REPLICATION: finished
0924 14:32:01.235792 (+164669us) write_op.cc:301] APPLY: starting
0924 14:32:01.235953 (+   161us) tablet.cc:1365] starting BulkCheckPresence
0924 14:32:01.317616 (+ 81663us) tablet.cc:1368] finished BulkCheckPresence
0924 14:32:01.317767 (+   151us) tablet.cc:1370] starting ApplyRowOperation cycle
0924 14:32:01.546063 (+228296us) tablet.cc:1381] finished ApplyRowOperation cycle
0924 14:32:01.590220 (+ 44157us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0924 14:32:01.606555 (+ 16335us) write_op.cc:312] APPLY: finished
0924 14:32:01.606869 (+   314us) log.cc:842] Serialized 112 byte log entry
0924 14:32:01.607062 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:01.607208 (+   146us) write_op.cc:454] Released schema lock
0924 14:32:01.607272 (+    64us) write_op.cc:341] FINISH: Updating metrics
0924 14:32:01.627912 (+ 20640us) write_op.cc:489] Releasing partition, row and schema locks
0924 14:32:01.627946 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":25333,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":148937,"apply.run_cpu_time_us":22080,"apply.run_wall_time_us":392654,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54335,"num_ops":12,"prepare.queue_time_us":39737,"prepare.run_cpu_time_us":5172,"prepare.run_wall_time_us":50837,"replication_time_us":61607,"thread_start_us":136104,"threads_started":2}]]}
I20260924 14:32:02.221138   418 raft_consensus.cc:2781] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader election won for term 16
W20260924 14:32:02.243227   448 log.cc:924] Time spent T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260924 14:32:02.242966   300 consensus_queue.cc:237] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:02.244262 26077 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55204 (request call id 144) took 1630 ms. Trace:
I20260924 14:32:02.244692 26077 rpcz_store.cc:276] 0924 14:32:00.514938 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:32:00.515386 (+   448us) service_pool.cc:227] Handling call
0924 14:32:02.127010 (+1611624us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":115670,"reported_tablets":37}
W20260924 14:32:02.246816   443 log.cc:924] Time spent T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260924 14:32:02.247130   446 log.cc:924] Time spent T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
W20260924 14:32:02.243302 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:02.191566   394 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:02.179924 32610 raft_consensus.cc:504] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:02.273703 32610 leader_election.cc:288] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.268828   394 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:02.270637   424 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:02.295598 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.297668 26295 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.304244 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.240442   416 raft_consensus.cc:686] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:02.329270   418 raft_consensus.cc:686] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:02.383006   418 consensus_queue.cc:237] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:02.399793 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:02.403151 26324 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c22052275034c369b7f7dca086fbd1e in current term 20: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:02.404448 26406 raft_consensus.cc:1261] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260924 14:32:02.407279 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:02.415755 26323 raft_consensus.cc:1261] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:32:02.420172 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:02.421777 26249 raft_consensus.cc:1261] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260924 14:32:02.426008 26255 raft_consensus.cc:2379] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f59f3234edf84aab852fa989fb754d31 in current term 20: Already voted for candidate 6c22052275034c369b7f7dca086fbd1e in this term.
I20260924 14:32:02.429406   432 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:02.429986   432 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:02.432284   432 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.432987   300 consensus_queue.cc:1035] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:32:02.437059   456 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:32:02.440179   456 raft_consensus.cc:482] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:02.440546   456 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
W20260924 14:32:02.442579 26369 leader_election.cc:334] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.443853 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.443863 26400 raft_consensus.cc:1261] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260924 14:32:02.391150   416 consensus_queue.cc:237] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:02.448697 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.451459 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00361946476c4fecbd244a86b9cfb752" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
W20260924 14:32:02.453089 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:02.449297 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.463510 23135 thread.cc:536] Waited for 2000ms trying to join with diag-logger (tid 27219)
I20260924 14:32:02.464501 26249 raft_consensus.cc:3037] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Advancing to term 13
I20260924 14:32:02.480007   455 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:02.480724   455 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:02.483024   455 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.490808 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:02.491966 26400 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 17.
I20260924 14:32:02.496596   456 raft_consensus.cc:504] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:02.505894   456 leader_election.cc:288] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.509893 26327 raft_consensus.cc:1261] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260924 14:32:02.510152   409 consensus_queue.cc:1035] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
W20260924 14:32:02.513281 26295 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 48305.9us
I20260924 14:32:02.524785 26249 raft_consensus.cc:2454] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ab97b763804767846e9d27a01b561f in term 13.
I20260924 14:32:02.513005 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:02.534961 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:02.535681   456 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260924 14:32:02.536377   456 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:02.536882   456 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:02.553115 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:32:02.562103 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:02.574232 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.574781 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.575203 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.575654 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:02.577639 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:02.559788   455 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:32:02.584174 26371 leader_election.cc:302] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e; no voters: 
I20260924 14:32:02.587594   434 consensus_queue.cc:1035] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260924 14:32:02.541581   411 consensus_queue.cc:1035] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260924 14:32:02.591697 26369 leader_election.cc:334] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.595685 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:32:02.580576   412 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260924 14:32:02.600718   465 raft_consensus.cc:2781] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Leader election won for term 13
I20260924 14:32:02.601356   465 raft_consensus.cc:686] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Becoming Leader. State: Replica: 45ab97b763804767846e9d27a01b561f, State: Running, Role: LEADER
I20260924 14:32:02.621197   414 consensus_queue.cc:1035] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260924 14:32:02.623528 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:02.624164 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:02.624738 26295 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.625720 26295 leader_election.cc:334] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:02.628700   465 consensus_queue.cc:237] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:02.642706 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:02.669487 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78540b1453ae42188ee3fcd5d9d4cb94" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:02.670413 26250 raft_consensus.cc:3037] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:02.719270   456 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:02.721029 26324 raft_consensus.cc:2379] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45ab97b763804767846e9d27a01b561f in current term 18: Already voted for candidate f59f3234edf84aab852fa989fb754d31 in this term.
I20260924 14:32:02.730103 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f9b654ec4542a781bdc872c6ddbdce" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:02.730754 26406 raft_consensus.cc:3037] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Advancing to term 18
I20260924 14:32:02.731269 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:02.744012 26406 raft_consensus.cc:2454] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:32:02.750885 26250 raft_consensus.cc:2454] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:32:02.762773 26298 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:02.749902 26076 catalog_manager.cc:5495] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 12, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 12 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:02.762745   434 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260924 14:32:02.771481 26076 catalog_manager.cc:5495] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 20 to 21, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 21 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:02.778179 26323 raft_consensus.cc:1261] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260924 14:32:02.784958 26327 raft_consensus.cc:1261] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260924 14:32:02.789968 26076 catalog_manager.cc:5495] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 12 to 13, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 13 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:02.792265 26076 catalog_manager.cc:5495] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 15 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 16 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } }
I20260924 14:32:02.768452   456 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.773393   394 raft_consensus.cc:2781] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:32:02.835887 26076 catalog_manager.cc:5495] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 21 to 22, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 22 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:02.846752 26076 catalog_manager.cc:5495] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 14 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 15 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:02.842895   406 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:02.872077   406 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:02.874310   406 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.875733 26400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:02.876663 26400 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 25.
I20260924 14:32:02.879261 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:32:02.880046 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.880461   416 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260924 14:32:02.880872   416 raft_consensus.cc:482] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:02.881207   416 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 25 FOLLOWER]: Advancing to term 26
W20260924 14:32:02.888998 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:02.850550   394 raft_consensus.cc:686] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:02.858194   390 consensus_queue.cc:1035] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260924 14:32:02.866889   385 consensus_queue.cc:1035] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260924 14:32:02.854069 26076 catalog_manager.cc:5495] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 17 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 18 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:02.901979 26076 catalog_manager.cc:5495] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 14 to 15, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 15 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:02.903319 26076 catalog_manager.cc:5495] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 13 to 14, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 14 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:02.905452 26076 catalog_manager.cc:5495] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 21 to 22, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 22 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:02.907883 26076 catalog_manager.cc:5495] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 17 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 18 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:02.909447 26076 catalog_manager.cc:5495] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 11 to 12, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 12 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } }
W20260924 14:32:02.912650 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 59760.2us
I20260924 14:32:02.922034 26297 leader_election.cc:302] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
W20260924 14:32:02.930655 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.936167 26249 raft_consensus.cc:1261] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260924 14:32:02.923996 26081 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 9) took 19322 ms (client timeout 15000 ms). Trace:
W20260924 14:32:02.938017 26081 rpcz_store.cc:269] 0924 14:31:43.571533 (+     0us) service_pool.cc:170] Inserting onto call queue
0924 14:31:43.572537 (+  1004us) service_pool.cc:227] Handling call
0924 14:32:02.888542 (+19316005us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":197489,"reported_tablets":45}
I20260924 14:32:02.942670   394 consensus_queue.cc:237] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:02.945235 26295 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.947247 26067 connection.cc:441] server connection from 127.0.0.1:34948 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34948 (request call id 9) could send its response
I20260924 14:32:02.939678   406 raft_consensus.cc:2781] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader election won for term 18
I20260924 14:32:02.950393   406 raft_consensus.cc:686] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:02.951561   406 consensus_queue.cc:237] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:02.962169 26295 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:02.960335 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:02.978392 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:02.959376   416 raft_consensus.cc:504] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:02.982180   487 log.cc:924] Time spent T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260924 14:32:02.983846   416 leader_election.cc:288] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 election: Requested vote from peers 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:02.992311 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951fc36414c641b7a4bf7b6b3e6dc3f2" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:02.993005 26406 raft_consensus.cc:3037] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 25 FOLLOWER]: Advancing to term 26
W20260924 14:32:02.998919   481 log.cc:924] Time spent T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260924 14:32:03.007160 26406 raft_consensus.cc:2454] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 26.
W20260924 14:32:03.002041 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:03.013666 26298 leader_election.cc:302] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:03.025475 26249 raft_consensus.cc:1261] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260924 14:32:03.029346   416 raft_consensus.cc:2781] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 26 FOLLOWER]: Leader election won for term 26
W20260924 14:32:03.030187 26295 leader_election.cc:334] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:03.042912   480 log.cc:924] Time spent T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: Append to log took a long time: real 0.190s	user 0.001s	sys 0.002s
I20260924 14:32:03.031145   416 raft_consensus.cc:686] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 26 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
W20260924 14:32:03.049203 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:03.075342   416 consensus_queue.cc:237] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
W20260924 14:32:03.082986 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:03.084635 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:03.087980 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 58521.7us
W20260924 14:32:03.085358 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260924 14:32:03.119264 26400 raft_consensus.cc:1261] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260924 14:32:03.133720 32700 consensus_queue.cc:1035] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260924 14:32:03.136433   386 consensus_queue.cc:1035] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260924 14:32:03.145520   392 consensus_queue.cc:1035] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 14:32:03.154513 26256 raft_consensus.cc:1261] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260924 14:32:03.181744 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:03.192189 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:03.220667   386 consensus_queue.cc:1035] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260924 14:32:03.241282 26075 catalog_manager.cc:5495] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 20 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.243870 26075 catalog_manager.cc:5495] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 20 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.255511 26256 raft_consensus.cc:1261] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Refusing update from remote peer 45ab97b763804767846e9d27a01b561f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260924 14:32:03.266280 26075 catalog_manager.cc:5495] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 20 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.268520 26075 catalog_manager.cc:5495] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 20 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.274857 26075 catalog_manager.cc:5495] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 15 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 16 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.303570   398 consensus_queue.cc:1035] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
W20260924 14:32:03.347571 26295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111) [suppressed 298 similar messages]
W20260924 14:32:03.359534 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260924 14:32:03.362067 26249 raft_consensus.cc:1261] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260924 14:32:03.365201   461 consensus_queue.cc:1035] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260924 14:32:03.378485 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:03.396796 26077 catalog_manager.cc:5495] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 19 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 20 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:03.398763 26077 catalog_manager.cc:5495] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 15 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:03.400676 26077 catalog_manager.cc:5495] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 18 to 19, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 19 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
W20260924 14:32:03.464442 23135 thread.cc:536] Waited for 3000ms trying to join with diag-logger (tid 27219)
I20260924 14:32:03.487255 26400 raft_consensus.cc:1261] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 26 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260924 14:32:03.490865   461 consensus_queue.cc:1035] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
W20260924 14:32:03.497176 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:03.501318 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:03.518544 26397 raft_consensus.cc:1261] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260924 14:32:03.539119 26076 catalog_manager.cc:5495] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 12 to 13, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 45ab97b763804767846e9d27a01b561f (127.22.151.196). New cstate: current_term: 13 leader_uuid: "45ab97b763804767846e9d27a01b561f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:32:03.546667   406 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:03.547264   406 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:03.549454   406 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:03.544971   461 consensus_queue.cc:1035] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260924 14:32:03.565034 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:03.565819 26250 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 15.
W20260924 14:32:03.568387 26295 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:03.572075 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:03.573122   461 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260924 14:32:03.573525   461 raft_consensus.cc:482] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:03.573858   461 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Advancing to term 16
W20260924 14:32:03.604069 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:03.610446 26077 catalog_manager.cc:5495] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 15 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 18 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } }
I20260924 14:32:03.630054   461 raft_consensus.cc:504] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:03.645905   461 leader_election.cc:288] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:03.662119 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0b4058f18400dab940d3832586746" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:03.662894 26250 raft_consensus.cc:3037] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Advancing to term 16
I20260924 14:32:03.654438 26076 catalog_manager.cc:5495] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 24 to 26, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 26 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
I20260924 14:32:03.672518 26076 catalog_manager.cc:5495] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f reported cstate change: term changed from 16 to 18, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 18 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } }
W20260924 14:32:03.694213 26295 leader_election.cc:334] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:03.703424   394 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:03.688237 26250 raft_consensus.cc:2454] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 16.
I20260924 14:32:03.704123   394 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:03.705986   394 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:03.711321 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:03.772408 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:03.773126 26250 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:03.774356 26297 leader_election.cc:302] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:03.775310   416 raft_consensus.cc:2781] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader election won for term 16
I20260924 14:32:03.775908   416 raft_consensus.cc:686] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:03.776888   416 consensus_queue.cc:237] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:03.786785 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260924 14:32:03.833629 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:03.834873   416 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:03.835280   416 raft_consensus.cc:482] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:03.835727   416 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:03.885890 26075 catalog_manager.cc:5495] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 14 to 16, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 16 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:03.913276   416 raft_consensus.cc:504] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:03.927506   416 leader_election.cc:288] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:03.938515 26295 leader_election.cc:334] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:03.939975 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcac7743e33046019152a570ae473ca3" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e"
I20260924 14:32:03.940824 26255 raft_consensus.cc:3037] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:03.972628 26255 raft_consensus.cc:2454] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
W20260924 14:32:04.013576 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:04.016120 26297 leader_election.cc:302] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:04.017671   461 raft_consensus.cc:2781] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:04.018180   461 raft_consensus.cc:686] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:04.019066   461 consensus_queue.cc:237] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:04.063398 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:32:04.062372 26075 catalog_manager.cc:5495] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 18 to 20, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 20 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } }
W20260924 14:32:04.084978 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:04.096690 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.100103 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:32:04.206463 32700 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:04.207029 32700 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:04.209573 32700 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:04.212435 26325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:04.213238 26325 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 20.
I20260924 14:32:04.215767 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 
I20260924 14:32:04.218844   414 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260924 14:32:04.219266   414 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:04.219659   414 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:32:04.252046 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:04.257329   414 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:04.266438   414 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: Requested vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:04.273851 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "168958697ba24715a127c9c7b7c520ff" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31"
I20260924 14:32:04.274566 26324 raft_consensus.cc:3037] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Advancing to term 21
W20260924 14:32:04.295336 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.299718 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:04.303277 26324 raft_consensus.cc:2454] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c22052275034c369b7f7dca086fbd1e in term 21.
W20260924 14:32:04.313498 26221 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:04.314790 26221 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31; no voters: 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:04.316524   485 raft_consensus.cc:2781] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Leader election won for term 21
I20260924 14:32:04.317181   485 raft_consensus.cc:686] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 21 LEADER]: Becoming Leader. State: Replica: 6c22052275034c369b7f7dca086fbd1e, State: Running, Role: LEADER
I20260924 14:32:04.318157   485 consensus_queue.cc:237] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:04.338184 26075 catalog_manager.cc:5495] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e reported cstate change: term changed from 19 to 21, leader changed from 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193) to 6c22052275034c369b7f7dca086fbd1e (127.22.151.194). New cstate: current_term: 21 leader_uuid: "6c22052275034c369b7f7dca086fbd1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:04.350034 26255 raft_consensus.cc:1261] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260924 14:32:04.352231   461 consensus_queue.cc:1035] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260924 14:32:04.395020 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.396862 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:04.461097 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.465000 23135 thread.cc:536] Waited for 4000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:04.477735 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.551044 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:04.582283 26255 raft_consensus.cc:1261] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Refusing update from remote peer f59f3234edf84aab852fa989fb754d31: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260924 14:32:04.586233 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:04.589020 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:04.591241   521 consensus_queue.cc:1035] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260924 14:32:04.664917 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.743701 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260924 14:32:04.858108 26325 raft_consensus.cc:1261] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Refusing update from remote peer 6c22052275034c369b7f7dca086fbd1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260924 14:32:04.878567 32700 consensus_queue.cc:1035] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260924 14:32:04.886654 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.886701 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.934119 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:04.938823 26221 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:04.957712 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.961930 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:04.999856 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.043694 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:05.085246 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:05.127848 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.147665 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.213132 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.251555 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:05.254011 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:05.380529 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.423321 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.465592 23135 thread.cc:536] Waited for 5000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:05.474762 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:05.618351 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.622720 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.660635 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.663894 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.696477 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.752954 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:05.753487 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:05.777654 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:05.877118 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:05.909227 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:06.118983 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:06.188231 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:06.358372 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:06.466147 23135 thread.cc:536] Waited for 6000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:06.537040 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.539530 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.579761 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.665647 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.679105 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.764724 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:06.843623 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:06.882862 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:06.923712 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.981437 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:06.985966 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:07.152168 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:07.161672 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.189718 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.274817 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:07.298156 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.308178 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:07.326917 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.380906 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:07.395241 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:07.466741 23135 thread.cc:536] Waited for 7000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:07.593981 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:07.601353 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.608397 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.633653 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.648619 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:07.649770 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.667253 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.667994 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:07.752130 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.779520 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:07.816569 26221 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:07.930763 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.042743 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.059064 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.089241 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.106432 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.160486 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:08.234573 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.292104 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.359018 26295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111) [suppressed 451 similar messages]
W20260924 14:32:08.450655 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:08.467319 23135 thread.cc:536] Waited for 8000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:08.531231 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:08.557864 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:08.649145 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:08.800710 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:08.920392 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:08.961392 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:08.998746 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.079344 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 43516.1us
W20260924 14:32:09.122993 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.165695 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.196924 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.258316 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:09.265825 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:09.342201 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.365617 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:09.468688 23135 thread.cc:536] Waited for 9000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:09.533756 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.559929 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:09.702145 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.761616 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:09.784289 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:09.875020 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:09.938082 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:09.994871 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 56829.5us
W20260924 14:32:09.986171 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:10.017674 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:10.121649 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:10.134660 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.136168 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.191007 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.202419 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.207734 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:10.231009 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:10.241228 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.277853 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.278766 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.306414 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.439465 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.454231 26221 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260924 14:32:10.469246 23135 thread.cc:536] Waited for 10000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:10.535307 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.537303 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.548631 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.622416 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.803813 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 46896.4us
W20260924 14:32:10.816978 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.817703 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.818161 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:10.934212 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:11.000535 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:11.026770 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.027469 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.214046 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:11.270044 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:11.316033 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:11.346325 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:11.469996 23135 thread.cc:536] Waited for 11000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:11.639575 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:11.662381 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:11.668916 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.690331 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:11.744500 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:11.865516 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.895305 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.921384 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:11.990017 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.089957 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.127990 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:12.156176 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.186362 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.232553 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:12.306815 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:12.382205 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:12.385437 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:12.470602 23135 thread.cc:536] Waited for 12000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:12.527870 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.530433 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.590946 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:12.621915 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.641074 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.721047 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.737201 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.750085 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.779796 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.785885 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:45507, user_credentials={real_user=slave}} blocked reactor thread for 44812.1us
W20260924 14:32:12.788272 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:12.890061 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.922580 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:12.930160 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:12.977823 26221 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260924 14:32:13.011983 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.047322 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.194326 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.216627 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.218266 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.359862 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.366847 26369 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111) [suppressed 449 similar messages]
W20260924 14:32:13.371505 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:13.471166 23135 thread.cc:536] Waited for 13000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:13.540091 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:13.586851 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:13.597900 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:13.628669 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:13.656071 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:13.806099 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:13.861379 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.076220 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.083030 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.085004 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260924 14:32:14.195957 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.230487 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.362087 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.366910 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.432564 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.461149 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:14.471807 23135 thread.cc:536] Waited for 14000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:14.512750 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.575033 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.695355 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.701464 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.705886 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:14.889232 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:14.909308 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260924 14:32:14.994355 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.007460 26221 consensus_peers.cc:487] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.009084 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.046173 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.089067 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.110836 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:15.169303 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:15.223198 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.283408 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.300957 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:15.306639 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.308439 26221 consensus_peers.cc:487] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.320853 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.461051 26297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:46433, user_credentials={real_user=slave}} blocked reactor thread for 104636us
W20260924 14:32:15.472361 23135 thread.cc:536] Waited for 15000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:15.541515 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.544548 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:15.618414 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260924 14:32:16.416213   552 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.418824   529 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.418464   552 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:16.419503   529 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:16.421555   552 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:32:16.421720   529 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:16.444757   552 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.449097   552 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:16.451212   552 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:16.473031 23135 thread.cc:536] Waited for 16000ms trying to join with diag-logger (tid 27219)
I20260924 14:32:16.475121 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:32:16.541529 26224 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 1.02258e+06us
W20260924 14:32:16.544014 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260924 14:32:16.544991 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260924 14:32:16.546465 26295 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:16.547555 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260924 14:32:16.548591   535 raft_consensus.cc:482] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 45ab97b763804767846e9d27a01b561f)
I20260924 14:32:16.551819   535 raft_consensus.cc:504] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:16.552901 26250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:16.554500 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968ded8aff90408e9baafab16ae17e14" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:16.560379 26251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a92b51c57b441158371bdfdf2d193c1" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:16.562561   535 leader_election.cc:288] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:32:16.568228   549 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.568873   549 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:16.571384   549 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:16.579276   549 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:32:16.548743 26295 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:16.579874   549 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:16.582037   549 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:16.595162 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 105609us
I20260924 14:32:16.597663   549 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.598448   555 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:16.599061   555 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
W20260924 14:32:16.600780 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:16.601250   555 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:16.598315   549 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:16.605435   549 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:16.602563 26369 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 94217.8us
I20260924 14:32:16.612133 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:16.617640 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
I20260924 14:32:16.620920 26297 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:32:16.621950   557 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:16.624109 26297 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:16.624972   557 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260924 14:32:16.633728 26297 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:16.634629   555 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:32:16.631276 26295 leader_election.cc:334] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:16.637068 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:16.645599 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fae41ae7fca440da0de40b0ee6973c7" candidate_uuid: "6c22052275034c369b7f7dca086fbd1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
W20260924 14:32:16.652107 26279 debug-util.cc:398] Leaking SignalData structure 0x7b0800143f40 after lost signal to thread 23138
W20260924 14:32:16.653436 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08001b3b20 after lost signal to thread 26115
W20260924 14:32:16.654861 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08001ae1a0 after lost signal to thread 26282
W20260924 14:32:16.658756 26279 debug-util.cc:398] Leaking SignalData structure 0x7b0800039440 after lost signal to thread 26356
W20260924 14:32:16.660147 26279 debug-util.cc:398] Leaking SignalData structure 0x7b08001ff200 after lost signal to thread 26430
I20260924 14:32:16.669009 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ddc9746f0a41f08d38d9ab46bac5ba" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:32:16.672854 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260924 14:32:16.674644 26253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eec7b6dbbd94479aa179fa23fc1f403" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:16.675246 26258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
I20260924 14:32:16.676157 26255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81d1761568f40418d9259b74501d677" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c22052275034c369b7f7dca086fbd1e" is_pre_election: true
W20260924 14:32:16.681625 26221 consensus_peers.cc:487] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260924 14:32:16.683204 26224 leader_election.cc:302] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c22052275034c369b7f7dca086fbd1e; no voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:16.684299   560 raft_consensus.cc:2726] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:32:16.696200 26369 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:16.696887 26369 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:16.701152 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:16.702467   545 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:16.703783 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:16.704640   549 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:16.705372 26371 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:16.706161   549 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:16.706959 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:16.707993   549 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260924 14:32:16.708837 26221 consensus_peers.cc:487] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260924 14:32:16.709681 23140 debug-util.cc:398] Leaking SignalData structure 0x7b080015c8a0 after lost signal to thread 26224
W20260924 14:32:16.709439 26221 consensus_peers.cc:487] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:16.853425 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:16.890489 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:16.958623 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:17.027992 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260924 14:32:17.123510 26221 consensus_peers.cc:487] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:17.154446 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:17.341912 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:17.430331 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:17.473570 23135 thread.cc:536] Waited for 17000ms trying to join with diag-logger (tid 27219)
W20260924 14:32:17.559871 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:17.593525 26221 consensus_peers.cc:487] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:17.611068 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:17.621913 26221 consensus_peers.cc:487] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260924 14:32:17.668937 23135 tablet_server.cc:194] TabletServer@127.22.151.193:38943 shutdown complete.
W20260924 14:32:17.772559 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260924 14:32:17.947808 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.014549 26221 consensus_peers.cc:487] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:18.078135 26221 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:38943, user_credentials={real_user=slave}} blocked reactor thread for 63683.7us
W20260924 14:32:18.074388 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260924 14:32:18.079051 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.053593 26298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.196:45583, user_credentials={real_user=slave}} blocked reactor thread for 109982us
W20260924 14:32:18.079041 26221 consensus_peers.cc:487] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.110038 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:18.128418 26221 consensus_peers.cc:487] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:18.129487 26221 consensus_peers.cc:487] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:18.148240 26221 consensus_peers.cc:487] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.190512 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.252377 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:18.330431 23135 tablet_server.cc:178] TabletServer@127.22.151.194:0 shutting down...
W20260924 14:32:18.340801 26295 consensus_peers.cc:487] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.365608 26371 consensus_peers.cc:487] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.376348 26297 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 384 similar messages]
W20260924 14:32:18.380014 26297 consensus_peers.cc:487] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.388661 26371 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.398794 26297 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.404399 26371 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.406152 26297 consensus_peers.cc:487] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.446000 26295 consensus_peers.cc:487] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.446077 26371 consensus_peers.cc:487] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.467770 26221 consensus_peers.cc:487] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260924 14:32:18.470865 26221 consensus_peers.cc:487] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.501717 26295 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:18.506345 26371 consensus_peers.cc:487] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.540619 26297 consensus_peers.cc:487] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.544828 26295 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:18.559530 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:32:18.563313 23135 tablet_replica.cc:330] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.564400 23135 raft_consensus.cc:2229] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.565737 23135 raft_consensus.cc:2258] T 9aa1d9a124214b1089cb23dd48f8d322 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.570778 23135 tablet_replica.cc:330] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:18.579526 26371 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.584306 23135 raft_consensus.cc:2229] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.585678 23135 raft_consensus.cc:2258] T c4ddc9746f0a41f08d38d9ab46bac5ba P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.588786 23135 tablet_replica.cc:330] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.589807 23135 raft_consensus.cc:2229] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.591254 23135 raft_consensus.cc:2258] T 5581950b69104df485f842c6b78b1047 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:18.595682 26297 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.609375 26371 consensus_peers.cc:487] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.611354 23135 tablet_replica.cc:330] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.612601 23135 raft_consensus.cc:2229] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.613910 23135 raft_consensus.cc:2258] T 4a92b51c57b441158371bdfdf2d193c1 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.617079 23135 tablet_replica.cc:330] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.620443 23135 raft_consensus.cc:2229] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.621763 23135 raft_consensus.cc:2258] T 70ed8dcd463347619717298c81a42998 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:18.633249 26297 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.634368 26297 consensus_peers.cc:487] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.634572 23135 tablet_replica.cc:330] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.635649 23135 raft_consensus.cc:2229] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.636857 23135 raft_consensus.cc:2258] T 96f2f08e7f5a416886c9a652e4ad11f9 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.639994 23135 tablet_replica.cc:330] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.640933 23135 raft_consensus.cc:2229] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Raft consensus shutting down.
W20260924 14:32:18.646214 26371 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.652047 26371 consensus_peers.cc:487] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.653678 23135 raft_consensus.cc:2258] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:18.654726 26297 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.657130 23135 tablet_replica.cc:330] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.658162 23135 raft_consensus.cc:2229] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 21 LEADER]: Raft consensus shutting down.
W20260924 14:32:18.659325 26297 consensus_peers.cc:487] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.659369 23135 raft_consensus.cc:2258] T e188e3a6a42340858d5f3ada3f5e867d P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.664417 23135 tablet_replica.cc:330] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:18.665073 26297 consensus_peers.cc:487] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.666733 23135 raft_consensus.cc:2229] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:18.681123 26295 consensus_peers.cc:487] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260924 14:32:18.712698 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:18.667384 23135 raft_consensus.cc:2258] T bcac7743e33046019152a570ae473ca3 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.720700 23135 tablet_replica.cc:330] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.732659 23135 raft_consensus.cc:2229] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Raft consensus shutting down.
W20260924 14:32:18.737448 26297 consensus_peers.cc:487] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:18.740141 26221 consensus_peers.cc:487] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:18.747148 23135 raft_consensus.cc:2258] T 9bb2dfc0ec214f5497b6887c8f98ad9f P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.750255 23135 tablet_replica.cc:330] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.751144 23135 raft_consensus.cc:2229] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Raft consensus shutting down.
W20260924 14:32:18.754933 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:18.759353 23135 raft_consensus.cc:2258] T 612b21159cac4f00b88b7076d49d4fec P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.762657 23135 tablet_replica.cc:330] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.763578 23135 raft_consensus.cc:2229] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.764456 23135 raft_consensus.cc:2258] T 3a94b735387741fca7f83059b64c0917 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:18.770335 26371 consensus_peers.cc:487] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.779896 23135 tablet_replica.cc:330] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.781232 23135 raft_consensus.cc:2229] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.782124 23135 raft_consensus.cc:2258] T 986260198b71418c831bfc4617da4603 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.785611 23135 tablet_replica.cc:330] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:18.789125 26295 consensus_peers.cc:487] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260924 14:32:18.806779 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:18.801492 23135 raft_consensus.cc:2229] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 LEADER]: Raft consensus shutting down.
W20260924 14:32:18.807562 26221 consensus_peers.cc:487] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:18.808418 23135 raft_consensus.cc:2258] T a2799f93a20d4fd58d617adf2eda3070 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.811614 23135 tablet_replica.cc:330] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.821934 23135 raft_consensus.cc:2229] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.823223 23135 raft_consensus.cc:2258] T 97ce0c6b37e04bf89405065d71329eb8 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.826202 23135 tablet_replica.cc:330] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:18.828771 26297 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.828991 23135 raft_consensus.cc:2229] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.830346 23135 raft_consensus.cc:2258] T 16dc088d2abf460eb278e44fc4f60620 P 6c22052275034c369b7f7dca086fbd1e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.833050 23135 tablet_replica.cc:330] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.838567 23135 raft_consensus.cc:2229] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.846748 23135 raft_consensus.cc:2258] T 1455727abfd74a9aa50ba77fa9c79771 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.849704 23135 tablet_replica.cc:330] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.850494 23135 raft_consensus.cc:2229] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.851362 23135 raft_consensus.cc:2258] T 28de2d954e094686bc5d3b931d2924ce P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.854121 23135 tablet_replica.cc:330] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.864084 23135 raft_consensus.cc:2229] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.864755 23135 raft_consensus.cc:2258] T 64e0b4058f18400dab940d3832586746 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.867586 23135 tablet_replica.cc:330] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.868438 23135 raft_consensus.cc:2229] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.869555 23135 raft_consensus.cc:2258] T 0756687d99e643c094531124c9438376 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.872422 23135 tablet_replica.cc:330] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:18.876981 26371 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.883754 23135 raft_consensus.cc:2229] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.884516 23135 raft_consensus.cc:2258] T cf849a0475754232bd6bf5cd41d3821c P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:18.885229 26371 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:18.892935 23135 tablet_replica.cc:330] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.893846 23135 raft_consensus.cc:2229] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.894661 23135 raft_consensus.cc:2258] T e912296c0f6f4c4d89bbec1581a46cf9 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.897934 23135 tablet_replica.cc:330] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.898801 23135 raft_consensus.cc:2229] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 12 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.900507 23135 raft_consensus.cc:2258] T e81d1761568f40418d9259b74501d677 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.903558 23135 tablet_replica.cc:330] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.911612 23135 raft_consensus.cc:2229] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:18.913079 23135 raft_consensus.cc:2258] T 0eec7b6dbbd94479aa179fa23fc1f403 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:18.916152 23135 tablet_replica.cc:330] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:18.918943 23135 raft_consensus.cc:2229] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:18.919655 23135 raft_consensus.cc:2258] T 78540b1453ae42188ee3fcd5d9d4cb94 P 6c22052275034c369b7f7dca086fbd1e [term 18 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:19.016225 26353 debug-util.cc:398] Leaking SignalData structure 0x7b0800261fc0 after lost signal to thread 23138
W20260924 14:32:19.041570 26353 debug-util.cc:398] Leaking SignalData structure 0x7b08000fb000 after lost signal to thread 26115
W20260924 14:32:19.043036 26353 debug-util.cc:398] Leaking SignalData structure 0x7b08001f8d60 after lost signal to thread 26282
W20260924 14:32:19.044479 26353 debug-util.cc:398] Leaking SignalData structure 0x7b080025b6e0 after lost signal to thread 26356
W20260924 14:32:19.045689 26353 debug-util.cc:398] Leaking SignalData structure 0x7b08000d52c0 after lost signal to thread 26430
I20260924 14:32:19.047983 23135 tablet_replica.cc:330] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.048913 23135 raft_consensus.cc:2229] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.049685 23135 raft_consensus.cc:2258] T 62575580f9ac4403b5f531f616fe41fd P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.053020 23135 tablet_replica.cc:330] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.078039 23135 raft_consensus.cc:2229] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.079231 23135 raft_consensus.cc:2258] T 80f626c203394b4f8b85ab089d6aa044 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.112622 23135 tablet_replica.cc:330] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.113555 23135 raft_consensus.cc:2229] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 LEADER]: Raft consensus shutting down.
I20260924 14:32:19.114854 23135 raft_consensus.cc:2258] T 968ded8aff90408e9baafab16ae17e14 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.127461 23135 tablet_replica.cc:330] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.128191 23135 raft_consensus.cc:2229] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.133363 23135 raft_consensus.cc:2258] T 8f9812df92004da0ba2c451e7b423e66 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.136253 23135 tablet_replica.cc:330] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.136973 23135 raft_consensus.cc:2229] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.137739 23135 raft_consensus.cc:2258] T 9301429ace424dc58cbf9b57301e674c P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.146116 23135 tablet_replica.cc:330] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.146978 23135 raft_consensus.cc:2229] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:19.148171 23135 raft_consensus.cc:2258] T 42cfb7094048467797704ab8ee156f83 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.150987 23135 tablet_replica.cc:330] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
W20260924 14:32:19.153993 26295 consensus_peers.cc:487] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260924 14:32:19.160554 23135 raft_consensus.cc:2229] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.161512 23135 raft_consensus.cc:2258] T 67038d5a253c4d3e8d78c9fc7446aac8 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.164832 23135 tablet_replica.cc:330] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.165659 23135 raft_consensus.cc:2229] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.166491 23135 raft_consensus.cc:2258] T 7ad26252e4284fb18db8d64f502c4a10 P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.169714 23135 tablet_replica.cc:330] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.170553 23135 raft_consensus.cc:2229] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 LEADER]: Raft consensus shutting down.
I20260924 14:32:19.171872 23135 raft_consensus.cc:2258] T 898441fbc17a42e09c7639e00843680b P 6c22052275034c369b7f7dca086fbd1e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.174804 23135 tablet_replica.cc:330] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.175683 23135 raft_consensus.cc:2229] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.176419 23135 raft_consensus.cc:2258] T 051d85003b754a6e95eaf1d19022c217 P 6c22052275034c369b7f7dca086fbd1e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.190414 23135 tablet_replica.cc:330] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.195945 23135 raft_consensus.cc:2229] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.196841 23135 raft_consensus.cc:2258] T 00361946476c4fecbd244a86b9cfb752 P 6c22052275034c369b7f7dca086fbd1e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.208304 23135 tablet_replica.cc:330] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.209239 23135 raft_consensus.cc:2229] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 21 LEADER]: Raft consensus shutting down.
I20260924 14:32:19.216506 23135 raft_consensus.cc:2258] T 168958697ba24715a127c9c7b7c520ff P 6c22052275034c369b7f7dca086fbd1e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.220135 23135 tablet_replica.cc:330] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.221069 23135 raft_consensus.cc:2229] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.221843 23135 raft_consensus.cc:2258] T 7ed216090b8f4d88a827c18c45fd0422 P 6c22052275034c369b7f7dca086fbd1e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.226089 23135 tablet_replica.cc:330] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.226979 23135 raft_consensus.cc:2229] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.227918 23135 raft_consensus.cc:2258] T ededbbb82305459a8fbb9dc1f5731ee8 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.246472 23135 tablet_replica.cc:330] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.247500 23135 raft_consensus.cc:2229] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:19.248291 26295 consensus_peers.cc:487] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:19.248292 23135 raft_consensus.cc:2258] T 4630fa065ede40338b319f0846fdd511 P 6c22052275034c369b7f7dca086fbd1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.252257 23135 tablet_replica.cc:330] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.253113 23135 raft_consensus.cc:2229] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.253758 23135 raft_consensus.cc:2258] T 5e6029c377f94de7b21145dc0ac2f5a7 P 6c22052275034c369b7f7dca086fbd1e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:19.256636 26295 consensus_peers.cc:487] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:19.256806 23135 tablet_replica.cc:330] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.257834 23135 raft_consensus.cc:2229] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.258658 23135 raft_consensus.cc:2258] T dda86ff6977641a8af47dad8d1d1a049 P 6c22052275034c369b7f7dca086fbd1e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.263123 23135 tablet_replica.cc:330] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.263989 23135 raft_consensus.cc:2229] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.264699 23135 raft_consensus.cc:2258] T dad9bf5791f146a4b74218b2dc7e458f P 6c22052275034c369b7f7dca086fbd1e [term 11 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:19.268537 23135 tablet_replica.cc:330] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e: stopping tablet replica
I20260924 14:32:19.269280 23135 raft_consensus.cc:2229] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:19.270128 23135 raft_consensus.cc:2258] T 0fae41ae7fca440da0de40b0ee6973c7 P 6c22052275034c369b7f7dca086fbd1e [term 15 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:19.342978 26295 consensus_peers.cc:487] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:19.556044 26295 consensus_peers.cc:487] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260924 14:32:19.652318 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:19.701676 26369 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:19.740130   575 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.740695   575 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:19.742730   575 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:19.744280 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:32:19.749516 26371 leader_election.cc:334] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:19.749939 26371 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:19.751178   575 raft_consensus.cc:2726] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:19.764598   575 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.765170   575 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:19.767282   575 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:19.776144 26371 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:19.793493   575 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.794186   575 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:19.800388   567 raft_consensus.cc:482] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.801011   567 raft_consensus.cc:504] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:19.803002   567 leader_election.cc:288] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:19.805472   575 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:19.807646 26371 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:19.809374 26297 leader_election.cc:334] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:19.811221 26295 leader_election.cc:334] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:19.811772 26295 leader_election.cc:302] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:19.812994   567 raft_consensus.cc:2726] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260924 14:32:19.824493 26369 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:19.825232 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:19.827016   575 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:19.828769 26323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "45ab97b763804767846e9d27a01b561f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f59f3234edf84aab852fa989fb754d31" is_pre_election: true
W20260924 14:32:19.831350 26295 consensus_peers.cc:487] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:19.831740 26369 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:19.832561   575 raft_consensus.cc:2726] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:32:19.843760   575 raft_consensus.cc:482] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.844285   575 raft_consensus.cc:504] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:19.846359   575 leader_election.cc:288] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:19.855724 26371 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:19.858784 26369 leader_election.cc:334] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:19.859280 26369 leader_election.cc:302] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:19.860817   573 raft_consensus.cc:2726] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:32:19.930563   575 raft_consensus.cc:482] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.931156   575 raft_consensus.cc:504] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:19.933431   575 leader_election.cc:288] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:19.938386 26371 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:19.939091 26369 leader_election.cc:334] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:19.939582 26369 leader_election.cc:302] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:19.940346   575 raft_consensus.cc:2726] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:32:19.962396   575 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:19.963023   575 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:19.970981 26371 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:19.978031 26369 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:19.978503 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:19.982013   573 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:19.965386   575 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:20.011761 26369 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:20.014333   567 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.015060   567 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.019575   581 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.020125   581 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.021078   567 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:32:20.021919 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:20.022221   581 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:32:20.022604 26397 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
I20260924 14:32:20.025978 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 
W20260924 14:32:20.026899 26297 leader_election.cc:334] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.028426   581 raft_consensus.cc:482] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.028973   581 raft_consensus.cc:504] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.033608   581 leader_election.cc:288] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:20.034570 26297 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.026952   567 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260924 14:32:20.030687 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:20.037180   567 raft_consensus.cc:482] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.037436 26397 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 18.
I20260924 14:32:20.037678   567 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:20.044416 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:32:20.045600   581 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260924 14:32:20.045989   581 raft_consensus.cc:482] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.046361   581 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Advancing to term 19
W20260924 14:32:20.051247 26295 leader_election.cc:334] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.059851   575 raft_consensus.cc:482] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:32:20.060038 26297 leader_election.cc:334] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.060504 26297 leader_election.cc:302] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.061605   582 raft_consensus.cc:2726] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:20.060555   575 raft_consensus.cc:504] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
W20260924 14:32:20.071012 26369 leader_election.cc:334] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.073745   575 leader_election.cc:288] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.069621   581 raft_consensus.cc:504] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.090708   581 leader_election.cc:288] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
I20260924 14:32:20.081470   567 raft_consensus.cc:504] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.093591   567 leader_election.cc:288] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Requested vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:32:20.095129 26297 leader_election.cc:334] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.098850 26297 leader_election.cc:334] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.100247 26295 consensus_peers.cc:487] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260924 14:32:20.102516 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa1d9a124214b1089cb23dd48f8d322" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:20.104497 26397 raft_consensus.cc:3037] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Advancing to term 19
I20260924 14:32:20.104348 26406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f2f08e7f5a416886c9a652e4ad11f9" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f"
I20260924 14:32:20.105219 26406 raft_consensus.cc:3037] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Advancing to term 20
I20260924 14:32:20.113507 26406 raft_consensus.cc:2454] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 20.
I20260924 14:32:20.132366   581 raft_consensus.cc:482] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.133044   581 raft_consensus.cc:504] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.135094   581 leader_election.cc:288] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.142510 26397 raft_consensus.cc:2454] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f59f3234edf84aab852fa989fb754d31 in term 19.
W20260924 14:32:20.145285 26371 leader_election.cc:334] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.155045 26297 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.158547   575 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.170012   575 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:20.172338   567 raft_consensus.cc:482] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.172945   567 raft_consensus.cc:504] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.174937   567 leader_election.cc:288] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.174988   581 raft_consensus.cc:482] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.175559   581 raft_consensus.cc:504] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.175844   573 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.176414   573 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:20.177601   581 leader_election.cc:288] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.178326   573 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.169369 26371 leader_election.cc:302] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.183593   573 raft_consensus.cc:2726] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:32:20.182775 26297 leader_election.cc:334] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.183414 26369 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:20.184227 26297 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.185978 26295 leader_election.cc:334] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.187057 26298 leader_election.cc:302] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:32:20.188351   567 raft_consensus.cc:2781] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Leader election won for term 20
I20260924 14:32:20.189132   567 raft_consensus.cc:686] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:20.189997   567 consensus_queue.cc:237] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.186147   575 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.194136 26295 leader_election.cc:302] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.194350 26298 leader_election.cc:302] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45ab97b763804767846e9d27a01b561f, f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e
I20260924 14:32:20.199661   581 raft_consensus.cc:2726] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:20.200165   582 raft_consensus.cc:2781] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Leader election won for term 19
I20260924 14:32:20.200714   582 raft_consensus.cc:686] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 LEADER]: Becoming Leader. State: Replica: f59f3234edf84aab852fa989fb754d31, State: Running, Role: LEADER
I20260924 14:32:20.201625   582 consensus_queue.cc:237] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
W20260924 14:32:20.228495 26369 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.251655   573 raft_consensus.cc:482] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
W20260924 14:32:20.251853 26371 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.252394 26371 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.252238   573 raft_consensus.cc:504] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:20.254415   573 leader_election.cc:288] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:20.256512 26371 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.256831   582 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.257043 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.257443   582 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.259714   575 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260924 14:32:20.260263 26369 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.261646   582 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), 45ab97b763804767846e9d27a01b561f (127.22.151.196:45583)
W20260924 14:32:20.262639 26295 leader_election.cc:334] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.263033 26295 leader_election.cc:302] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.263629   582 raft_consensus.cc:2726] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260924 14:32:20.263839 26295 leader_election.cc:334] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.264269 26295 leader_election.cc:302] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.264874   582 raft_consensus.cc:2726] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260924 14:32:20.265072 26295 consensus_peers.cc:487] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:20.269641   594 raft_consensus.cc:2726] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260924 14:32:20.274562 26297 leader_election.cc:334] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.274533 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cfb7094048467797704ab8ee156f83" candidate_uuid: "f59f3234edf84aab852fa989fb754d31" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "45ab97b763804767846e9d27a01b561f" is_pre_election: true
I20260924 14:32:20.282199 26298 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 45ab97b763804767846e9d27a01b561f, 6c22052275034c369b7f7dca086fbd1e
I20260924 14:32:20.283609   582 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:20.282130 26075 catalog_manager.cc:5495] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 19 to 20, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 20 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
W20260924 14:32:20.285847 26371 leader_election.cc:334] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.286320 26371 leader_election.cc:302] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.287019   594 raft_consensus.cc:2726] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260924 14:32:20.286707 26075 catalog_manager.cc:5495] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 reported cstate change: term changed from 18 to 19, leader changed from 6c22052275034c369b7f7dca086fbd1e (127.22.151.194) to f59f3234edf84aab852fa989fb754d31 (127.22.151.195). New cstate: current_term: 19 leader_uuid: "f59f3234edf84aab852fa989fb754d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } health_report { overall_health: UNKNOWN } } }
W20260924 14:32:20.296924 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:20.310659   589 raft_consensus.cc:482] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.311378   589 raft_consensus.cc:504] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.320747   589 leader_election.cc:288] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:20.327977 26295 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
W20260924 14:32:20.336241 26297 leader_election.cc:334] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.336994 26297 leader_election.cc:302] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.338256   589 raft_consensus.cc:2726] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260924 14:32:20.338925   594 raft_consensus.cc:482] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.339530   594 raft_consensus.cc:504] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:20.342406   594 leader_election.cc:288] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.342653 23135 tablet_server.cc:194] TabletServer@127.22.151.194:0 shutdown complete.
W20260924 14:32:20.345252 26371 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.347048 26369 leader_election.cc:334] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.347585 26369 leader_election.cc:302] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.348781   594 raft_consensus.cc:2726] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260924 14:32:20.354873 26295 consensus_peers.cc:487] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260924 14:32:20.476528 23135 tablet_server.cc:178] TabletServer@127.22.151.195:0 shutting down...
W20260924 14:32:20.504868 26295 consensus_peers.cc:487] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:20.510370 26369 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:20.543242   594 raft_consensus.cc:482] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.543887   594 raft_consensus.cc:504] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:20.546155   594 leader_election.cc:288] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
W20260924 14:32:20.555068 26371 leader_election.cc:334] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
W20260924 14:32:20.560209 26369 leader_election.cc:334] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260924 14:32:20.560755 26369 leader_election.cc:302] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:20.561640   594 raft_consensus.cc:2726] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:20.568717 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:32:20.574366 23135 tablet_replica.cc:330] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.575567 23135 raft_consensus.cc:2229] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.576340   582 raft_consensus.cc:482] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.576905 23135 pending_rounds.cc:62] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Trying to abort 1 pending ops.
I20260924 14:32:20.577315 23135 pending_rounds.cc:68] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31: Aborting op as it isn't in flight: id { term: 19 index: 12 } timestamp: 7332906353479512064 op_type: NO_OP noop_request { }
I20260924 14:32:20.578966 23135 raft_consensus.cc:2866] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 14:32:20.579419 23135 raft_consensus.cc:2258] T 9aa1d9a124214b1089cb23dd48f8d322 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.576905   582 raft_consensus.cc:504] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.582849 23135 tablet_replica.cc:330] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.584384 23135 raft_consensus.cc:2229] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
W20260924 14:32:20.585373 26295 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.585670 23135 raft_consensus.cc:2258] T f272a708f6ca4bd8bc3a3c4925abad6d P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.589224 23135 tablet_replica.cc:330] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.590278 23135 raft_consensus.cc:2229] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.590919 23135 raft_consensus.cc:2258] T 78da914f30dc4c65be5ff03ec099253b P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.591787 26297 leader_election.cc:334] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.592273 26297 leader_election.cc:302] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.593041   589 raft_consensus.cc:2726] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260924 14:32:20.594137 23135 tablet_replica.cc:330] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.588405   582 leader_election.cc:288] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.595619   589 raft_consensus.cc:482] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:20.596151   589 raft_consensus.cc:504] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } }
I20260924 14:32:20.596612 23135 raft_consensus.cc:2229] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.597605 23135 raft_consensus.cc:2258] T ededbbb82305459a8fbb9dc1f5731ee8 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.600479 23135 tablet_replica.cc:330] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.602094 23135 raft_consensus.cc:2229] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.602825 23135 raft_consensus.cc:2258] T c4ddc9746f0a41f08d38d9ab46bac5ba P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.603564 26295 consensus_peers.cc:487] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260924 14:32:20.605995 23135 tablet_replica.cc:330] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.606976 23135 raft_consensus.cc:2229] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.607232   589 leader_election.cc:288] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:20.607823 23135 raft_consensus.cc:2258] T 7ad26252e4284fb18db8d64f502c4a10 P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.611418 23135 tablet_replica.cc:330] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.612457 23135 raft_consensus.cc:2229] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.613234 23135 raft_consensus.cc:2258] T 3a94b735387741fca7f83059b64c0917 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.616106 23135 tablet_replica.cc:330] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.616992 23135 raft_consensus.cc:2229] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 11 LEADER]: Raft consensus shutting down.
W20260924 14:32:20.618256 26297 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:20.619275 23135 raft_consensus.cc:2258] T dad9bf5791f146a4b74218b2dc7e458f P f59f3234edf84aab852fa989fb754d31 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.622704 23135 tablet_replica.cc:330] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.623658 23135 raft_consensus.cc:2229] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.625011 23135 raft_consensus.cc:2258] T 00498335d0b847f299d347155be5c437 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.627808 23135 tablet_replica.cc:330] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.628746 23135 raft_consensus.cc:2229] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.629565 23135 pending_rounds.cc:62] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Trying to abort 1 pending ops.
I20260924 14:32:20.629849 23135 pending_rounds.cc:68] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7332906353427582976 op_type: NO_OP noop_request { }
I20260924 14:32:20.630285 23135 raft_consensus.cc:2866] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260924 14:32:20.630656 23135 raft_consensus.cc:2258] T 96f2f08e7f5a416886c9a652e4ad11f9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.631937 26295 leader_election.cc:334] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:20.635607 26295 leader_election.cc:302] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f59f3234edf84aab852fa989fb754d31; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:20.637574   582 raft_consensus.cc:2726] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260924 14:32:20.639236 26369 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:20.646582 23135 tablet_replica.cc:330] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.647691 23135 raft_consensus.cc:2229] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.648420 23135 raft_consensus.cc:2258] T 968ded8aff90408e9baafab16ae17e14 P f59f3234edf84aab852fa989fb754d31 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.651503 23135 tablet_replica.cc:330] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.652309 23135 raft_consensus.cc:2229] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.653594 23135 raft_consensus.cc:2258] T f59f0af5dd6048918c6579b295ac5510 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.656509 23135 tablet_replica.cc:330] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
W20260924 14:32:20.656616 26369 consensus_peers.cc:487] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.657428 23135 raft_consensus.cc:2229] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.658169 23135 raft_consensus.cc:2258] T 97ce0c6b37e04bf89405065d71329eb8 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.661293 23135 tablet_replica.cc:330] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.662246 23135 raft_consensus.cc:2229] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.666383 23135 raft_consensus.cc:2258] T 16dc088d2abf460eb278e44fc4f60620 P f59f3234edf84aab852fa989fb754d31 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.671438 23135 tablet_replica.cc:330] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.674365 23135 raft_consensus.cc:2229] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.675594 23135 raft_consensus.cc:2258] T 1455727abfd74a9aa50ba77fa9c79771 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.678705 23135 tablet_replica.cc:330] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.679689 23135 raft_consensus.cc:2229] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.681979 23135 raft_consensus.cc:2258] T 28de2d954e094686bc5d3b931d2924ce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.684840 23135 tablet_replica.cc:330] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.685684 23135 raft_consensus.cc:2229] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.686364 23135 raft_consensus.cc:2258] T fbdb392c304c4e15b1b6c10c61a95323 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.690737 23135 tablet_replica.cc:330] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.691707 23135 raft_consensus.cc:2229] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
W20260924 14:32:20.699717 26369 consensus_peers.cc:487] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.700645 23135 raft_consensus.cc:2258] T 64e0b4058f18400dab940d3832586746 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.703979 23135 tablet_replica.cc:330] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.704888 23135 raft_consensus.cc:2229] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.706063 23135 raft_consensus.cc:2258] T ac352e5066864ac6be3f50d39a066493 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.709087 23135 tablet_replica.cc:330] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.709970 23135 raft_consensus.cc:2229] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.711236 23135 raft_consensus.cc:2258] T d2f9b654ec4542a781bdc872c6ddbdce P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.714181 23135 tablet_replica.cc:330] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.715063 23135 raft_consensus.cc:2229] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.715956 23135 raft_consensus.cc:2258] T 9bb2dfc0ec214f5497b6887c8f98ad9f P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.718729 23135 tablet_replica.cc:330] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.719591 23135 raft_consensus.cc:2229] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.720829 23135 raft_consensus.cc:2258] T 014ae451ea994b5e9926dc80215382e4 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.724939 26295 consensus_peers.cc:487] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260924 14:32:20.725569 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.764430 23135 tablet_replica.cc:330] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.768420 23135 raft_consensus.cc:2229] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Raft consensus shutting down.
W20260924 14:32:20.780337 26369 consensus_peers.cc:487] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.785007 23135 raft_consensus.cc:2258] T bcac7743e33046019152a570ae473ca3 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.789222 26369 consensus_peers.cc:487] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.791559 23135 tablet_replica.cc:330] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.793839 23135 raft_consensus.cc:2229] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.797737 23135 raft_consensus.cc:2258] T 67038d5a253c4d3e8d78c9fc7446aac8 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.800458 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:20.801757 23135 tablet_replica.cc:330] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.805200 23135 raft_consensus.cc:2229] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.806350 23135 raft_consensus.cc:2258] T 9f81913531e84a379f3bc3886ea64e95 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.810393 23135 tablet_replica.cc:330] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.811990 23135 raft_consensus.cc:2229] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.813992 23135 raft_consensus.cc:2258] T 0756687d99e643c094531124c9438376 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.819355 23135 tablet_replica.cc:330] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.820537 23135 raft_consensus.cc:2229] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.822606 23135 raft_consensus.cc:2258] T cf849a0475754232bd6bf5cd41d3821c P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.826314 23135 tablet_replica.cc:330] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.827842 23135 raft_consensus.cc:2229] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.830291 23135 raft_consensus.cc:2258] T e912296c0f6f4c4d89bbec1581a46cf9 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.834012 23135 tablet_replica.cc:330] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.834873 23135 raft_consensus.cc:2229] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.838030 23135 raft_consensus.cc:2258] T 5e6029c377f94de7b21145dc0ac2f5a7 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.843384 23135 tablet_replica.cc:330] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.844344 23135 raft_consensus.cc:2229] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 19 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.847525 23135 raft_consensus.cc:2258] T dda86ff6977641a8af47dad8d1d1a049 P f59f3234edf84aab852fa989fb754d31 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.852530 23135 tablet_replica.cc:330] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
W20260924 14:32:20.853818 26369 consensus_peers.cc:487] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:20.860987 26371 consensus_peers.cc:487] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:32:20.853897 23135 raft_consensus.cc:2229] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.864837 23135 raft_consensus.cc:2258] T 26229a9b02a646ac80993cd410cd7237 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.868923 23135 tablet_replica.cc:330] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.869902 23135 raft_consensus.cc:2229] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.871351 23135 raft_consensus.cc:2258] T 4a92b51c57b441158371bdfdf2d193c1 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.875974 23135 tablet_replica.cc:330] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.876915 23135 raft_consensus.cc:2229] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:20.877907 26371 consensus_peers.cc:487] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260924 14:32:20.878228 23135 raft_consensus.cc:2258] T a83a2b98291f41d7beed3c1329ba4bc9 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.882902 23135 tablet_replica.cc:330] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.883945 23135 raft_consensus.cc:2229] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.886997 23135 raft_consensus.cc:2258] T 8f9812df92004da0ba2c451e7b423e66 P f59f3234edf84aab852fa989fb754d31 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.891323 23135 tablet_replica.cc:330] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.892376 23135 raft_consensus.cc:2229] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.893333 23135 raft_consensus.cc:2258] T 9301429ace424dc58cbf9b57301e674c P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.897696 23135 tablet_replica.cc:330] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.898619 23135 raft_consensus.cc:2229] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.902264 23135 raft_consensus.cc:2258] T 78540b1453ae42188ee3fcd5d9d4cb94 P f59f3234edf84aab852fa989fb754d31 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.906579 23135 tablet_replica.cc:330] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.907662 23135 raft_consensus.cc:2229] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.909608 23135 raft_consensus.cc:2258] T 1c5083097b524eb584b0af59aace9850 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.914467 23135 tablet_replica.cc:330] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.915637 23135 raft_consensus.cc:2229] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.917023 23135 raft_consensus.cc:2258] T 42cfb7094048467797704ab8ee156f83 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.921238 23135 tablet_replica.cc:330] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.922166 23135 raft_consensus.cc:2229] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.924371 23135 raft_consensus.cc:2258] T 051d85003b754a6e95eaf1d19022c217 P f59f3234edf84aab852fa989fb754d31 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.928529 23135 tablet_replica.cc:330] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.929448 23135 raft_consensus.cc:2229] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 17 LEADER]: Raft consensus shutting down.
W20260924 14:32:20.931389 26369 consensus_peers.cc:487] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260924 14:32:20.932300 23135 raft_consensus.cc:2258] T c6e4a72c4886471c8096037b808a6543 P f59f3234edf84aab852fa989fb754d31 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.936990 23135 tablet_replica.cc:330] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.937865 23135 raft_consensus.cc:2229] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.939168 23135 raft_consensus.cc:2258] T 168958697ba24715a127c9c7b7c520ff P f59f3234edf84aab852fa989fb754d31 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.942528 26369 consensus_peers.cc:487] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260924 14:32:20.943146 23135 tablet_replica.cc:330] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.944168 23135 raft_consensus.cc:2229] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 26 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.945822 23135 raft_consensus.cc:2258] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P f59f3234edf84aab852fa989fb754d31 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.949870 23135 tablet_replica.cc:330] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.950976 23135 raft_consensus.cc:2229] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:20.951965 23135 raft_consensus.cc:2258] T 0fae41ae7fca440da0de40b0ee6973c7 P f59f3234edf84aab852fa989fb754d31 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:20.957347 23135 tablet_replica.cc:330] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31: stopping tablet replica
I20260924 14:32:20.958338 23135 raft_consensus.cc:2229] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 10 LEADER]: Raft consensus shutting down.
I20260924 14:32:20.960721 23135 raft_consensus.cc:2258] T d4d74021f6fc4625982decf68a590eac P f59f3234edf84aab852fa989fb754d31 [term 10 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:20.970311 26369 consensus_peers.cc:487] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:20.993076 26369 consensus_peers.cc:487] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:21.019672 26371 consensus_peers.cc:487] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.022562 26371 consensus_peers.cc:487] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.036401 26369 consensus_peers.cc:487] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f -> Peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Couldn't send request to peer f59f3234edf84aab852fa989fb754d31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:21.053937 26371 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.097529 26369 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260924 14:32:21.098291 26371 consensus_peers.cc:487] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.110530 26371 consensus_peers.cc:487] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.142534 26371 consensus_peers.cc:487] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.204540 26371 consensus_peers.cc:487] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.357369 26371 consensus_peers.cc:487] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.421296 26371 consensus_peers.cc:487] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.425014 26371 consensus_peers.cc:487] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f -> Peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Couldn't send request to peer 6c22052275034c369b7f7dca086fbd1e. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260924 14:32:21.491405 26369 consensus_peers.cc:487] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f -> Peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Couldn't send request to peer 7e09fbebe29a49d291d23f9d395fe335. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260924 14:32:21.561273 23135 tablet_server.cc:194] TabletServer@127.22.151.195:0 shutdown complete.
I20260924 14:32:21.616537   594 raft_consensus.cc:482] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:32:21.617383   594 raft_consensus.cc:504] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:21.626499   601 raft_consensus.cc:482] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:21.627336   601 raft_consensus.cc:504] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:21.630025   601 leader_election.cc:288] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
W20260924 14:32:21.635528 26371 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:21.620981   594 leader_election.cc:288] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:21.641641 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Network error: Client connection negotiation failed: client connection to 127.22.151.195:45507: connect: Connection refused (error 111)
W20260924 14:32:21.643658 26369 leader_election.cc:334] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.644212 26369 leader_election.cc:302] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:21.645145   594 raft_consensus.cc:2726] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260924 14:32:21.649324 26369 leader_election.cc:334] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.649901 26369 leader_election.cc:302] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:21.650810   594 raft_consensus.cc:2726] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260924 14:32:21.674921 23135 tablet_server.cc:178] TabletServer@127.22.151.196:0 shutting down...
I20260924 14:32:21.745576   594 raft_consensus.cc:482] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:32:21.746374   594 raft_consensus.cc:504] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:21.748896   594 leader_election.cc:288] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
W20260924 14:32:21.756474 26369 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Network error: Client connection negotiation failed: client connection to 127.22.151.195:45507: connect: Connection refused (error 111)
W20260924 14:32:21.757248 26369 leader_election.cc:334] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.757684 26369 leader_election.cc:302] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:21.758469   594 raft_consensus.cc:2726] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:21.774099 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:32:21.775378 23135 tablet_replica.cc:330] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.776630 23135 raft_consensus.cc:2229] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.777767   594 raft_consensus.cc:482] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:32:21.778342 23135 raft_consensus.cc:2258] T f272a708f6ca4bd8bc3a3c4925abad6d P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.778328   594 raft_consensus.cc:504] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } } peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } }
I20260924 14:32:21.784106 23135 tablet_replica.cc:330] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.785684 23135 raft_consensus.cc:2229] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:21.785715 26371 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:21.787261 23135 raft_consensus.cc:2258] T 5e6029c377f94de7b21145dc0ac2f5a7 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.791538 23135 tablet_replica.cc:330] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.792837 23135 raft_consensus.cc:2229] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.793655 23135 raft_consensus.cc:2258] T 9aa1d9a124214b1089cb23dd48f8d322 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:21.798586 26369 leader_election.cc:334] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Network error: Client connection negotiation failed: client connection to 127.22.151.195:45507: connect: Connection refused (error 111)
I20260924 14:32:21.799141 26369 leader_election.cc:302] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:21.799974   601 raft_consensus.cc:2726] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260924 14:32:21.800832 23135 tablet_replica.cc:330] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.801934 23135 raft_consensus.cc:2229] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.803115 23135 raft_consensus.cc:2258] T 78da914f30dc4c65be5ff03ec099253b P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.806807   601 raft_consensus.cc:482] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f59f3234edf84aab852fa989fb754d31)
I20260924 14:32:21.806944   594 leader_election.cc:288] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433), f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507)
I20260924 14:32:21.807461   601 raft_consensus.cc:504] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f59f3234edf84aab852fa989fb754d31" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 45507 } } peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } }
I20260924 14:32:21.809818   601 leader_election.cc:288] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507), 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943)
I20260924 14:32:21.811715 23135 tablet_replica.cc:330] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.814378 23135 raft_consensus.cc:2229] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.815920 23135 raft_consensus.cc:2258] T 80f626c203394b4f8b85ab089d6aa044 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.820498 23135 tablet_replica.cc:330] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.829037 23135 raft_consensus.cc:2229] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.830577 23135 raft_consensus.cc:2258] T 3a94b735387741fca7f83059b64c0917 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.833830 23135 tablet_replica.cc:330] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.834928 23135 raft_consensus.cc:2229] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.835804 23135 raft_consensus.cc:2258] T dad9bf5791f146a4b74218b2dc7e458f P 45ab97b763804767846e9d27a01b561f [term 11 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.835865   594 raft_consensus.cc:482] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:21.836611   594 raft_consensus.cc:504] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:21.838814   594 leader_election.cc:288] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:21.839855 23135 tablet_replica.cc:330] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.841127 23135 raft_consensus.cc:2229] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:21.841899 26369 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.842155 23135 raft_consensus.cc:2258] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:21.848136 26371 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:21.848897 23135 tablet_replica.cc:330] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.850034 23135 raft_consensus.cc:2229] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.850915 23135 raft_consensus.cc:2258] T 70ed8dcd463347619717298c81a42998 P 45ab97b763804767846e9d27a01b561f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.854542 23135 tablet_replica.cc:330] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.855646 23135 raft_consensus.cc:2229] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus shutting down.
W20260924 14:32:21.855681 26369 leader_election.cc:334] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f59f3234edf84aab852fa989fb754d31 (127.22.151.195:45507): Network error: Client connection negotiation failed: client connection to 127.22.151.195:45507: connect: Connection refused (error 111)
I20260924 14:32:21.856238 26369 leader_election.cc:302] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 7e09fbebe29a49d291d23f9d395fe335, f59f3234edf84aab852fa989fb754d31
I20260924 14:32:21.856855 23135 raft_consensus.cc:2258] T 96f2f08e7f5a416886c9a652e4ad11f9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:21.856940 26369 raft_consensus.cc:2670] T 00498335d0b847f299d347155be5c437 P 45ab97b763804767846e9d27a01b561f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 14:32:21.860586 23135 tablet_replica.cc:330] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
W20260924 14:32:21.861724 26369 leader_election.cc:334] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.861737 23135 raft_consensus.cc:2229] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.862298 26369 leader_election.cc:302] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:21.863185   594 raft_consensus.cc:2726] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260924 14:32:21.863224 23135 raft_consensus.cc:2258] T f59f0af5dd6048918c6579b295ac5510 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.867738 23135 tablet_replica.cc:330] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.868628 23135 raft_consensus.cc:2229] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.869424 23135 raft_consensus.cc:2258] T 1455727abfd74a9aa50ba77fa9c79771 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.872877 23135 tablet_replica.cc:330] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.873719 23135 raft_consensus.cc:2229] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.875252 23135 raft_consensus.cc:2258] T 28de2d954e094686bc5d3b931d2924ce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.877749   594 raft_consensus.cc:482] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:21.878410 23135 tablet_replica.cc:330] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.878335   594 raft_consensus.cc:504] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:21.879420 23135 raft_consensus.cc:2229] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 13 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.880936 23135 raft_consensus.cc:2258] T 00361946476c4fecbd244a86b9cfb752 P 45ab97b763804767846e9d27a01b561f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.884450 23135 tablet_replica.cc:330] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.885329 23135 raft_consensus.cc:2229] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 LEADER]: Raft consensus shutting down.
W20260924 14:32:21.885756 26371 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:21.886669   594 leader_election.cc:288] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:21.886706 23135 raft_consensus.cc:2258] T 9f81913531e84a379f3bc3886ea64e95 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.890213 23135 tablet_replica.cc:330] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.891225 23135 raft_consensus.cc:2229] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.892354 23135 raft_consensus.cc:2258] T ac352e5066864ac6be3f50d39a066493 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:21.892598 26369 leader_election.cc:334] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:21.893301 26369 leader_election.cc:302] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:21.894388   594 raft_consensus.cc:2726] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260924 14:32:21.895712 23135 tablet_replica.cc:330] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.896713 23135 raft_consensus.cc:2229] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.897446 23135 raft_consensus.cc:2258] T d2f9b654ec4542a781bdc872c6ddbdce P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.900774 23135 tablet_replica.cc:330] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.901726 23135 raft_consensus.cc:2229] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.902760 23135 raft_consensus.cc:2258] T e81d1761568f40418d9259b74501d677 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.906111 23135 tablet_replica.cc:330] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.907032 23135 raft_consensus.cc:2229] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.908119 23135 raft_consensus.cc:2258] T 26229a9b02a646ac80993cd410cd7237 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.911474 23135 tablet_replica.cc:330] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.912482 23135 raft_consensus.cc:2229] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.913568 23135 raft_consensus.cc:2258] T cf849a0475754232bd6bf5cd41d3821c P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.916882 23135 tablet_replica.cc:330] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.917820 23135 raft_consensus.cc:2229] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.920172 23135 raft_consensus.cc:2258] T a2799f93a20d4fd58d617adf2eda3070 P 45ab97b763804767846e9d27a01b561f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.923625 23135 tablet_replica.cc:330] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.924726 23135 raft_consensus.cc:2229] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.926170 23135 raft_consensus.cc:2258] T 612b21159cac4f00b88b7076d49d4fec P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.931248 23135 tablet_replica.cc:330] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.932232 23135 raft_consensus.cc:2229] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 12 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.933825 23135 raft_consensus.cc:2258] T 7ed216090b8f4d88a827c18c45fd0422 P 45ab97b763804767846e9d27a01b561f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.937429 23135 tablet_replica.cc:330] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.938390 23135 raft_consensus.cc:2229] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.939864 23135 raft_consensus.cc:2258] T ededbbb82305459a8fbb9dc1f5731ee8 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.943142 23135 tablet_replica.cc:330] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.944069 23135 raft_consensus.cc:2229] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.945863 23135 raft_consensus.cc:2258] T 4630fa065ede40338b319f0846fdd511 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.949971 23135 tablet_replica.cc:330] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.950860 23135 raft_consensus.cc:2229] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.951583 23135 raft_consensus.cc:2258] T 1c5083097b524eb584b0af59aace9850 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.954906 23135 tablet_replica.cc:330] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.955772 23135 raft_consensus.cc:2229] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.956576 23135 raft_consensus.cc:2258] T 42cfb7094048467797704ab8ee156f83 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.960004 23135 tablet_replica.cc:330] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.960829 23135 raft_consensus.cc:2229] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.961619 23135 raft_consensus.cc:2258] T 67038d5a253c4d3e8d78c9fc7446aac8 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.964763 23135 tablet_replica.cc:330] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.965588 23135 raft_consensus.cc:2229] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 21 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.966977 23135 raft_consensus.cc:2258] T 7ad26252e4284fb18db8d64f502c4a10 P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.970346 23135 tablet_replica.cc:330] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.971266 23135 raft_consensus.cc:2229] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.972332 23135 raft_consensus.cc:2258] T 898441fbc17a42e09c7639e00843680b P 45ab97b763804767846e9d27a01b561f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.975617 23135 tablet_replica.cc:330] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.976464 23135 raft_consensus.cc:2229] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:21.977541 23135 raft_consensus.cc:2258] T 5581950b69104df485f842c6b78b1047 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.980970 23135 tablet_replica.cc:330] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.981953 23135 raft_consensus.cc:2229] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.983870 23135 raft_consensus.cc:2258] T a83a2b98291f41d7beed3c1329ba4bc9 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.987504 23135 tablet_replica.cc:330] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.988330 23135 raft_consensus.cc:2229] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.989741 23135 raft_consensus.cc:2258] T 9301429ace424dc58cbf9b57301e674c P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.993489 23135 tablet_replica.cc:330] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:21.994413 23135 raft_consensus.cc:2229] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:21.995949 23135 raft_consensus.cc:2258] T 986260198b71418c831bfc4617da4603 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:21.999215 23135 tablet_replica.cc:330] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.000106 23135 raft_consensus.cc:2229] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.001080 23135 raft_consensus.cc:2258] T e188e3a6a42340858d5f3ada3f5e867d P 45ab97b763804767846e9d27a01b561f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.004222 23135 tablet_replica.cc:330] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.004868   594 raft_consensus.cc:482] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c22052275034c369b7f7dca086fbd1e)
I20260924 14:32:22.005108 23135 raft_consensus.cc:2229] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.005508   594 raft_consensus.cc:504] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45ab97b763804767846e9d27a01b561f" member_type: VOTER last_known_addr { host: "127.22.151.196" port: 45583 } } peers { permanent_uuid: "7e09fbebe29a49d291d23f9d395fe335" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 38943 } } peers { permanent_uuid: "6c22052275034c369b7f7dca086fbd1e" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 46433 } }
I20260924 14:32:22.006090 23135 raft_consensus.cc:2258] T a24655c8fcef4b9882a0ab16bfc5bbb8 P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.007861   594 leader_election.cc:288] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943), 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433)
I20260924 14:32:22.009518 23135 tablet_replica.cc:330] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.010529 23135 raft_consensus.cc:2229] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.012122 23135 raft_consensus.cc:2258] T d4d74021f6fc4625982decf68a590eac P 45ab97b763804767846e9d27a01b561f [term 10 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:22.014026 26369 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7e09fbebe29a49d291d23f9d395fe335 (127.22.151.193:38943): Network error: Client connection negotiation failed: client connection to 127.22.151.193:38943: connect: Connection refused (error 111)
I20260924 14:32:22.015954 23135 tablet_replica.cc:330] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.016775 23135 raft_consensus.cc:2229] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.017673 23135 raft_consensus.cc:2258] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:22.019191 26371 leader_election.cc:334] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6c22052275034c369b7f7dca086fbd1e (127.22.151.194:46433): Network error: Client connection negotiation failed: client connection to 127.22.151.194:46433: connect: Connection refused (error 111)
I20260924 14:32:22.019904 26371 leader_election.cc:302] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45ab97b763804767846e9d27a01b561f; no voters: 6c22052275034c369b7f7dca086fbd1e, 7e09fbebe29a49d291d23f9d395fe335
I20260924 14:32:22.020498 23135 tablet_replica.cc:330] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
W20260924 14:32:22.020627 26371 raft_consensus.cc:2670] T 0eec7b6dbbd94479aa179fa23fc1f403 P 45ab97b763804767846e9d27a01b561f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260924 14:32:22.021274 23135 raft_consensus.cc:2229] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 16 LEADER]: Raft consensus shutting down.
I20260924 14:32:22.022550 23135 raft_consensus.cc:2258] T 62575580f9ac4403b5f531f616fe41fd P 45ab97b763804767846e9d27a01b561f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.025625 23135 tablet_replica.cc:330] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.026288 23135 raft_consensus.cc:2229] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.027248 23135 raft_consensus.cc:2258] T 014ae451ea994b5e9926dc80215382e4 P 45ab97b763804767846e9d27a01b561f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.030099 23135 tablet_replica.cc:330] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.030804 23135 raft_consensus.cc:2229] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.031461 23135 raft_consensus.cc:2258] T c6e4a72c4886471c8096037b808a6543 P 45ab97b763804767846e9d27a01b561f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.034339 23135 tablet_replica.cc:330] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.035002 23135 raft_consensus.cc:2229] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 20 LEADER]: Raft consensus shutting down.
I20260924 14:32:22.036366 23135 raft_consensus.cc:2258] T fbdb392c304c4e15b1b6c10c61a95323 P 45ab97b763804767846e9d27a01b561f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.039244 23135 tablet_replica.cc:330] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.039966 23135 raft_consensus.cc:2229] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 26 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:22.040710 23135 raft_consensus.cc:2258] T 951fc36414c641b7a4bf7b6b3e6dc3f2 P 45ab97b763804767846e9d27a01b561f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:22.043579 23135 tablet_replica.cc:330] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f: stopping tablet replica
I20260924 14:32:22.044209 23135 raft_consensus.cc:2229] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 LEADER]: Raft consensus shutting down.
I20260924 14:32:22.045374 23135 raft_consensus.cc:2258] T 0fae41ae7fca440da0de40b0ee6973c7 P 45ab97b763804767846e9d27a01b561f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:23.246433 23135 tablet_server.cc:194] TabletServer@127.22.151.196:0 shutdown complete.
I20260924 14:32:23.342757 23135 master.cc:537] Master@127.22.151.254:43789 shutting down...
I20260924 14:32:23.387421 23135 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:32:23.388386 23135 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:23.388871 23135 tablet_replica.cc:330] T 00000000000000000000000000000000 P 277acd3330ed4b40b2cea74b4e602a13: stopping tablet replica
I20260924 14:32:23.604394 23135 master.cc:552] Master@127.22.151.254:43789 shutdown complete.
I20260924 14:32:23.644793 23135 test_util.cc:175] -----------------------------------------------
I20260924 14:32:23.644984 23135 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1790260019751168-23135-0
[  FAILED  ] LeaderRebalancerTest.RestartTserver (208010 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:471: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
[  SKIPPED ] LeaderRebalancerTest.TestMaintenanceMode (6 ms)
[----------] 5 tests from LeaderRebalancerTest (323716 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260924 14:32:23.654342 23135 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.22.151.254:39009
I20260924 14:32:23.655397 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:32:23.661650   603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:23.661876   602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:23.662061   605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:23.663763 23135 server_base.cc:1034] running on GCE node
I20260924 14:32:23.667303 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:32:23.667582 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:32:23.667747 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260343667733 us; error 0 us; skew 500 ppm
I20260924 14:32:23.668323 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:32:23.674548 23135 webserver.cc:460] Webserver started at http://127.22.151.254:35379/ using document root <none> and password file <none>
I20260924 14:32:23.675084 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:32:23.675289 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:32:23.675622 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:32:23.677033 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/master-0-root/instance:
uuid: "8133d61326764837995824b7933a9276"
format_stamp: "Formatted at 2026-09-24 14:32:23 on dist-test-slave-pm5x"
I20260924 14:32:23.681540 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260924 14:32:23.685168   611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:23.686107 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260924 14:32:23.686723 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/master-0-root
uuid: "8133d61326764837995824b7933a9276"
format_stamp: "Formatted at 2026-09-24 14:32:23 on dist-test-slave-pm5x"
I20260924 14:32:23.687019 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:32:23.700438 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:32:23.701584 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:32:23.742934 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.254:39009
I20260924 14:32:23.743116   663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.254:39009 every 8 connection(s)
I20260924 14:32:23.747206   664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:23.759321   664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276: Bootstrap starting.
I20260924 14:32:23.764407   664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:23.769258   664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276: No bootstrap required, opened a new log
I20260924 14:32:23.771606   664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8133d61326764837995824b7933a9276" member_type: VOTER }
I20260924 14:32:23.772063   664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:23.772308   664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8133d61326764837995824b7933a9276, State: Initialized, Role: FOLLOWER
I20260924 14:32:23.772970   664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8133d61326764837995824b7933a9276" member_type: VOTER }
I20260924 14:32:23.773463   664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 14:32:23.773713   664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 14:32:23.774025   664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:23.779119   664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8133d61326764837995824b7933a9276" member_type: VOTER }
I20260924 14:32:23.779791   664 leader_election.cc:302] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8133d61326764837995824b7933a9276; no voters: 
I20260924 14:32:23.781081   664 leader_election.cc:288] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260924 14:32:23.781502   667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:23.783020   667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 1 LEADER]: Becoming Leader. State: Replica: 8133d61326764837995824b7933a9276, State: Running, Role: LEADER
I20260924 14:32:23.783751   667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8133d61326764837995824b7933a9276" member_type: VOTER }
I20260924 14:32:23.784188   664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 14:32:23.789585   668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8133d61326764837995824b7933a9276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8133d61326764837995824b7933a9276" member_type: VOTER } }
I20260924 14:32:23.790692   668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [sys.catalog]: This master's current role is: LEADER
I20260924 14:32:23.789672   669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8133d61326764837995824b7933a9276. Latest consensus state: current_term: 1 leader_uuid: "8133d61326764837995824b7933a9276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8133d61326764837995824b7933a9276" member_type: VOTER } }
I20260924 14:32:23.791576   669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [sys.catalog]: This master's current role is: LEADER
I20260924 14:32:23.797569   674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 14:32:23.806191   674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 14:32:23.807504 23135 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 14:32:23.816481   674 catalog_manager.cc:1347] Generated new cluster ID: 9cfe2e3ca4fc487fac46cdb1f2e199a1
I20260924 14:32:23.816826   674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 14:32:23.840433   674 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 14:32:23.842429   674 catalog_manager.cc:1504] Loading token signing keys...
I20260924 14:32:23.860713   674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276: Generated new TSK 0
I20260924 14:32:23.861385   674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 14:32:23.874625 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:32:23.881137   686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:23.882485   687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:23.885308 23135 server_base.cc:1034] running on GCE node
W20260924 14:32:23.885506   689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:23.887981 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:32:23.888200 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:32:23.888373 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260343888355 us; error 0 us; skew 500 ppm
I20260924 14:32:23.889014 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:32:23.891605 23135 webserver.cc:460] Webserver started at http://127.22.151.193:46163/ using document root <none> and password file <none>
I20260924 14:32:23.892151 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:32:23.892359 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:32:23.892655 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:32:23.893980 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-0-root/instance:
uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
format_stamp: "Formatted at 2026-09-24 14:32:23 on dist-test-slave-pm5x"
I20260924 14:32:23.899107 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260924 14:32:23.903597   694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:23.904496 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260924 14:32:23.904875 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-0-root
uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
format_stamp: "Formatted at 2026-09-24 14:32:23 on dist-test-slave-pm5x"
I20260924 14:32:23.905211 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:32:23.919711 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:32:23.921121 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:32:23.922662 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:32:23.925154 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:32:23.925388 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:23.925645 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:32:23.925840 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:23.972594 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.193:40407
I20260924 14:32:23.972728   756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.193:40407 every 8 connection(s)
I20260924 14:32:23.977901 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:32:23.985184   762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:23.986342   763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:23.990036   766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:23.991094 23135 server_base.cc:1034] running on GCE node
I20260924 14:32:23.994200 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:32:23.994451 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:32:23.994645 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260343994633 us; error 0 us; skew 500 ppm
I20260924 14:32:23.995508 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:32:23.996616   757 heartbeater.cc:347] Connected to a master server at 127.22.151.254:39009
I20260924 14:32:23.997123   757 heartbeater.cc:464] Registering TS with master...
I20260924 14:32:23.997859   757 heartbeater.cc:511] Master 127.22.151.254:39009 requested a full tablet report, sending...
I20260924 14:32:23.998646 23135 webserver.cc:460] Webserver started at http://127.22.151.194:43939/ using document root <none> and password file <none>
I20260924 14:32:23.999294 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:32:23.999550 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:32:23.999856 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:32:24.000217   628 ts_manager.cc:194] Registered new tserver with Master: ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407)
I20260924 14:32:24.001206 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-1-root/instance:
uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
format_stamp: "Formatted at 2026-09-24 14:32:23 on dist-test-slave-pm5x"
I20260924 14:32:24.002363   628 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53826
I20260924 14:32:24.006292 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260924 14:32:24.010438   771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:24.011911 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260924 14:32:24.012310 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-1-root
uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
format_stamp: "Formatted at 2026-09-24 14:32:23 on dist-test-slave-pm5x"
I20260924 14:32:24.012859 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:32:24.043292 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:32:24.044552 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:32:24.046164 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:32:24.048770 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:32:24.049041 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:24.049330 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:32:24.049518 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:24.094127 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.194:39655
I20260924 14:32:24.094270   833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.194:39655 every 8 connection(s)
I20260924 14:32:24.099071 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:32:24.106695   837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:24.106980   838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:24.111954 23135 server_base.cc:1034] running on GCE node
W20260924 14:32:24.112679   840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:24.113933 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260924 14:32:24.114028   834 heartbeater.cc:347] Connected to a master server at 127.22.151.254:39009
W20260924 14:32:24.114225 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:32:24.114663 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260344114639 us; error 0 us; skew 500 ppm
I20260924 14:32:24.114697   834 heartbeater.cc:464] Registering TS with master...
I20260924 14:32:24.115314 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:32:24.115707   834 heartbeater.cc:511] Master 127.22.151.254:39009 requested a full tablet report, sending...
I20260924 14:32:24.118348 23135 webserver.cc:460] Webserver started at http://127.22.151.195:35821/ using document root <none> and password file <none>
I20260924 14:32:24.118422   628 ts_manager.cc:194] Registered new tserver with Master: bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:24.119266 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:32:24.119576 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:32:24.119904 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:32:24.120958   628 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53842
I20260924 14:32:24.121542 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-2-root/instance:
uuid: "4e24a1d2af5241c183194320fdc3915d"
format_stamp: "Formatted at 2026-09-24 14:32:24 on dist-test-slave-pm5x"
I20260924 14:32:24.127063 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260924 14:32:24.130748   845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:24.131631 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260924 14:32:24.131994 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-2-root
uuid: "4e24a1d2af5241c183194320fdc3915d"
format_stamp: "Formatted at 2026-09-24 14:32:24 on dist-test-slave-pm5x"
I20260924 14:32:24.132288 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1790260019751168-23135-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:32:24.173478 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:32:24.174818 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:32:24.176386 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:32:24.178640 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:32:24.178869 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:24.179104 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:32:24.179275 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:24.222985 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.195:33303
I20260924 14:32:24.223120   908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.195:33303 every 8 connection(s)
I20260924 14:32:24.240144   909 heartbeater.cc:347] Connected to a master server at 127.22.151.254:39009
I20260924 14:32:24.240587   909 heartbeater.cc:464] Registering TS with master...
I20260924 14:32:24.241334   909 heartbeater.cc:511] Master 127.22.151.254:39009 requested a full tablet report, sending...
I20260924 14:32:24.243221   628 ts_manager.cc:194] Registered new tserver with Master: 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:24.243739 23135 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017423482s
I20260924 14:32:24.245100   628 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53848
I20260924 14:32:24.246287 23135 test_util.cc:268] Using random seed: 2011156472
I20260924 14:32:24.272256   628 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53854:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260924 14:32:24.275113   628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 14:32:24.366870   722 tablet_service.cc:1467] Processing CreateTablet for tablet 015d56000be4414ca2ba6701f347bef0 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:24.368432   722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 015d56000be4414ca2ba6701f347bef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.368559   872 tablet_service.cc:1467] Processing CreateTablet for tablet 20d0cbc9dc474a7a9f609fcad1afd0af (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:24.370127   872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d0cbc9dc474a7a9f609fcad1afd0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.370574   873 tablet_service.cc:1467] Processing CreateTablet for tablet 015d56000be4414ca2ba6701f347bef0 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:24.372491   871 tablet_service.cc:1467] Processing CreateTablet for tablet d5d35424e0f84f9ebd9f02523e4e9526 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:24.372762   721 tablet_service.cc:1467] Processing CreateTablet for tablet 20d0cbc9dc474a7a9f609fcad1afd0af (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:24.373342   720 tablet_service.cc:1467] Processing CreateTablet for tablet d5d35424e0f84f9ebd9f02523e4e9526 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:24.374215   721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d0cbc9dc474a7a9f609fcad1afd0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.374845   720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5d35424e0f84f9ebd9f02523e4e9526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.374634   870 tablet_service.cc:1467] Processing CreateTablet for tablet 6b5138cabfa044f5847754cfd426e9e5 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:24.376622   869 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc6db8074954aa786d6b4df26708dcb (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:24.372627   868 tablet_service.cc:1467] Processing CreateTablet for tablet cafb25778918400688896e81638526b1 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:24.379014   873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 015d56000be4414ca2ba6701f347bef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.384537   869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc6db8074954aa786d6b4df26708dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.385381   719 tablet_service.cc:1467] Processing CreateTablet for tablet 6b5138cabfa044f5847754cfd426e9e5 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:24.386762   719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b5138cabfa044f5847754cfd426e9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.388208   871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5d35424e0f84f9ebd9f02523e4e9526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.392316   868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cafb25778918400688896e81638526b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.396329   870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b5138cabfa044f5847754cfd426e9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.399633   925 tablet_bootstrap.cc:492] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:24.400441   718 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc6db8074954aa786d6b4df26708dcb (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:24.401901   718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc6db8074954aa786d6b4df26708dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.401958   799 tablet_service.cc:1467] Processing CreateTablet for tablet 015d56000be4414ca2ba6701f347bef0 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:24.405690   798 tablet_service.cc:1467] Processing CreateTablet for tablet 20d0cbc9dc474a7a9f609fcad1afd0af (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:24.407401   925 tablet_bootstrap.cc:654] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.409124   721 tablet_service.cc:1467] Processing CreateTablet for tablet cafb25778918400688896e81638526b1 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:24.409418   797 tablet_service.cc:1467] Processing CreateTablet for tablet d5d35424e0f84f9ebd9f02523e4e9526 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:24.410655   721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cafb25778918400688896e81638526b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.411021   867 tablet_service.cc:1467] Processing CreateTablet for tablet c9fbbb43543c46609c18b6b74d2a0c7d (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:24.412415   867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9fbbb43543c46609c18b6b74d2a0c7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.412645   794 tablet_service.cc:1467] Processing CreateTablet for tablet cafb25778918400688896e81638526b1 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:24.415012   795 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc6db8074954aa786d6b4df26708dcb (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:24.416685   796 tablet_service.cc:1467] Processing CreateTablet for tablet 6b5138cabfa044f5847754cfd426e9e5 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:24.420486   798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d0cbc9dc474a7a9f609fcad1afd0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.421209   866 tablet_service.cc:1467] Processing CreateTablet for tablet 300151d841e346df92555439cc0f57b8 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:24.422544   866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 300151d841e346df92555439cc0f57b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.424779   865 tablet_service.cc:1467] Processing CreateTablet for tablet 7d68635ab99d499696c0d9719093cdf6 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:24.426304   794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cafb25778918400688896e81638526b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.425851   717 tablet_service.cc:1467] Processing CreateTablet for tablet c9fbbb43543c46609c18b6b74d2a0c7d (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:24.427675   717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9fbbb43543c46609c18b6b74d2a0c7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.430011   795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc6db8074954aa786d6b4df26708dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.432116   716 tablet_service.cc:1467] Processing CreateTablet for tablet 300151d841e346df92555439cc0f57b8 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:24.433897   716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 300151d841e346df92555439cc0f57b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.434489   796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b5138cabfa044f5847754cfd426e9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.438540   797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5d35424e0f84f9ebd9f02523e4e9526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.440166   799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 015d56000be4414ca2ba6701f347bef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.440853   793 tablet_service.cc:1467] Processing CreateTablet for tablet c9fbbb43543c46609c18b6b74d2a0c7d (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:24.442175   793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9fbbb43543c46609c18b6b74d2a0c7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.443145   865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d68635ab99d499696c0d9719093cdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.444068   715 tablet_service.cc:1467] Processing CreateTablet for tablet 7d68635ab99d499696c0d9719093cdf6 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:24.445578   715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d68635ab99d499696c0d9719093cdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.455510   792 tablet_service.cc:1467] Processing CreateTablet for tablet 300151d841e346df92555439cc0f57b8 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:24.457535   792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 300151d841e346df92555439cc0f57b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.461678   791 tablet_service.cc:1467] Processing CreateTablet for tablet 7d68635ab99d499696c0d9719093cdf6 (DEFAULT_TABLE table=test-workload [id=e8098043738c4a23be7ab7b95521c354]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:24.463490   791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d68635ab99d499696c0d9719093cdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:24.606431   924 tablet_bootstrap.cc:492] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:24.641166   924 tablet_bootstrap.cc:654] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.668463   925 tablet_bootstrap.cc:492] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:24.668998   925 ts_tablet_manager.cc:1397] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.270s	user 0.056s	sys 0.142s
I20260924 14:32:24.672677   925 raft_consensus.cc:348] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.683359   925 raft_consensus.cc:374] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.683727   925 raft_consensus.cc:729] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.685361   925 consensus_queue.cc:260] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.685922   927 tablet_bootstrap.cc:492] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:24.689917   924 tablet_bootstrap.cc:492] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:24.690539   924 ts_tablet_manager.cc:1397] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.017s
I20260924 14:32:24.693372   924 raft_consensus.cc:348] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.694134   924 raft_consensus.cc:374] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.694432   924 raft_consensus.cc:729] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.700078   924 consensus_queue.cc:260] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.702112   927 tablet_bootstrap.cc:654] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.703783   757 heartbeater.cc:503] Master 127.22.151.254:39009 was elected leader, sending a full tablet report...
I20260924 14:32:24.704228   925 ts_tablet_manager.cc:1428] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.034s	user 0.011s	sys 0.004s
I20260924 14:32:24.705303   925 tablet_bootstrap.cc:492] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:24.712345   924 ts_tablet_manager.cc:1428] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.001s
I20260924 14:32:24.712512   925 tablet_bootstrap.cc:654] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.713449   909 heartbeater.cc:503] Master 127.22.151.254:39009 was elected leader, sending a full tablet report...
I20260924 14:32:24.713877   924 tablet_bootstrap.cc:492] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:24.717180   927 tablet_bootstrap.cc:492] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:24.717654   927 ts_tablet_manager.cc:1397] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.032s	user 0.006s	sys 0.007s
W20260924 14:32:24.727376   758 tablet.cc:2369] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 14:32:24.730472   925 tablet_bootstrap.cc:492] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:24.731112   925 ts_tablet_manager.cc:1397] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.005s
I20260924 14:32:24.731083   927 raft_consensus.cc:348] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.732326   927 raft_consensus.cc:374] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.732664   927 raft_consensus.cc:729] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.732934   924 tablet_bootstrap.cc:654] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.733667   927 consensus_queue.cc:260] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.734398   925 raft_consensus.cc:348] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:24.735226   925 raft_consensus.cc:374] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.735633   925 raft_consensus.cc:729] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.736542   925 consensus_queue.cc:260] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:24.737023   927 ts_tablet_manager.cc:1428] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.019s	user 0.000s	sys 0.013s
I20260924 14:32:24.738070   834 heartbeater.cc:503] Master 127.22.151.254:39009 was elected leader, sending a full tablet report...
I20260924 14:32:24.739717   925 ts_tablet_manager.cc:1428] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:32:24.740770   925 tablet_bootstrap.cc:492] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:24.747797   925 tablet_bootstrap.cc:654] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.738137   927 tablet_bootstrap.cc:492] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:24.754529   924 tablet_bootstrap.cc:492] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:24.755053   924 ts_tablet_manager.cc:1397] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.041s	user 0.007s	sys 0.018s
I20260924 14:32:24.755549   925 tablet_bootstrap.cc:492] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:24.756145   925 ts_tablet_manager.cc:1397] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260924 14:32:24.757776   924 raft_consensus.cc:348] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.758580   924 raft_consensus.cc:374] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.758880   924 raft_consensus.cc:729] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.758780   925 raft_consensus.cc:348] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.759858   925 raft_consensus.cc:374] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.760334   925 raft_consensus.cc:729] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.759807   924 consensus_queue.cc:260] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.761418   927 tablet_bootstrap.cc:654] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.761157   925 consensus_queue.cc:260] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.762528   924 ts_tablet_manager.cc:1428] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:32:24.763700   925 ts_tablet_manager.cc:1428] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:32:24.764844   924 tablet_bootstrap.cc:492] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:24.764846   925 tablet_bootstrap.cc:492] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:24.771164   924 tablet_bootstrap.cc:654] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.771765   925 tablet_bootstrap.cc:654] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.776887   927 tablet_bootstrap.cc:492] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:24.777516   927 ts_tablet_manager.cc:1397] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.040s	user 0.001s	sys 0.020s
I20260924 14:32:24.778172   924 tablet_bootstrap.cc:492] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:24.778880   924 ts_tablet_manager.cc:1397] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.014s	user 0.005s	sys 0.005s
I20260924 14:32:24.780485   927 raft_consensus.cc:348] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.781351   927 raft_consensus.cc:374] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.781679   927 raft_consensus.cc:729] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.782371   924 raft_consensus.cc:348] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.784544   924 raft_consensus.cc:374] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.785778   924 raft_consensus.cc:729] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.786551   924 consensus_queue.cc:260] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.787549   925 tablet_bootstrap.cc:492] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:24.788201   925 ts_tablet_manager.cc:1397] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.008s
I20260924 14:32:24.788707   924 ts_tablet_manager.cc:1428] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.004s
I20260924 14:32:24.782445   927 consensus_queue.cc:260] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.790633   924 tablet_bootstrap.cc:492] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:24.790967   927 ts_tablet_manager.cc:1428] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
I20260924 14:32:24.791153   925 raft_consensus.cc:348] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.791962   927 tablet_bootstrap.cc:492] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:24.791991   925 raft_consensus.cc:374] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.792757   925 raft_consensus.cc:729] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.793922   925 consensus_queue.cc:260] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.797638   925 ts_tablet_manager.cc:1428] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260924 14:32:24.798410   927 tablet_bootstrap.cc:654] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.798571   925 tablet_bootstrap.cc:492] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
W20260924 14:32:24.800905   677 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:32:24.805593   925 tablet_bootstrap.cc:654] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.814291   924 tablet_bootstrap.cc:654] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.814885   925 tablet_bootstrap.cc:492] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:24.815470   925 ts_tablet_manager.cc:1397] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.002s
I20260924 14:32:24.818614   925 raft_consensus.cc:348] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.819589   925 raft_consensus.cc:374] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.819952   925 raft_consensus.cc:729] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.820706   927 tablet_bootstrap.cc:492] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:24.821412   927 ts_tablet_manager.cc:1397] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.006s
I20260924 14:32:24.824710   924 tablet_bootstrap.cc:492] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:24.824370   927 raft_consensus.cc:348] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.825194   927 raft_consensus.cc:374] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.825239   924 ts_tablet_manager.cc:1397] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.005s
I20260924 14:32:24.825583   927 raft_consensus.cc:729] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.825915   929 raft_consensus.cc:482] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:24.826385   929 raft_consensus.cc:504] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.827636   925 consensus_queue.cc:260] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.828244   924 raft_consensus.cc:348] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:24.828830   929 leader_election.cc:288] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:24.829125   924 raft_consensus.cc:374] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.826339   927 consensus_queue.cc:260] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.829599   924 raft_consensus.cc:729] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.831158   927 ts_tablet_manager.cc:1428] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260924 14:32:24.830688   924 consensus_queue.cc:260] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:24.832180   927 tablet_bootstrap.cc:492] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:24.838832   927 tablet_bootstrap.cc:654] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.841493   924 ts_tablet_manager.cc:1428] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260924 14:32:24.842926   924 tablet_bootstrap.cc:492] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:24.866573   924 tablet_bootstrap.cc:654] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.874186   925 ts_tablet_manager.cc:1428] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.058s	user 0.022s	sys 0.024s
I20260924 14:32:24.874584   927 tablet_bootstrap.cc:492] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:24.875211   927 ts_tablet_manager.cc:1397] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.026s
I20260924 14:32:24.875784   925 tablet_bootstrap.cc:492] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:24.878139   927 raft_consensus.cc:348] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.878899   927 raft_consensus.cc:374] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.879177   927 raft_consensus.cc:729] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.880071   927 consensus_queue.cc:260] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.884438   927 ts_tablet_manager.cc:1428] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260924 14:32:24.885540   927 tablet_bootstrap.cc:492] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:24.888034   925 tablet_bootstrap.cc:654] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.891744   927 tablet_bootstrap.cc:654] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.898813   924 tablet_bootstrap.cc:492] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:24.899560   924 ts_tablet_manager.cc:1397] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.011s
I20260924 14:32:24.903282   924 raft_consensus.cc:348] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.904354   924 raft_consensus.cc:374] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.904742   924 raft_consensus.cc:729] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.905591   924 consensus_queue.cc:260] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.909453   924 ts_tablet_manager.cc:1428] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260924 14:32:24.910804   924 tablet_bootstrap.cc:492] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:24.919087   924 tablet_bootstrap.cc:654] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.931895   924 tablet_bootstrap.cc:492] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:24.931890   809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300151d841e346df92555439cc0f57b8" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:24.932809   924 ts_tablet_manager.cc:1397] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.003s
W20260924 14:32:24.934235   696 leader_election.cc:341] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:24.936023   883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300151d841e346df92555439cc0f57b8" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:24.936782   883 raft_consensus.cc:2454] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 0.
I20260924 14:32:24.937332   924 raft_consensus.cc:348] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.938186   698 leader_election.cc:302] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:24.938189   924 raft_consensus.cc:374] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.938813   924 raft_consensus.cc:729] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.939213   943 raft_consensus.cc:2781] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:24.939728   943 raft_consensus.cc:482] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:24.939837   925 tablet_bootstrap.cc:492] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:24.940356   943 raft_consensus.cc:3037] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:24.939754   924 consensus_queue.cc:260] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.940642   925 ts_tablet_manager.cc:1397] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.065s	user 0.028s	sys 0.025s
I20260924 14:32:24.940925   927 tablet_bootstrap.cc:492] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:24.941471   927 ts_tablet_manager.cc:1397] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.056s	user 0.020s	sys 0.032s
I20260924 14:32:24.943547   925 raft_consensus.cc:348] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.944355   925 raft_consensus.cc:374] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.944192   927 raft_consensus.cc:348] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.945125   927 raft_consensus.cc:374] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.945152   924 ts_tablet_manager.cc:1428] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260924 14:32:24.945720   927 raft_consensus.cc:729] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.946606   924 tablet_bootstrap.cc:492] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:24.946621   927 consensus_queue.cc:260] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.950263   943 raft_consensus.cc:504] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.944744   925 raft_consensus.cc:729] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.952344   927 ts_tablet_manager.cc:1428] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260924 14:32:24.953387   809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300151d841e346df92555439cc0f57b8" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:24.960429   927 tablet_bootstrap.cc:492] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:24.960613   933 raft_consensus.cc:482] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:24.959889   925 consensus_queue.cc:260] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.961434   933 raft_consensus.cc:504] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.967059   925 ts_tablet_manager.cc:1428] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.002s
I20260924 14:32:24.967983   927 tablet_bootstrap.cc:654] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.968596   925 tablet_bootstrap.cc:492] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:24.970388   933 leader_election.cc:288] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:24.952502   943 leader_election.cc:288] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 election: Requested vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:24.976725   930 raft_consensus.cc:482] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:24.977429   930 raft_consensus.cc:504] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:24.979682   927 tablet_bootstrap.cc:492] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:24.980237   927 ts_tablet_manager.cc:1397] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260924 14:32:24.981896   930 leader_election.cc:288] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:24.983039   927 raft_consensus.cc:348] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.984328   927 raft_consensus.cc:374] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:24.984735   927 raft_consensus.cc:729] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:24.985705   927 consensus_queue.cc:260] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:24.976675   925 tablet_bootstrap.cc:654] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:24.988554   809 raft_consensus.cc:3037] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:24.953641   883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300151d841e346df92555439cc0f57b8" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:24.989049   927 ts_tablet_manager.cc:1428] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260924 14:32:24.989228   883 raft_consensus.cc:3037] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:24.990515   927 tablet_bootstrap.cc:492] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:24.999128   927 tablet_bootstrap.cc:654] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:25.000946   924 tablet_bootstrap.cc:654] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:25.005126   927 tablet_bootstrap.cc:492] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:25.005641   927 ts_tablet_manager.cc:1397] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260924 14:32:25.006219   883 raft_consensus.cc:2454] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 1.
I20260924 14:32:25.008002   927 raft_consensus.cc:348] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.008623   927 raft_consensus.cc:374] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:25.008863   927 raft_consensus.cc:729] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:25.009444   927 consensus_queue.cc:260] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.011240   927 ts_tablet_manager.cc:1428] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260924 14:32:25.012056   927 tablet_bootstrap.cc:492] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:25.016943   927 tablet_bootstrap.cc:654] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:25.018327   943 raft_consensus.cc:482] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:25.018913   943 raft_consensus.cc:504] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.021082   943 leader_election.cc:288] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.021705   933 raft_consensus.cc:482] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:25.022469   933 raft_consensus.cc:504] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.024993   933 leader_election.cc:288] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.027115   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc6db8074954aa786d6b4df26708dcb" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:25.027952   808 raft_consensus.cc:2454] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 0.
I20260924 14:32:25.028980   696 leader_election.cc:302] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.029940   943 raft_consensus.cc:2781] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:25.030323   943 raft_consensus.cc:482] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:25.030648   943 raft_consensus.cc:3037] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.030710   698 leader_election.cc:302] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:25.035897   929 raft_consensus.cc:2781] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:25.037977   883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc6db8074954aa786d6b4df26708dcb" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:25.038771   883 raft_consensus.cc:2454] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 0.
I20260924 14:32:25.050660   809 raft_consensus.cc:2454] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 1.
I20260924 14:32:25.062976   943 raft_consensus.cc:504] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.068984   809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015d56000be4414ca2ba6701f347bef0" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:25.069949   925 tablet_bootstrap.cc:492] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:25.072275   927 tablet_bootstrap.cc:492] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:25.072332   924 tablet_bootstrap.cc:492] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:25.075284   962 raft_consensus.cc:482] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:25.076114   924 ts_tablet_manager.cc:1397] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.130s	user 0.028s	sys 0.006s
I20260924 14:32:25.076752   927 ts_tablet_manager.cc:1397] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.065s	user 0.023s	sys 0.024s
I20260924 14:32:25.079662   927 raft_consensus.cc:348] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:25.080358   927 raft_consensus.cc:374] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:25.080626   927 raft_consensus.cc:729] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:25.081314   925 ts_tablet_manager.cc:1397] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.113s	user 0.017s	sys 0.008s
I20260924 14:32:25.081238   927 consensus_queue.cc:260] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:25.083427   927 ts_tablet_manager.cc:1428] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260924 14:32:25.073580   929 raft_consensus.cc:686] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:25.081900   943 leader_election.cc:288] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 election: Requested vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.075841   962 raft_consensus.cc:504] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
W20260924 14:32:25.086247   847 leader_election.cc:341] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:25.086441   933 raft_consensus.cc:482] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:25.086953   933 raft_consensus.cc:504] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.086241   925 raft_consensus.cc:348] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.088030   925 raft_consensus.cc:374] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:25.088352   925 raft_consensus.cc:729] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:25.089313   962 leader_election.cc:288] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.085002   924 raft_consensus.cc:348] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.089409   933 leader_election.cc:288] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.090571   924 raft_consensus.cc:374] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:25.091090   924 raft_consensus.cc:729] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:25.084909   929 consensus_queue.cc:237] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.092192   924 consensus_queue.cc:260] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.092736   732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cafb25778918400688896e81638526b1" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:25.094048   925 consensus_queue.cc:260] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.094483   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b5138cabfa044f5847754cfd426e9e5" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:25.095191   732 raft_consensus.cc:2454] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:25.095252   731 raft_consensus.cc:2454] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:25.096601   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:25.097143   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:25.097507   927 tablet_bootstrap.cc:492] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:25.097867   731 raft_consensus.cc:2454] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:25.097463   808 raft_consensus.cc:2454] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 0.
I20260924 14:32:25.099326   809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc6db8074954aa786d6b4df26708dcb" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:25.095320   924 ts_tablet_manager.cc:1428] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260924 14:32:25.100957   924 tablet_bootstrap.cc:492] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:25.101137   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:25.101577   809 raft_consensus.cc:3037] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.101862   882 raft_consensus.cc:2454] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 0.
I20260924 14:32:25.099675   883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc6db8074954aa786d6b4df26708dcb" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:25.103351   883 raft_consensus.cc:3037] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.103719   696 leader_election.cc:302] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.104835   963 raft_consensus.cc:2781] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:25.105239   963 raft_consensus.cc:482] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:25.105793   774 leader_election.cc:302] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.106871   933 raft_consensus.cc:2781] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:25.107235   933 raft_consensus.cc:482] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:25.107604   933 raft_consensus.cc:3037] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.109968   924 tablet_bootstrap.cc:654] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:25.114034   927 tablet_bootstrap.cc:654] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:25.114405   774 leader_election.cc:302] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.117113   930 raft_consensus.cc:482] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:25.117673   930 raft_consensus.cc:504] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.105566   963 raft_consensus.cc:3037] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.120615   930 leader_election.cc:288] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:25.124927   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d35424e0f84f9ebd9f02523e4e9526" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:25.125788   808 raft_consensus.cc:2454] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 0.
I20260924 14:32:25.125217   933 raft_consensus.cc:504] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.126832   847 leader_election.cc:302] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.127642   925 ts_tablet_manager.cc:1428] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.001s
I20260924 14:32:25.127779   930 raft_consensus.cc:2781] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:25.128244   930 raft_consensus.cc:482] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:25.128592   930 raft_consensus.cc:3037] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.130774   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015d56000be4414ca2ba6701f347bef0" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:25.131556   731 raft_consensus.cc:2454] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 0.
I20260924 14:32:25.132336   732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d35424e0f84f9ebd9f02523e4e9526" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:25.132759   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cafb25778918400688896e81638526b1" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:25.133112   732 raft_consensus.cc:2454] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 0.
I20260924 14:32:25.134559   925 tablet_bootstrap.cc:492] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:25.134896   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b5138cabfa044f5847754cfd426e9e5" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:25.135572   882 raft_consensus.cc:2454] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:25.136056   848 leader_election.cc:302] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:25.141212   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:25.144271   882 raft_consensus.cc:2454] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:25.154742   883 raft_consensus.cc:2454] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 1.
I20260924 14:32:25.156662   698 leader_election.cc:302] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:25.159329   929 raft_consensus.cc:2781] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:25.165355   924 tablet_bootstrap.cc:492] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:25.170251   924 ts_tablet_manager.cc:1397] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.004s
W20260924 14:32:25.158844   774 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:40407, user_credentials={real_user=slave}} blocked reactor thread for 44565.5us
I20260924 14:32:25.165982   628 catalog_manager.cc:5495] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
W20260924 14:32:25.162187   775 leader_election.cc:341] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:25.175974   732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b5138cabfa044f5847754cfd426e9e5" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:25.176685   732 raft_consensus.cc:3037] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.177629   927 tablet_bootstrap.cc:492] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:25.178333   927 ts_tablet_manager.cc:1397] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.081s	user 0.027s	sys 0.010s
I20260924 14:32:25.169550   929 raft_consensus.cc:686] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:25.173403   933 leader_election.cc:288] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.180215   965 raft_consensus.cc:2781] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260924 14:32:25.180464   848 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:40407, user_credentials={real_user=slave}} blocked reactor thread for 44652.5us
I20260924 14:32:25.189154   927 raft_consensus.cc:348] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.190037   809 raft_consensus.cc:2454] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 1.
I20260924 14:32:25.190171   927 raft_consensus.cc:374] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:25.190685   965 raft_consensus.cc:482] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:25.190887   927 raft_consensus.cc:729] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:25.191428   933 raft_consensus.cc:482] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:25.192160   927 consensus_queue.cc:260] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.192265   933 raft_consensus.cc:504] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.194553   927 ts_tablet_manager.cc:1428] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260924 14:32:25.195218   933 leader_election.cc:288] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.198669   963 raft_consensus.cc:504] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.177299   775 leader_election.cc:302] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.203390   925 tablet_bootstrap.cc:654] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:25.204097   929 consensus_queue.cc:237] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.216687   968 raft_consensus.cc:482] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:25.210078   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:25.217556   968 raft_consensus.cc:504] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.218616   882 raft_consensus.cc:2454] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:25.211735   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:25.219555   881 raft_consensus.cc:3037] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.220047   965 raft_consensus.cc:3037] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.212862   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:25.224648   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:25.224984   968 leader_election.cc:288] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:25.212929   930 raft_consensus.cc:504] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
W20260924 14:32:25.226114   848 leader_election.cc:341] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:25.210866   883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b5138cabfa044f5847754cfd426e9e5" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:25.228349   883 raft_consensus.cc:3037] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.191471   732 raft_consensus.cc:2454] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:25.212747   924 raft_consensus.cc:348] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.214262   809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:25.215544   963 leader_election.cc:288] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 election: Requested vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.228649   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d35424e0f84f9ebd9f02523e4e9526" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:25.230185   730 raft_consensus.cc:3037] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.233666   930 leader_election.cc:288] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:25.221537   775 leader_election.cc:302] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.229447   924 raft_consensus.cc:374] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:25.238134   933 raft_consensus.cc:2781] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:25.238581   933 raft_consensus.cc:482] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:25.238916   933 raft_consensus.cc:3037] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.239404   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:25.240007   807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d35424e0f84f9ebd9f02523e4e9526" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:25.241448   807 raft_consensus.cc:3037] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.243070   965 raft_consensus.cc:504] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.208313   971 raft_consensus.cc:2781] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:25.245412   971 raft_consensus.cc:482] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:25.237885   924 raft_consensus.cc:729] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:25.245645   965 leader_election.cc:288] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.246692   924 consensus_queue.cc:260] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
W20260924 14:32:25.247967   774 leader_election.cc:341] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:25.246942   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cafb25778918400688896e81638526b1" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:25.246938   732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cafb25778918400688896e81638526b1" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:25.249094   774 leader_election.cc:302] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.249423   732 raft_consensus.cc:3037] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.249886   933 raft_consensus.cc:504] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.251530   881 raft_consensus.cc:2454] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 1.
I20260924 14:32:25.245790   971 raft_consensus.cc:3037] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.252892   698 leader_election.cc:302] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:25.254786   963 raft_consensus.cc:2781] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:25.254999   883 raft_consensus.cc:2454] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:25.255294   963 raft_consensus.cc:686] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:25.249094   924 ts_tablet_manager.cc:1428] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.000s
I20260924 14:32:25.256263   963 consensus_queue.cc:237] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.256668   965 raft_consensus.cc:2781] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:25.258780   732 raft_consensus.cc:2454] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:25.261248   933 leader_election.cc:288] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.261909   808 raft_consensus.cc:2379] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e24a1d2af5241c183194320fdc3915d in current term 1: Already voted for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in this term.
I20260924 14:32:25.262063   730 raft_consensus.cc:2454] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 1.
I20260924 14:32:25.263346   848 leader_election.cc:302] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:25.264219   930 raft_consensus.cc:2781] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election won for term 1
W20260924 14:32:25.264267   775 leader_election.cc:341] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:25.264865   930 raft_consensus.cc:686] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Becoming Leader. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:25.265205   847 leader_election.cc:302] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d; no voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:25.265801   930 consensus_queue.cc:237] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.266767   968 raft_consensus.cc:3037] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.270443   732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:25.271586   971 raft_consensus.cc:504] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:25.272528   965 raft_consensus.cc:686] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:25.274042   965 consensus_queue.cc:237] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.275800   883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:25.274565   732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015d56000be4414ca2ba6701f347bef0" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:25.275954   971 leader_election.cc:288] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:25.276691   732 raft_consensus.cc:3037] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.209015   969 raft_consensus.cc:2781] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:25.280121   969 raft_consensus.cc:482] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:25.280524   969 raft_consensus.cc:3037] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.282420   807 raft_consensus.cc:2454] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 1.
I20260924 14:32:25.265226   774 leader_election.cc:302] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 4e24a1d2af5241c183194320fdc3915d
I20260924 14:32:25.289068   975 raft_consensus.cc:2781] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:25.290280   975 raft_consensus.cc:686] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:25.291627   975 consensus_queue.cc:237] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.299687   925 tablet_bootstrap.cc:492] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:25.298679   628 catalog_manager.cc:5495] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e24a1d2af5241c183194320fdc3915d (127.22.151.195). New cstate: current_term: 1 leader_uuid: "4e24a1d2af5241c183194320fdc3915d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:25.301440   968 raft_consensus.cc:2726] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260924 14:32:25.303402   774 leader_election.cc:341] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:25.256742   924 tablet_bootstrap.cc:492] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:25.307741   732 raft_consensus.cc:2454] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 1.
I20260924 14:32:25.309242   848 leader_election.cc:302] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:25.310238   968 raft_consensus.cc:2781] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:25.310942   968 raft_consensus.cc:686] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Becoming Leader. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
W20260924 14:32:25.312107   847 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.194:39655, user_credentials={real_user=slave}} blocked reactor thread for 47010.7us
I20260924 14:32:25.311935   968 consensus_queue.cc:237] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:25.314631   807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015d56000be4414ca2ba6701f347bef0" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:25.315352   807 raft_consensus.cc:3037] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.300741   925 ts_tablet_manager.cc:1397] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.166s	user 0.015s	sys 0.009s
I20260924 14:32:25.324084   807 raft_consensus.cc:2454] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 1.
I20260924 14:32:25.328006   628 catalog_manager.cc:5495] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e24a1d2af5241c183194320fdc3915d (127.22.151.195). New cstate: current_term: 1 leader_uuid: "4e24a1d2af5241c183194320fdc3915d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:25.329943   627 catalog_manager.cc:5495] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:25.330757   925 raft_consensus.cc:348] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.331638   925 raft_consensus.cc:374] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:25.331929   925 raft_consensus.cc:729] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:25.332510   969 raft_consensus.cc:504] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.333808   809 raft_consensus.cc:2379] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ace4fb4d0d594edfb5ceb70d49a01d21 in current term 1: Already voted for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in this term.
I20260924 14:32:25.335330   732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:25.336519   924 tablet_bootstrap.cc:654] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:25.337113   969 leader_election.cc:288] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.338285   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:25.339107   881 raft_consensus.cc:2379] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bf5ce16ee2ca4b4e9ef50f91febaffde in current term 1: Already voted for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in this term.
I20260924 14:32:25.339665   625 catalog_manager.cc:5495] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 0 to 1, leader changed from <none> to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 1 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:25.340358   775 leader_election.cc:302] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:25.340581   883 raft_consensus.cc:2454] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:25.339730   925 consensus_queue.cc:260] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.342433   775 leader_election.cc:302] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:25.343218   925 ts_tablet_manager.cc:1428] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260924 14:32:25.343627   965 raft_consensus.cc:2726] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 14:32:25.344283   965 raft_consensus.cc:2781] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:25.344791   965 raft_consensus.cc:686] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:25.331784   627 catalog_manager.cc:5495] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:25.345580   965 consensus_queue.cc:237] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.344230   925 tablet_bootstrap.cc:492] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:25.353008   924 tablet_bootstrap.cc:492] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:25.353552   924 ts_tablet_manager.cc:1397] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.097s	user 0.009s	sys 0.002s
I20260924 14:32:25.355031   925 tablet_bootstrap.cc:654] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:25.358793   625 catalog_manager.cc:5495] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 0 to 1, leader changed from <none> to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 1 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:25.355783   924 raft_consensus.cc:348] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.361019   924 raft_consensus.cc:374] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:25.361366   924 raft_consensus.cc:729] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:25.362077   924 consensus_queue.cc:260] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.364336   924 ts_tablet_manager.cc:1428] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260924 14:32:25.364995   625 catalog_manager.cc:5495] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 0 to 1, leader changed from <none> to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 1 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:25.365989   925 tablet_bootstrap.cc:492] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:25.366549   925 ts_tablet_manager.cc:1397] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.001s
I20260924 14:32:25.369164   925 raft_consensus.cc:348] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.369782   925 raft_consensus.cc:374] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:25.370004   925 raft_consensus.cc:729] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:25.370539   925 consensus_queue.cc:260] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.372531   925 ts_tablet_manager.cc:1428] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260924 14:32:25.532078   929 consensus_queue.cc:1035] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:25.543614   929 consensus_queue.cc:1035] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:25.563704   968 raft_consensus.cc:482] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:25.563983   965 raft_consensus.cc:482] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:25.564419   968 raft_consensus.cc:504] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.564529   965 raft_consensus.cc:504] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.566574   965 leader_election.cc:288] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.566574   968 leader_election.cc:288] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:25.567955   883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fbbb43543c46609c18b6b74d2a0c7d" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:25.567889   732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fbbb43543c46609c18b6b74d2a0c7d" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:25.567932   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fbbb43543c46609c18b6b74d2a0c7d" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:25.567996   809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fbbb43543c46609c18b6b74d2a0c7d" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:25.569023   883 raft_consensus.cc:2454] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:25.569077   732 raft_consensus.cc:2454] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:25.569526   809 raft_consensus.cc:2454] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 0.
I20260924 14:32:25.570093   775 leader_election.cc:302] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.570973   965 raft_consensus.cc:2781] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:25.571327   965 raft_consensus.cc:482] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:25.571688   965 raft_consensus.cc:3037] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.571607   847 leader_election.cc:302] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:25.572494   968 raft_consensus.cc:2781] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:25.572865   968 raft_consensus.cc:482] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:25.573118   968 raft_consensus.cc:3037] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.577080   965 raft_consensus.cc:504] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.578295   968 raft_consensus.cc:504] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.578889   965 leader_election.cc:288] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.579877   732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fbbb43543c46609c18b6b74d2a0c7d" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:25.580062   883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fbbb43543c46609c18b6b74d2a0c7d" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:25.580570   732 raft_consensus.cc:3037] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.580856   883 raft_consensus.cc:2379] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bf5ce16ee2ca4b4e9ef50f91febaffde in current term 1: Already voted for candidate 4e24a1d2af5241c183194320fdc3915d in this term.
I20260924 14:32:25.580906   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fbbb43543c46609c18b6b74d2a0c7d" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:25.580935   968 leader_election.cc:288] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:25.584548   809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fbbb43543c46609c18b6b74d2a0c7d" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:25.585230   809 raft_consensus.cc:2379] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e24a1d2af5241c183194320fdc3915d in current term 1: Already voted for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in this term.
I20260924 14:32:25.586074   847 leader_election.cc:302] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d; no voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:25.587005   968 raft_consensus.cc:2726] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 14:32:25.588888   732 raft_consensus.cc:2454] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:25.589939   774 leader_election.cc:302] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 4e24a1d2af5241c183194320fdc3915d
I20260924 14:32:25.590710   965 raft_consensus.cc:2781] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:25.591221   965 raft_consensus.cc:686] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:25.592013   965 consensus_queue.cc:237] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.599980   627 catalog_manager.cc:5495] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 0 to 1, leader changed from <none> to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 1 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:25.693207   929 consensus_queue.cc:1035] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:25.694204   963 consensus_queue.cc:1035] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:25.700718   963 consensus_queue.cc:1035] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:25.703907   963 raft_consensus.cc:482] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:25.704433   963 raft_consensus.cc:504] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.707935   963 leader_election.cc:288] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.709142   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:25.709403   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:25.710448   881 raft_consensus.cc:2379] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ace4fb4d0d594edfb5ceb70d49a01d21 in current term 1: Already voted for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in this term.
I20260924 14:32:25.718869   965 consensus_queue.cc:1035] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:25.731089   965 consensus_queue.cc:1035] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:25.737339   968 consensus_queue.cc:1035] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:25.737673   696 leader_election.cc:302] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:25.741134   963 raft_consensus.cc:3037] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.751101   969 consensus_queue.cc:1035] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:25.771977   881 raft_consensus.cc:3037] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:25.794450   968 consensus_queue.cc:1035] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:25.803725   975 consensus_queue.cc:1035] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:25.803556   929 consensus_queue.cc:1035] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260924 14:32:25.812919   975 consensus_queue.cc:1035] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:25.819141   963 raft_consensus.cc:2726] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:25.852541   883 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "015d56000be4414ca2ba6701f347bef0"
dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
mode: GRACEFUL
new_leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:41862
I20260924 14:32:25.853439   883 raft_consensus.cc:595] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Received request to transfer leadership to TS ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:25.863605   969 consensus_queue.cc:1035] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:25.866653   882 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d5d35424e0f84f9ebd9f02523e4e9526"
dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
mode: GRACEFUL
new_leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:41862
I20260924 14:32:25.867578   882 raft_consensus.cc:595] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Received request to transfer leadership to TS ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:25.874047   968 raft_consensus.cc:979] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d: : Instructing follower ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:25.874536   968 raft_consensus.cc:1067] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Signalling peer ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:25.877213   731 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d5d35424e0f84f9ebd9f02523e4e9526"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:33380
I20260924 14:32:25.877840   731 raft_consensus.cc:482] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:25.878232   731 raft_consensus.cc:3037] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:25.901728   731 raft_consensus.cc:504] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.905220   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d35424e0f84f9ebd9f02523e4e9526" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:25.905398   732 raft_consensus.cc:1226] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Rejecting Update request from peer 4e24a1d2af5241c183194320fdc3915d for earlier term 1. Current term is 2. Ops: []
I20260924 14:32:25.905994   806 raft_consensus.cc:3037] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:25.909158   968 consensus_queue.cc:1046] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:25.910135   968 raft_consensus.cc:3032] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:25.910459   968 raft_consensus.cc:729] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:25.911883   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d35424e0f84f9ebd9f02523e4e9526" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:25.911307   968 consensus_queue.cc:260] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.913341   968 raft_consensus.cc:3037] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:25.905221   731 leader_election.cc:288] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Requested vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:25.917104   806 raft_consensus.cc:2454] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:25.919049   696 leader_election.cc:302] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.919996   997 raft_consensus.cc:2781] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:25.920463   997 raft_consensus.cc:686] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:25.921322   997 consensus_queue.cc:237] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:25.922340   808 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6b5138cabfa044f5847754cfd426e9e5"
dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
mode: GRACEFUL
new_leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:47042
I20260924 14:32:25.923015   808 raft_consensus.cc:595] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Received request to transfer leadership to TS ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:25.931043   968 consensus_queue.cc:1035] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:25.936386   968 consensus_queue.cc:1035] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:25.937490   881 raft_consensus.cc:2454] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:25.937914   628 catalog_manager.cc:5495] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 1 to 2, leader changed from 4e24a1d2af5241c183194320fdc3915d (127.22.151.195) to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:25.949472   731 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2bc6db8074954aa786d6b4df26708dcb"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
mode: GRACEFUL
new_leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
 from {username='slave'} at 127.0.0.1:33394
I20260924 14:32:25.950177   731 raft_consensus.cc:595] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Received request to transfer leadership to TS bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:25.971015  1008 raft_consensus.cc:979] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d: : Instructing follower ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:25.971746  1017 raft_consensus.cc:1067] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Signalling peer ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:25.973980   730 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "015d56000be4414ca2ba6701f347bef0"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:33380
I20260924 14:32:25.974596   730 raft_consensus.cc:482] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:25.974979   730 raft_consensus.cc:3037] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:25.978058   731 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "300151d841e346df92555439cc0f57b8"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
mode: GRACEFUL
new_leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
 from {username='slave'} at 127.0.0.1:33394
I20260924 14:32:25.978674   731 raft_consensus.cc:595] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Received request to transfer leadership to TS bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:25.982935   730 raft_consensus.cc:504] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:25.984715   806 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6"
dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
mode: GRACEFUL
new_leader_uuid: "4e24a1d2af5241c183194320fdc3915d"
 from {username='slave'} at 127.0.0.1:47042
I20260924 14:32:25.984957   730 leader_election.cc:288] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Requested vote from peers 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:25.985512   806 raft_consensus.cc:595] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Received request to transfer leadership to TS 4e24a1d2af5241c183194320fdc3915d
I20260924 14:32:25.985658   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015d56000be4414ca2ba6701f347bef0" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:25.986053   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015d56000be4414ca2ba6701f347bef0" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:25.986454   881 raft_consensus.cc:3032] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:25.986861   881 raft_consensus.cc:729] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:25.986651   808 raft_consensus.cc:3037] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:25.987962   881 consensus_queue.cc:260] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:25.989519   969 consensus_queue.cc:1035] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:25.989886   881 raft_consensus.cc:3037] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:25.995608   808 raft_consensus.cc:2454] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:25.996912   696 leader_election.cc:302] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:25.997807   997 raft_consensus.cc:2781] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:25.998283   997 raft_consensus.cc:686] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:25.999126   997 consensus_queue.cc:237] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } }
I20260924 14:32:26.001844   975 consensus_queue.cc:1035] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:26.003288   881 raft_consensus.cc:2454] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:26.011137   626 catalog_manager.cc:5495] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 1 to 2, leader changed from 4e24a1d2af5241c183194320fdc3915d (127.22.151.195) to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:26.078740  1006 raft_consensus.cc:979] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21: : Instructing follower bf5ce16ee2ca4b4e9ef50f91febaffde to start an election
I20260924 14:32:26.079319   997 raft_consensus.cc:1067] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Signalling peer bf5ce16ee2ca4b4e9ef50f91febaffde to start an election
I20260924 14:32:26.080825   808 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "300151d841e346df92555439cc0f57b8"
dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
 from {username='slave'} at 127.0.0.1:47026
I20260924 14:32:26.081357   808 raft_consensus.cc:482] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:26.081645   808 raft_consensus.cc:3037] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.088279   808 raft_consensus.cc:504] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.090222   808 leader_election.cc:288] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 2 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:26.091125   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300151d841e346df92555439cc0f57b8" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:26.091355   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300151d841e346df92555439cc0f57b8" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:26.091948   730 raft_consensus.cc:3032] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:26.092044   882 raft_consensus.cc:3037] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.092271   730 raft_consensus.cc:729] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:26.093031   730 consensus_queue.cc:260] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.094071   730 raft_consensus.cc:3037] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.098534   882 raft_consensus.cc:2454] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 2.
I20260924 14:32:26.099359   730 raft_consensus.cc:2454] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 2.
I20260924 14:32:26.099803   775 leader_election.cc:302] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:26.100750   975 raft_consensus.cc:2781] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:26.101169   975 raft_consensus.cc:686] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:26.101804   975 consensus_queue.cc:237] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.108085   625 catalog_manager.cc:5495] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 1 to 2, leader changed from ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193) to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 2 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:26.154484   965 raft_consensus.cc:979] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde: : Instructing follower ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:26.154945   975 raft_consensus.cc:1067] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Signalling peer ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:26.156566   730 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6b5138cabfa044f5847754cfd426e9e5"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:33374
I20260924 14:32:26.157117   730 raft_consensus.cc:482] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:26.157370   730 raft_consensus.cc:3037] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.162706   730 raft_consensus.cc:504] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.164446   730 leader_election.cc:288] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Requested vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:26.165598   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b5138cabfa044f5847754cfd426e9e5" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:26.165452   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b5138cabfa044f5847754cfd426e9e5" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:26.166337   882 raft_consensus.cc:3037] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.166445   808 raft_consensus.cc:3032] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:26.166857   808 raft_consensus.cc:729] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:26.167641   808 consensus_queue.cc:260] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.168673   808 raft_consensus.cc:3037] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.171772   882 raft_consensus.cc:2454] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:26.172829   698 leader_election.cc:302] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:26.173570   997 raft_consensus.cc:2781] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:26.174079   997 raft_consensus.cc:686] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:26.174170   808 raft_consensus.cc:2454] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:26.174954   997 consensus_queue.cc:237] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.182855   625 catalog_manager.cc:5495] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 1 to 2, leader changed from bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194) to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:26.252395  1006 raft_consensus.cc:979] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21: : Instructing follower bf5ce16ee2ca4b4e9ef50f91febaffde to start an election
I20260924 14:32:26.252867   997 raft_consensus.cc:1067] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Signalling peer bf5ce16ee2ca4b4e9ef50f91febaffde to start an election
I20260924 14:32:26.254257   808 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2bc6db8074954aa786d6b4df26708dcb"
dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
 from {username='slave'} at 127.0.0.1:47026
I20260924 14:32:26.254823   808 raft_consensus.cc:482] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:26.255148   808 raft_consensus.cc:3037] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.260171   808 raft_consensus.cc:504] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.261710   808 leader_election.cc:288] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 2 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:26.262804   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc6db8074954aa786d6b4df26708dcb" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:26.262876   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc6db8074954aa786d6b4df26708dcb" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:26.263577   730 raft_consensus.cc:3032] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:26.263644   882 raft_consensus.cc:3037] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.263927   730 raft_consensus.cc:729] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:26.264873   730 consensus_queue.cc:260] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.266243   730 raft_consensus.cc:3037] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.269145   882 raft_consensus.cc:2454] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 2.
I20260924 14:32:26.270226   775 leader_election.cc:302] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:26.270929   975 raft_consensus.cc:2781] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:26.271409   975 raft_consensus.cc:686] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:26.271973   730 raft_consensus.cc:2454] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 2.
I20260924 14:32:26.272312   975 consensus_queue.cc:237] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.279297   625 catalog_manager.cc:5495] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 1 to 2, leader changed from ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193) to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 2 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:26.367605   975 raft_consensus.cc:979] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde: : Instructing follower 4e24a1d2af5241c183194320fdc3915d to start an election
I20260924 14:32:26.368042   975 raft_consensus.cc:1067] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Signalling peer 4e24a1d2af5241c183194320fdc3915d to start an election
I20260924 14:32:26.369453   882 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6"
dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
 from {username='slave'} at 127.0.0.1:41848
I20260924 14:32:26.369937   882 raft_consensus.cc:482] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:26.370229   882 raft_consensus.cc:3037] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.375202   882 raft_consensus.cc:504] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.376914   882 leader_election.cc:288] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 2 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:26.377862   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:26.378021   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d68635ab99d499696c0d9719093cdf6" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:26.378458   730 raft_consensus.cc:3037] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.378655   808 raft_consensus.cc:3032] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:26.379083   808 raft_consensus.cc:729] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:26.379941   808 consensus_queue.cc:260] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.381274   808 raft_consensus.cc:3037] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:26.384507   730 raft_consensus.cc:2454] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 2.
I20260924 14:32:26.385658   848 leader_election.cc:302] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:26.386523  1017 raft_consensus.cc:2781] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:26.386787   808 raft_consensus.cc:2454] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 2.
I20260924 14:32:26.386920  1017 raft_consensus.cc:686] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 2 LEADER]: Becoming Leader. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:26.388016  1017 consensus_queue.cc:237] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:26.394500   625 catalog_manager.cc:5495] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d reported cstate change: term changed from 1 to 2, leader changed from bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194) to 4e24a1d2af5241c183194320fdc3915d (127.22.151.195). New cstate: current_term: 2 leader_uuid: "4e24a1d2af5241c183194320fdc3915d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:26.428247   808 raft_consensus.cc:1261] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.429486   963 consensus_queue.cc:1035] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:26.437626   882 raft_consensus.cc:1261] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.439240  1020 consensus_queue.cc:1035] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:26.538975   882 raft_consensus.cc:1261] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.540382  1020 consensus_queue.cc:1035] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:26.549037   808 raft_consensus.cc:1261] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.550675   963 consensus_queue.cc:1035] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:26.580590   882 raft_consensus.cc:1261] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.581856   963 consensus_queue.cc:1035] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:26.585500   730 raft_consensus.cc:1261] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Refusing update from remote peer bf5ce16ee2ca4b4e9ef50f91febaffde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.587096   975 consensus_queue.cc:1035] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:26.590747   808 raft_consensus.cc:1261] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.593245   997 consensus_queue.cc:1035] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:26.605401   882 raft_consensus.cc:1261] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Refusing update from remote peer bf5ce16ee2ca4b4e9ef50f91febaffde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.607095   975 consensus_queue.cc:1035] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:26.718171   882 raft_consensus.cc:1261] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Refusing update from remote peer bf5ce16ee2ca4b4e9ef50f91febaffde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.719581   975 consensus_queue.cc:1035] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:26.729218   730 raft_consensus.cc:1261] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Refusing update from remote peer bf5ce16ee2ca4b4e9ef50f91febaffde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.730747   965 consensus_queue.cc:1035] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:26.835866   730 raft_consensus.cc:1261] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Refusing update from remote peer 4e24a1d2af5241c183194320fdc3915d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.837309  1017 consensus_queue.cc:1035] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:26.846699   808 raft_consensus.cc:1261] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Refusing update from remote peer 4e24a1d2af5241c183194320fdc3915d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:26.848717  1017 consensus_queue.cc:1035] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:28.989059 23135 test_util.cc:268] Using random seed: 2015899204
I20260924 14:32:29.025919   625 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53866:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260924 14:32:29.028035   625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 14:32:29.089026   715 tablet_service.cc:1467] Processing CreateTablet for tablet b70d16e3eab34d8b8d14dc793f6b3d16 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:29.089630   868 tablet_service.cc:1467] Processing CreateTablet for tablet b70d16e3eab34d8b8d14dc793f6b3d16 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:29.090436   715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b70d16e3eab34d8b8d14dc793f6b3d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.091012   868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b70d16e3eab34d8b8d14dc793f6b3d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.093204   717 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed05026e64d41969eae5584bc00ecfb (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:29.089026   793 tablet_service.cc:1467] Processing CreateTablet for tablet b70d16e3eab34d8b8d14dc793f6b3d16 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:29.094552   717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed05026e64d41969eae5584bc00ecfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.094995   793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b70d16e3eab34d8b8d14dc793f6b3d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.097400   867 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed05026e64d41969eae5584bc00ecfb (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:29.098793   867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed05026e64d41969eae5584bc00ecfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.099516   799 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed05026e64d41969eae5584bc00ecfb (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:29.100855   799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed05026e64d41969eae5584bc00ecfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.101328   721 tablet_service.cc:1467] Processing CreateTablet for tablet 28e3e80523cc4c4fabe9188b05b82f3e (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:29.102628   721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e3e80523cc4c4fabe9188b05b82f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.099516   795 tablet_service.cc:1467] Processing CreateTablet for tablet ff761eb6473d48e5984c882cd2a79f08 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:29.104772   795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff761eb6473d48e5984c882cd2a79f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.104503   870 tablet_service.cc:1467] Processing CreateTablet for tablet ff761eb6473d48e5984c882cd2a79f08 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:29.104928   722 tablet_service.cc:1467] Processing CreateTablet for tablet 072183f2e81447e39727d1d0735c1935 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:29.105901   870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff761eb6473d48e5984c882cd2a79f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.106225   722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072183f2e81447e39727d1d0735c1935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.107766   719 tablet_service.cc:1467] Processing CreateTablet for tablet 83cd7cae43ae489daf7c1c6b9baa5a44 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:29.109138   719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83cd7cae43ae489daf7c1c6b9baa5a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.110396   872 tablet_service.cc:1467] Processing CreateTablet for tablet 072183f2e81447e39727d1d0735c1935 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:29.110935   791 tablet_service.cc:1467] Processing CreateTablet for tablet 28e3e80523cc4c4fabe9188b05b82f3e (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:29.111810   872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072183f2e81447e39727d1d0735c1935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.112262   791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e3e80523cc4c4fabe9188b05b82f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.114605   865 tablet_service.cc:1467] Processing CreateTablet for tablet 28e3e80523cc4c4fabe9188b05b82f3e (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:29.111825   792 tablet_service.cc:1467] Processing CreateTablet for tablet 072183f2e81447e39727d1d0735c1935 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:29.115955   865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e3e80523cc4c4fabe9188b05b82f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.116525   792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072183f2e81447e39727d1d0735c1935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.118284   718 tablet_service.cc:1467] Processing CreateTablet for tablet 6ecaeb9d96a24ef48cba480a8967ee2e (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:29.119427   797 tablet_service.cc:1467] Processing CreateTablet for tablet 83cd7cae43ae489daf7c1c6b9baa5a44 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:29.120688   797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83cd7cae43ae489daf7c1c6b9baa5a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.124891   718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ecaeb9d96a24ef48cba480a8967ee2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.125931   794 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3becd0b7224034901e3967a8a28aa9 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:29.127177   794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3becd0b7224034901e3967a8a28aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.097404   716 tablet_service.cc:1467] Processing CreateTablet for tablet ff761eb6473d48e5984c882cd2a79f08 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:29.127749   714 tablet_service.cc:1467] Processing CreateTablet for tablet 481a3cffff2f4a21bed6fa7c0febc169 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:29.128743   716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff761eb6473d48e5984c882cd2a79f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.129729   869 tablet_service.cc:1467] Processing CreateTablet for tablet 6ecaeb9d96a24ef48cba480a8967ee2e (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:29.130663   871 tablet_service.cc:1467] Processing CreateTablet for tablet 481a3cffff2f4a21bed6fa7c0febc169 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:29.129746   873 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3becd0b7224034901e3967a8a28aa9 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:29.131028   869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ecaeb9d96a24ef48cba480a8967ee2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.132700   871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481a3cffff2f4a21bed6fa7c0febc169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.132431   866 tablet_service.cc:1467] Processing CreateTablet for tablet 83cd7cae43ae489daf7c1c6b9baa5a44 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:29.133279   873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3becd0b7224034901e3967a8a28aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.124948   720 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3becd0b7224034901e3967a8a28aa9 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:29.132133   714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481a3cffff2f4a21bed6fa7c0febc169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.136085   866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83cd7cae43ae489daf7c1c6b9baa5a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.139811   720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3becd0b7224034901e3967a8a28aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.140859   798 tablet_service.cc:1467] Processing CreateTablet for tablet 6ecaeb9d96a24ef48cba480a8967ee2e (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:29.143337   796 tablet_service.cc:1467] Processing CreateTablet for tablet 481a3cffff2f4a21bed6fa7c0febc169 (DEFAULT_TABLE table=soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:29.144588   796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481a3cffff2f4a21bed6fa7c0febc169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.147851   798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ecaeb9d96a24ef48cba480a8967ee2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:29.163014  1045 tablet_bootstrap.cc:492] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:29.169853  1045 tablet_bootstrap.cc:654] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.177749  1046 tablet_bootstrap.cc:492] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:29.185177  1045 tablet_bootstrap.cc:492] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:29.185890  1045 ts_tablet_manager.cc:1397] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.010s
I20260924 14:32:29.187960  1046 tablet_bootstrap.cc:654] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.189235  1045 raft_consensus.cc:348] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.190037  1045 raft_consensus.cc:374] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.190330  1045 raft_consensus.cc:729] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.191076  1045 consensus_queue.cc:260] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.201853  1045 ts_tablet_manager.cc:1428] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260924 14:32:29.204452  1046 tablet_bootstrap.cc:492] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:29.205087  1046 ts_tablet_manager.cc:1397] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.004s
I20260924 14:32:29.210275  1045 tablet_bootstrap.cc:492] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:29.211184  1046 raft_consensus.cc:348] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.212081  1046 raft_consensus.cc:374] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.212373  1046 raft_consensus.cc:729] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.217410  1045 tablet_bootstrap.cc:654] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.213104  1046 consensus_queue.cc:260] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.220570  1048 tablet_bootstrap.cc:492] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:29.229128  1046 ts_tablet_manager.cc:1428] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260924 14:32:29.230095  1045 tablet_bootstrap.cc:492] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:29.230742  1045 ts_tablet_manager.cc:1397] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260924 14:32:29.233837  1045 raft_consensus.cc:348] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.234643  1045 raft_consensus.cc:374] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.234925  1045 raft_consensus.cc:729] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.235733  1045 consensus_queue.cc:260] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.245244  1045 ts_tablet_manager.cc:1428] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.005s
I20260924 14:32:29.259378  1046 tablet_bootstrap.cc:492] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:29.267066  1046 tablet_bootstrap.cc:654] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.273553  1046 tablet_bootstrap.cc:492] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:29.274135  1046 ts_tablet_manager.cc:1397] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.000s
I20260924 14:32:29.276963  1046 raft_consensus.cc:348] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.277695  1046 raft_consensus.cc:374] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.277987  1046 raft_consensus.cc:729] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.278681  1046 consensus_queue.cc:260] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.283610  1046 ts_tablet_manager.cc:1428] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260924 14:32:29.284584  1046 tablet_bootstrap.cc:492] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:29.283028  1048 tablet_bootstrap.cc:654] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.292896  1045 tablet_bootstrap.cc:492] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:29.292896  1048 tablet_bootstrap.cc:492] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:29.295228  1048 ts_tablet_manager.cc:1397] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.000s
I20260924 14:32:29.298075  1048 raft_consensus.cc:348] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.298856  1048 raft_consensus.cc:374] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.299126  1048 raft_consensus.cc:729] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.300191  1045 tablet_bootstrap.cc:654] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.299814  1048 consensus_queue.cc:260] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.302436  1048 ts_tablet_manager.cc:1428] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260924 14:32:29.303493  1048 tablet_bootstrap.cc:492] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:29.312155  1048 tablet_bootstrap.cc:654] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.318720  1045 tablet_bootstrap.cc:492] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:29.319233  1045 ts_tablet_manager.cc:1397] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.016s
I20260924 14:32:29.322006  1045 raft_consensus.cc:348] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.322729  1045 raft_consensus.cc:374] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.323017  1045 raft_consensus.cc:729] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.323721  1045 consensus_queue.cc:260] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.334573  1046 tablet_bootstrap.cc:654] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.339622  1045 ts_tablet_manager.cc:1428] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260924 14:32:29.350054  1045 tablet_bootstrap.cc:492] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:29.366926  1045 tablet_bootstrap.cc:654] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.366956  1031 raft_consensus.cc:482] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.367633  1031 raft_consensus.cc:504] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.370671  1031 leader_election.cc:288] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
W20260924 14:32:29.371490   696 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:39655, user_credentials={real_user=slave}} blocked reactor thread for 46725.4us
I20260924 14:32:29.388087  1046 tablet_bootstrap.cc:492] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:29.388968  1046 ts_tablet_manager.cc:1397] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.105s	user 0.020s	sys 0.012s
W20260924 14:32:29.390856   698 leader_election.cc:341] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303): Not found: Tablet not found: 2ed05026e64d41969eae5584bc00ecfb
I20260924 14:32:29.392319  1046 raft_consensus.cc:348] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.393292  1046 raft_consensus.cc:374] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.393618  1046 raft_consensus.cc:729] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.397763   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed05026e64d41969eae5584bc00ecfb" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
W20260924 14:32:29.400890   696 leader_election.cc:341] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.399142  1046 consensus_queue.cc:260] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.401365   696 leader_election.cc:302] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:29.402127  1058 raft_consensus.cc:2726] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:29.404078  1046 ts_tablet_manager.cc:1428] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260924 14:32:29.405274  1046 tablet_bootstrap.cc:492] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:29.405778  1048 tablet_bootstrap.cc:492] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:29.406359  1048 ts_tablet_manager.cc:1397] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.103s	user 0.025s	sys 0.026s
I20260924 14:32:29.407948  1045 tablet_bootstrap.cc:492] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:29.408566  1045 ts_tablet_manager.cc:1397] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.059s	user 0.028s	sys 0.019s
I20260924 14:32:29.409487  1048 raft_consensus.cc:348] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.410277  1048 raft_consensus.cc:374] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.410564  1048 raft_consensus.cc:729] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.411402  1048 consensus_queue.cc:260] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.411646  1045 raft_consensus.cc:348] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.412628  1045 raft_consensus.cc:374] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.412974  1045 raft_consensus.cc:729] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.413792  1045 consensus_queue.cc:260] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.416188  1046 tablet_bootstrap.cc:654] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.416584  1045 ts_tablet_manager.cc:1428] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:32:29.417819  1045 tablet_bootstrap.cc:492] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:29.417930  1058 raft_consensus.cc:482] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.418426  1058 raft_consensus.cc:504] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.420656  1058 leader_election.cc:288] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.422813   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
W20260924 14:32:29.424311   698 leader_election.cc:341] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.425041  1045 tablet_bootstrap.cc:654] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.425657   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
W20260924 14:32:29.427182   696 leader_election.cc:341] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.427664   696 leader_election.cc:302] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:29.429149  1058 raft_consensus.cc:2726] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:29.437269  1045 tablet_bootstrap.cc:492] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:29.437812  1045 ts_tablet_manager.cc:1397] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260924 14:32:29.440145  1045 raft_consensus.cc:348] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.440949  1045 raft_consensus.cc:374] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.441256  1045 raft_consensus.cc:729] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.441967  1048 ts_tablet_manager.cc:1428] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260924 14:32:29.441972  1045 consensus_queue.cc:260] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.443500  1048 tablet_bootstrap.cc:492] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:29.446548  1045 ts_tablet_manager.cc:1428] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260924 14:32:29.447742  1045 tablet_bootstrap.cc:492] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:29.450191  1048 tablet_bootstrap.cc:654] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.457180  1045 tablet_bootstrap.cc:654] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.465950   965 raft_consensus.cc:482] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.466801   965 raft_consensus.cc:504] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.468920  1030 raft_consensus.cc:482] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.469637  1030 raft_consensus.cc:504] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.472769  1046 tablet_bootstrap.cc:492] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:29.473537  1046 ts_tablet_manager.cc:1397] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.068s	user 0.024s	sys 0.028s
I20260924 14:32:29.474572   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ecaeb9d96a24ef48cba480a8967ee2e" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
W20260924 14:32:29.477762   848 leader_election.cc:341] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.477515  1046 raft_consensus.cc:348] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.478669  1046 raft_consensus.cc:374] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.479089  1046 raft_consensus.cc:729] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.480548  1046 consensus_queue.cc:260] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.480846   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ecaeb9d96a24ef48cba480a8967ee2e" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:29.482640  1048 tablet_bootstrap.cc:492] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:29.483383  1048 ts_tablet_manager.cc:1397] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.018s
I20260924 14:32:29.485077  1030 leader_election.cc:288] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:29.485816  1030 raft_consensus.cc:482] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260924 14:32:29.486202   847 leader_election.cc:341] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.486712   847 leader_election.cc:302] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d; no voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:29.487982  1045 tablet_bootstrap.cc:492] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:29.488274   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:29.488703  1045 ts_tablet_manager.cc:1397] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.041s	user 0.007s	sys 0.028s
I20260924 14:32:29.489099   730 raft_consensus.cc:2454] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:29.489936   965 leader_election.cc:288] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.491734   774 leader_election.cc:302] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:29.486244  1030 raft_consensus.cc:504] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.494493  1063 raft_consensus.cc:2726] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:29.496196   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481a3cffff2f4a21bed6fa7c0febc169" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:29.497345  1046 ts_tablet_manager.cc:1428] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.023s	user 0.012s	sys 0.011s
I20260924 14:32:29.497731   730 raft_consensus.cc:2454] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 0.
I20260924 14:32:29.497781   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481a3cffff2f4a21bed6fa7c0febc169" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:29.496382  1030 leader_election.cc:288] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:29.498451  1046 tablet_bootstrap.cc:492] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:29.493085  1045 raft_consensus.cc:348] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.499575  1045 raft_consensus.cc:374] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.499411   848 leader_election.cc:302] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:29.500088  1045 raft_consensus.cc:729] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
W20260924 14:32:29.500595   847 leader_election.cc:341] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.502031   965 raft_consensus.cc:2781] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:29.503363   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:29.503964   965 raft_consensus.cc:482] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:29.504364   965 raft_consensus.cc:3037] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.505293  1046 tablet_bootstrap.cc:654] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
W20260924 14:32:29.505219   775 leader_election.cc:341] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.506456  1030 raft_consensus.cc:2781] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:29.506909  1030 raft_consensus.cc:482] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:29.507540  1030 raft_consensus.cc:3037] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.503243  1048 raft_consensus.cc:348] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.511157  1048 raft_consensus.cc:374] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.511484  1048 raft_consensus.cc:729] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.512209  1048 consensus_queue.cc:260] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.512727  1046 tablet_bootstrap.cc:492] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:29.513522  1046 ts_tablet_manager.cc:1397] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.008s
I20260924 14:32:29.512769   965 raft_consensus.cc:504] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.501035  1045 consensus_queue.cc:260] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.514732  1048 ts_tablet_manager.cc:1428] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260924 14:32:29.516845  1046 raft_consensus.cc:348] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.516871   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:29.517807  1046 raft_consensus.cc:374] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.517976   730 raft_consensus.cc:3037] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.518126  1046 raft_consensus.cc:729] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.519351  1046 consensus_queue.cc:260] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.526093  1046 ts_tablet_manager.cc:1428] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.008s
I20260924 14:32:29.527031  1030 raft_consensus.cc:504] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.528412  1046 tablet_bootstrap.cc:492] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:29.529126  1030 leader_election.cc:288] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:29.530272  1048 tablet_bootstrap.cc:492] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:29.531422   730 raft_consensus.cc:2454] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:29.517359   965 leader_election.cc:288] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.516736  1045 ts_tablet_manager.cc:1428] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260924 14:32:29.533540  1045 tablet_bootstrap.cc:492] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:29.535818  1046 tablet_bootstrap.cc:654] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.535954   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
W20260924 14:32:29.537565   775 leader_election.cc:341] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.538024  1048 tablet_bootstrap.cc:654] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.539527   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481a3cffff2f4a21bed6fa7c0febc169" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:29.540302  1045 tablet_bootstrap.cc:654] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.541200  1030 raft_consensus.cc:482] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.541893  1030 raft_consensus.cc:504] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.544739  1030 leader_election.cc:288] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
W20260924 14:32:29.546588   847 leader_election.cc:341] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.548120  1046 tablet_bootstrap.cc:492] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:29.548184   774 leader_election.cc:302] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 4e24a1d2af5241c183194320fdc3915d
I20260924 14:32:29.549424   732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072183f2e81447e39727d1d0735c1935" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:29.550311   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481a3cffff2f4a21bed6fa7c0febc169" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:29.551198   731 raft_consensus.cc:3037] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.549459   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072183f2e81447e39727d1d0735c1935" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:29.548803  1046 ts_tablet_manager.cc:1397] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.011s
W20260924 14:32:29.555907   848 leader_election.cc:341] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.561515   965 raft_consensus.cc:2781] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:29.555645  1046 raft_consensus.cc:348] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.562650  1046 raft_consensus.cc:374] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.562997  1046 raft_consensus.cc:729] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
W20260924 14:32:29.564144   847 leader_election.cc:341] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.564751   847 leader_election.cc:302] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d; no voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:29.563787  1046 consensus_queue.cc:260] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.566105  1064 raft_consensus.cc:2726] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:29.562729   965 raft_consensus.cc:686] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:29.567637  1046 ts_tablet_manager.cc:1428] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.003s
I20260924 14:32:29.568866  1046 tablet_bootstrap.cc:492] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:29.568703   965 consensus_queue.cc:237] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.575969  1046 tablet_bootstrap.cc:654] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.576124   731 raft_consensus.cc:2454] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 1.
I20260924 14:32:29.581970  1058 raft_consensus.cc:482] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.582443  1030 raft_consensus.cc:482] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.586356  1030 raft_consensus.cc:504] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.586288  1058 raft_consensus.cc:504] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.588680  1030 leader_election.cc:288] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:29.589852  1058 leader_election.cc:288] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.590868   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83cd7cae43ae489daf7c1c6b9baa5a44" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:29.591780   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83cd7cae43ae489daf7c1c6b9baa5a44" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
W20260924 14:32:29.592829   847 leader_election.cc:341] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.593639   628 catalog_manager.cc:5495] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 0 to 1, leader changed from <none> to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 1 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
W20260924 14:32:29.593703   848 leader_election.cc:341] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.593721   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff761eb6473d48e5984c882cd2a79f08" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:29.595685   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff761eb6473d48e5984c882cd2a79f08" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:29.595803   806 raft_consensus.cc:2454] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 0.
I20260924 14:32:29.595194   848 leader_election.cc:302] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d; no voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:29.596966   696 leader_election.cc:302] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
W20260924 14:32:29.597491   698 leader_election.cc:341] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.597734  1054 raft_consensus.cc:2781] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:29.598129   848 leader_election.cc:302] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:29.598145  1054 raft_consensus.cc:482] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:29.598887  1054 raft_consensus.cc:3037] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.599013  1045 tablet_bootstrap.cc:492] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:29.599363  1030 raft_consensus.cc:2726] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:29.599624  1045 ts_tablet_manager.cc:1397] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.066s	user 0.014s	sys 0.003s
I20260924 14:32:29.600033  1030 raft_consensus.cc:2781] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:29.600135  1046 tablet_bootstrap.cc:492] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:29.600764  1046 ts_tablet_manager.cc:1397] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.011s
I20260924 14:32:29.601727  1030 raft_consensus.cc:686] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Becoming Leader. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:29.602622  1030 consensus_queue.cc:237] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.603597  1046 raft_consensus.cc:348] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.604348  1046 raft_consensus.cc:374] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.604725  1046 raft_consensus.cc:729] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.605445  1046 consensus_queue.cc:260] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.607360  1045 raft_consensus.cc:348] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.608197  1045 raft_consensus.cc:374] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.608434  1048 tablet_bootstrap.cc:492] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:29.608541  1046 ts_tablet_manager.cc:1428] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:32:29.608510  1045 raft_consensus.cc:729] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.608377  1054 raft_consensus.cc:504] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.609575  1048 ts_tablet_manager.cc:1397] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.080s	user 0.051s	sys 0.005s
I20260924 14:32:29.610436  1046 tablet_bootstrap.cc:492] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:29.610085  1045 consensus_queue.cc:260] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.614401  1048 raft_consensus.cc:348] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.617110  1048 raft_consensus.cc:374] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.617172  1045 ts_tablet_manager.cc:1428] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.004s
I20260924 14:32:29.617496  1048 raft_consensus.cc:729] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.618368  1045 tablet_bootstrap.cc:492] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:29.620328  1046 tablet_bootstrap.cc:654] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.619951  1048 consensus_queue.cc:260] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.624153  1048 ts_tablet_manager.cc:1428] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.002s
I20260924 14:32:29.625216  1048 tablet_bootstrap.cc:492] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:29.625929  1045 tablet_bootstrap.cc:654] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.631871  1046 tablet_bootstrap.cc:492] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:29.632439  1046 ts_tablet_manager.cc:1397] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260924 14:32:29.635218  1046 raft_consensus.cc:348] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.635949  1046 raft_consensus.cc:374] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.636216  1046 raft_consensus.cc:729] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.636605  1045 tablet_bootstrap.cc:492] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:29.637316  1045 ts_tablet_manager.cc:1397] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.003s
I20260924 14:32:29.636951  1046 consensus_queue.cc:260] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.639591   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff761eb6473d48e5984c882cd2a79f08" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:29.640206   806 raft_consensus.cc:3037] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.641090  1045 raft_consensus.cc:348] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.642074  1045 raft_consensus.cc:374] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.642383  1045 raft_consensus.cc:729] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.641103  1048 tablet_bootstrap.cc:654] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.643604   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff761eb6473d48e5984c882cd2a79f08" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:29.644564   882 raft_consensus.cc:3037] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.643293  1045 consensus_queue.cc:260] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.649434   806 raft_consensus.cc:2454] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 1.
I20260924 14:32:29.650812   696 leader_election.cc:302] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:29.650954  1045 ts_tablet_manager.cc:1428] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260924 14:32:29.651721  1058 raft_consensus.cc:2781] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:29.652189  1058 raft_consensus.cc:686] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:29.652345  1045 tablet_bootstrap.cc:492] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21: Bootstrap starting.
I20260924 14:32:29.652736   882 raft_consensus.cc:2454] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 1.
I20260924 14:32:29.653064  1058 consensus_queue.cc:237] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.658560  1046 ts_tablet_manager.cc:1428] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260924 14:32:29.660336  1045 tablet_bootstrap.cc:654] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.661474  1046 tablet_bootstrap.cc:492] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d: Bootstrap starting.
I20260924 14:32:29.662729  1054 leader_election.cc:288] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 election: Requested vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.665848  1064 raft_consensus.cc:482] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.666352  1064 raft_consensus.cc:504] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.669070  1064 leader_election.cc:288] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:29.670624   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:29.670588   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:29.671640   731 raft_consensus.cc:2454] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 0.
W20260924 14:32:29.672652   847 leader_election.cc:341] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.673513   848 leader_election.cc:302] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:29.675091  1064 raft_consensus.cc:2781] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:29.675626  1064 raft_consensus.cc:482] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:29.676033  1064 raft_consensus.cc:3037] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.678009   625 catalog_manager.cc:5495] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:29.685716  1064 raft_consensus.cc:504] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.686458  1046 tablet_bootstrap.cc:654] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.686575  1048 tablet_bootstrap.cc:492] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:29.687321  1048 ts_tablet_manager.cc:1397] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.062s	user 0.018s	sys 0.007s
I20260924 14:32:29.690400  1048 raft_consensus.cc:348] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.691222  1048 raft_consensus.cc:374] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.691521  1048 raft_consensus.cc:729] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.692436  1048 consensus_queue.cc:260] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.694862  1048 ts_tablet_manager.cc:1428] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260924 14:32:29.695923  1048 tablet_bootstrap.cc:492] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:29.694566   627 catalog_manager.cc:5495] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e24a1d2af5241c183194320fdc3915d (127.22.151.195). New cstate: current_term: 1 leader_uuid: "4e24a1d2af5241c183194320fdc3915d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:29.699828   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:29.700945   731 raft_consensus.cc:3037] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.701028   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:29.702729  1048 tablet_bootstrap.cc:654] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
W20260924 14:32:29.704232   847 leader_election.cc:341] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:29.705046  1064 leader_election.cc:288] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:29.707293  1045 tablet_bootstrap.cc:492] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21: No bootstrap required, opened a new log
I20260924 14:32:29.707808  1045 ts_tablet_manager.cc:1397] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.004s
I20260924 14:32:29.711342  1045 raft_consensus.cc:348] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.712311  1045 raft_consensus.cc:374] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.712649  1045 raft_consensus.cc:729] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.713562  1045 consensus_queue.cc:260] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.716742  1045 ts_tablet_manager.cc:1428] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260924 14:32:29.719697   731 raft_consensus.cc:2454] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 1.
I20260924 14:32:29.721438   848 leader_election.cc:302] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:29.722970  1064 raft_consensus.cc:2781] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:29.723650  1064 raft_consensus.cc:686] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Becoming Leader. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:29.724838  1064 consensus_queue.cc:237] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.726647  1046 tablet_bootstrap.cc:492] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d: No bootstrap required, opened a new log
I20260924 14:32:29.727350  1046 ts_tablet_manager.cc:1397] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d: Time spent bootstrapping tablet: real 0.066s	user 0.024s	sys 0.020s
I20260924 14:32:29.730331  1046 raft_consensus.cc:348] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.731052  1046 raft_consensus.cc:374] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.731294  1046 raft_consensus.cc:729] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.731940  1046 consensus_queue.cc:260] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.732928  1048 tablet_bootstrap.cc:492] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:29.733773  1048 ts_tablet_manager.cc:1397] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.009s
I20260924 14:32:29.734217  1046 ts_tablet_manager.cc:1428] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:32:29.737354  1048 raft_consensus.cc:348] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.739044  1048 raft_consensus.cc:374] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.739319  1048 raft_consensus.cc:729] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.740046  1048 consensus_queue.cc:260] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.742237  1048 ts_tablet_manager.cc:1428] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260924 14:32:29.743155  1048 tablet_bootstrap.cc:492] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:29.748283  1048 tablet_bootstrap.cc:654] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.749322   628 catalog_manager.cc:5495] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e24a1d2af5241c183194320fdc3915d (127.22.151.195). New cstate: current_term: 1 leader_uuid: "4e24a1d2af5241c183194320fdc3915d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:29.757656  1048 tablet_bootstrap.cc:492] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:29.758185  1048 ts_tablet_manager.cc:1397] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.006s
I20260924 14:32:29.760840  1048 raft_consensus.cc:348] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.761386  1048 raft_consensus.cc:374] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.761672  1048 raft_consensus.cc:729] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.762347  1048 consensus_queue.cc:260] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.764428  1048 ts_tablet_manager.cc:1428] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260924 14:32:29.765208  1048 tablet_bootstrap.cc:492] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:29.770684  1048 tablet_bootstrap.cc:654] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.774533  1058 raft_consensus.cc:482] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.775270  1048 tablet_bootstrap.cc:492] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:29.775102  1058 raft_consensus.cc:504] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.775818  1048 ts_tablet_manager.cc:1397] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.011s	user 0.006s	sys 0.004s
I20260924 14:32:29.777247  1058 leader_election.cc:288] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.777982   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072183f2e81447e39727d1d0735c1935" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:29.778318  1048 raft_consensus.cc:348] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.778946  1048 raft_consensus.cc:374] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.779160  1048 raft_consensus.cc:729] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.779501   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072183f2e81447e39727d1d0735c1935" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:29.779731  1048 consensus_queue.cc:260] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.780061   882 raft_consensus.cc:2454] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 0.
I20260924 14:32:29.780593   806 raft_consensus.cc:2454] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 0.
I20260924 14:32:29.781281   698 leader_election.cc:302] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:29.781790  1048 ts_tablet_manager.cc:1428] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:32:29.782172  1058 raft_consensus.cc:2781] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:29.782590  1058 raft_consensus.cc:482] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:29.782964  1048 tablet_bootstrap.cc:492] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde: Bootstrap starting.
I20260924 14:32:29.783205  1058 raft_consensus.cc:3037] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.788918  1058 raft_consensus.cc:504] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.789333  1048 tablet_bootstrap.cc:654] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:29.791546   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072183f2e81447e39727d1d0735c1935" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:29.792115  1058 leader_election.cc:288] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 election: Requested vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.792822   882 raft_consensus.cc:3037] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.792689   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072183f2e81447e39727d1d0735c1935" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:29.793438   806 raft_consensus.cc:3037] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.800158   882 raft_consensus.cc:2454] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 1.
I20260924 14:32:29.800160   806 raft_consensus.cc:2454] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 1.
I20260924 14:32:29.801991   696 leader_election.cc:302] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:29.802691  1058 raft_consensus.cc:2781] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:29.803176  1058 raft_consensus.cc:686] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:29.804152  1048 tablet_bootstrap.cc:492] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde: No bootstrap required, opened a new log
I20260924 14:32:29.803932  1058 consensus_queue.cc:237] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.804643  1048 ts_tablet_manager.cc:1397] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260924 14:32:29.806833  1048 raft_consensus.cc:348] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.807593  1048 raft_consensus.cc:374] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:29.807893  1048 raft_consensus.cc:729] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Initialized, Role: FOLLOWER
I20260924 14:32:29.808537  1048 consensus_queue.cc:260] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.809073  1064 raft_consensus.cc:482] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.809592  1064 raft_consensus.cc:504] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.810536  1048 ts_tablet_manager.cc:1428] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:32:29.811542  1064 leader_election.cc:288] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:29.812790   627 catalog_manager.cc:5495] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:29.813444   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" is_pre_election: true
I20260924 14:32:29.813887   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:29.814133   806 raft_consensus.cc:2454] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 0.
I20260924 14:32:29.814585   731 raft_consensus.cc:2454] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 0.
I20260924 14:32:29.815760   848 leader_election.cc:302] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:29.816571  1064 raft_consensus.cc:2781] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:29.816936  1064 raft_consensus.cc:482] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:29.817248  1064 raft_consensus.cc:3037] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.824796  1064 raft_consensus.cc:504] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.827831  1064 leader_election.cc:288] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:29.827697   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:29.828084   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:29.828436   731 raft_consensus.cc:3037] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.828714   806 raft_consensus.cc:3037] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.833770   731 raft_consensus.cc:2454] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 1.
I20260924 14:32:29.833913   806 raft_consensus.cc:2454] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 1.
I20260924 14:32:29.834949   848 leader_election.cc:302] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:29.835619  1064 raft_consensus.cc:2781] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:29.836023  1064 raft_consensus.cc:686] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Becoming Leader. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:29.836373   965 raft_consensus.cc:482] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.836750  1064 consensus_queue.cc:237] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.836867   965 raft_consensus.cc:504] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.838696   965 leader_election.cc:288] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.839666   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed05026e64d41969eae5584bc00ecfb" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:29.840463   882 raft_consensus.cc:2454] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:29.839607   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed05026e64d41969eae5584bc00ecfb" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:29.841450   731 raft_consensus.cc:2454] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:29.841722   775 leader_election.cc:302] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:29.842525   965 raft_consensus.cc:2781] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:29.842868   965 raft_consensus.cc:482] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:29.843159   965 raft_consensus.cc:3037] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.846237   628 catalog_manager.cc:5495] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e24a1d2af5241c183194320fdc3915d (127.22.151.195). New cstate: current_term: 1 leader_uuid: "4e24a1d2af5241c183194320fdc3915d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:29.848228  1043 raft_consensus.cc:482] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.848716  1043 raft_consensus.cc:504] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.850097   965 raft_consensus.cc:504] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.850701  1043 leader_election.cc:288] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.852275   965 leader_election.cc:288] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.852942   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ecaeb9d96a24ef48cba480a8967ee2e" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:29.853381   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed05026e64d41969eae5584bc00ecfb" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:29.854511   881 raft_consensus.cc:3037] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.853735   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ecaeb9d96a24ef48cba480a8967ee2e" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:29.853833   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed05026e64d41969eae5584bc00ecfb" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:29.855180   730 raft_consensus.cc:3037] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.856309   731 raft_consensus.cc:2454] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:29.857285   774 leader_election.cc:302] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:29.857913  1043 raft_consensus.cc:2781] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:29.858274   882 raft_consensus.cc:2454] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:29.858299  1043 raft_consensus.cc:482] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:29.858863  1043 raft_consensus.cc:3037] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.861714   881 raft_consensus.cc:2454] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:29.862088   730 raft_consensus.cc:2454] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:29.862917   775 leader_election.cc:302] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:29.864410   965 raft_consensus.cc:2781] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:29.864972   965 raft_consensus.cc:686] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:29.865861   965 consensus_queue.cc:237] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.866601  1043 raft_consensus.cc:504] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.868594  1043 leader_election.cc:288] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.869661   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ecaeb9d96a24ef48cba480a8967ee2e" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:29.869395   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ecaeb9d96a24ef48cba480a8967ee2e" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:29.870424   881 raft_consensus.cc:3037] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.870829   731 raft_consensus.cc:3037] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.875821   628 catalog_manager.cc:5495] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 0 to 1, leader changed from <none> to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 1 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:29.877491   881 raft_consensus.cc:2454] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:29.878432   775 leader_election.cc:302] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:29.878890   731 raft_consensus.cc:2454] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:29.879813  1043 raft_consensus.cc:2781] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:29.880359  1043 raft_consensus.cc:686] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:29.881193  1043 consensus_queue.cc:237] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
W20260924 14:32:29.883013   677 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:32:29.892592   627 catalog_manager.cc:5495] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 0 to 1, leader changed from <none> to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 1 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:29.970677  1043 raft_consensus.cc:482] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:29.971344  1043 raft_consensus.cc:504] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.973136  1043 leader_election.cc:288] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.973870   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83cd7cae43ae489daf7c1c6b9baa5a44" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:29.974085   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83cd7cae43ae489daf7c1c6b9baa5a44" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:29.974509   731 raft_consensus.cc:2454] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:29.974725   881 raft_consensus.cc:2454] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 0.
I20260924 14:32:29.975586   774 leader_election.cc:302] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:29.976289  1043 raft_consensus.cc:2781] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:29.976663  1043 raft_consensus.cc:482] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:29.976902  1043 raft_consensus.cc:3037] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.982726  1043 raft_consensus.cc:504] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.984918  1043 leader_election.cc:288] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:29.985284   731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83cd7cae43ae489daf7c1c6b9baa5a44" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:29.986019   731 raft_consensus.cc:3037] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.986531   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83cd7cae43ae489daf7c1c6b9baa5a44" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:29.987166   882 raft_consensus.cc:3037] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:29.992014   731 raft_consensus.cc:2454] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:29.992713   882 raft_consensus.cc:2454] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 1.
I20260924 14:32:29.993081   774 leader_election.cc:302] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:29.994007  1043 raft_consensus.cc:2781] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:29.994434  1043 raft_consensus.cc:686] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:29.995183  1043 consensus_queue.cc:237] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:29.997020   882 raft_consensus.cc:3037] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:30.003379  1090 consensus_queue.cc:1035] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:30.006265   965 consensus_queue.cc:1035] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.009218   627 catalog_manager.cc:5495] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 0 to 1, leader changed from <none> to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 1 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:30.068482   806 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16"
dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
mode: GRACEFUL
new_leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:47042
I20260924 14:32:30.069061   806 raft_consensus.cc:595] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Received request to transfer leadership to TS ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:30.071633   965 raft_consensus.cc:482] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:30.072234   965 raft_consensus.cc:504] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.074353   965 leader_election.cc:288] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:30.074664   806 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2ed05026e64d41969eae5584bc00ecfb"
dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
mode: GRACEFUL
new_leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:47042
I20260924 14:32:30.075371   806 raft_consensus.cc:595] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Received request to transfer leadership to TS ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:30.075594   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481a3cffff2f4a21bed6fa7c0febc169" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:30.076013   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481a3cffff2f4a21bed6fa7c0febc169" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:30.076911   730 raft_consensus.cc:2379] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf5ce16ee2ca4b4e9ef50f91febaffde in current term 1: Already voted for candidate 4e24a1d2af5241c183194320fdc3915d in this term.
I20260924 14:32:30.077932   774 leader_election.cc:302] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:30.079082   965 raft_consensus.cc:3037] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:30.083567   882 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e"
dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
mode: GRACEFUL
new_leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:41862
I20260924 14:32:30.084233   882 raft_consensus.cc:595] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Received request to transfer leadership to TS ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:30.085708   965 raft_consensus.cc:2726] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:30.089576   730 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "072183f2e81447e39727d1d0735c1935"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
mode: GRACEFUL
new_leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
 from {username='slave'} at 127.0.0.1:33394
I20260924 14:32:30.090209   730 raft_consensus.cc:595] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Received request to transfer leadership to TS bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:30.098294   882 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9"
dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
mode: GRACEFUL
new_leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
 from {username='slave'} at 127.0.0.1:41862
I20260924 14:32:30.098889   882 raft_consensus.cc:595] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Received request to transfer leadership to TS bf5ce16ee2ca4b4e9ef50f91febaffde
I20260924 14:32:30.124277  1064 consensus_queue.cc:1035] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.134606  1064 consensus_queue.cc:1035] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.180191   965 raft_consensus.cc:482] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:30.180689   965 raft_consensus.cc:504] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.182607   965 leader_election.cc:288] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:30.183552   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" is_pre_election: true
I20260924 14:32:30.183736   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e24a1d2af5241c183194320fdc3915d" is_pre_election: true
I20260924 14:32:30.185467   775 leader_election.cc:302] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21
I20260924 14:32:30.185495  1064 consensus_queue.cc:1035] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:30.186667   965 raft_consensus.cc:2726] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:30.193223  1030 raft_consensus.cc:979] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d: : Instructing follower ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:30.193841  1100 raft_consensus.cc:1067] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Signalling peer ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:30.195595   730 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:33380
I20260924 14:32:30.197435   730 raft_consensus.cc:482] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:30.197973   730 raft_consensus.cc:3037] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.199487   806 raft_consensus.cc:3037] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:30.206576   730 raft_consensus.cc:504] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.209549   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:30.209661   730 leader_election.cc:288] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Requested vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:30.209856   882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e3e80523cc4c4fabe9188b05b82f3e" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:30.210207   808 raft_consensus.cc:3037] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.210412   882 raft_consensus.cc:3032] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:30.210806   882 raft_consensus.cc:729] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:30.211488   882 consensus_queue.cc:260] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.212659   882 raft_consensus.cc:3037] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.219129   882 raft_consensus.cc:2454] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:30.220037   698 leader_election.cc:302] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:30.220316   808 raft_consensus.cc:2454] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:30.220799  1058 raft_consensus.cc:2781] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:30.221213  1054 consensus_queue.cc:1035] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.221861  1058 raft_consensus.cc:686] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:30.222779  1058 consensus_queue.cc:237] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.233673   627 catalog_manager.cc:5495] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 1 to 2, leader changed from 4e24a1d2af5241c183194320fdc3915d (127.22.151.195) to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:30.235001  1058 consensus_queue.cc:1035] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:30.250686   965 consensus_queue.cc:1035] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.264878  1089 consensus_queue.cc:1035] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.277272  1090 raft_consensus.cc:979] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde: : Instructing follower ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:30.277724  1089 raft_consensus.cc:1067] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Signalling peer ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:30.279210   730 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2ed05026e64d41969eae5584bc00ecfb"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:33374
I20260924 14:32:30.279803   730 raft_consensus.cc:482] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:30.280097   730 raft_consensus.cc:3037] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.284994   730 raft_consensus.cc:504] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.286490   730 leader_election.cc:288] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Requested vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:30.287672   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed05026e64d41969eae5584bc00ecfb" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:30.287801   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed05026e64d41969eae5584bc00ecfb" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:30.288534   881 raft_consensus.cc:3037] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.288542   806 raft_consensus.cc:3032] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:30.289096   806 raft_consensus.cc:729] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:30.291316   806 consensus_queue.cc:260] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.292762   806 raft_consensus.cc:3037] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.294497   881 raft_consensus.cc:2454] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:30.295562   698 leader_election.cc:302] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:30.296239  1058 raft_consensus.cc:2781] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:30.296638  1058 raft_consensus.cc:686] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:30.297379  1058 consensus_queue.cc:237] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.297936   806 raft_consensus.cc:2454] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:30.304105   628 catalog_manager.cc:5495] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 1 to 2, leader changed from bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194) to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:30.353127  1058 consensus_queue.cc:1035] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.359577  1054 raft_consensus.cc:979] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21: : Instructing follower bf5ce16ee2ca4b4e9ef50f91febaffde to start an election
I20260924 14:32:30.360116  1054 raft_consensus.cc:1067] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Signalling peer bf5ce16ee2ca4b4e9ef50f91febaffde to start an election
I20260924 14:32:30.361699   806 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "072183f2e81447e39727d1d0735c1935"
dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
 from {username='slave'} at 127.0.0.1:47026
I20260924 14:32:30.363240   806 raft_consensus.cc:482] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:30.363656   806 raft_consensus.cc:3037] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.366676  1054 consensus_queue.cc:1035] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.373615   806 raft_consensus.cc:504] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.376341   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072183f2e81447e39727d1d0735c1935" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:30.376600   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072183f2e81447e39727d1d0735c1935" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:30.376955   730 raft_consensus.cc:3032] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:30.377267   730 raft_consensus.cc:729] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:30.377373   806 leader_election.cc:288] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 2 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:30.378032   730 consensus_queue.cc:260] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.377297   881 raft_consensus.cc:3037] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.379369   730 raft_consensus.cc:3037] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.386436   881 raft_consensus.cc:2454] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 2.
I20260924 14:32:30.387576   730 raft_consensus.cc:2454] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 2.
I20260924 14:32:30.388688   774 leader_election.cc:302] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:30.389503  1089 raft_consensus.cc:2781] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:30.390201  1089 raft_consensus.cc:686] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:30.391019  1089 consensus_queue.cc:237] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.394675  1090 consensus_queue.cc:1035] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.400290   627 catalog_manager.cc:5495] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 1 to 2, leader changed from ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193) to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 2 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:30.408546  1090 consensus_queue.cc:1035] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.415616   965 consensus_queue.cc:1035] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.444252  1043 consensus_queue.cc:1035] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.461597  1064 consensus_queue.cc:1035] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:30.466437  1089 raft_consensus.cc:979] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde: : Instructing follower ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:30.467140  1090 raft_consensus.cc:1067] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Signalling peer ace4fb4d0d594edfb5ceb70d49a01d21 to start an election
I20260924 14:32:30.468874   731 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
 from {username='slave'} at 127.0.0.1:33374
I20260924 14:32:30.469446   731 raft_consensus.cc:482] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:30.468924  1064 consensus_queue.cc:1035] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:30.469821   731 raft_consensus.cc:3037] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.478622   731 raft_consensus.cc:504] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.482168   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:30.482388   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16" candidate_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:30.482887   806 raft_consensus.cc:3032] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:30.483117   881 raft_consensus.cc:3037] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.483561   806 raft_consensus.cc:729] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:30.484527   806 consensus_queue.cc:260] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.485726   806 raft_consensus.cc:3037] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.487716   731 leader_election.cc:288] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Requested vote from peers bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:30.482925  1100 raft_consensus.cc:979] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d: : Instructing follower bf5ce16ee2ca4b4e9ef50f91febaffde to start an election
I20260924 14:32:30.492779  1100 raft_consensus.cc:1067] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Signalling peer bf5ce16ee2ca4b4e9ef50f91febaffde to start an election
I20260924 14:32:30.493121   881 raft_consensus.cc:2454] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:30.494118   806 raft_consensus.cc:2454] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace4fb4d0d594edfb5ceb70d49a01d21 in term 2.
I20260924 14:32:30.494392   698 leader_election.cc:302] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:30.499567  1054 raft_consensus.cc:2781] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:30.496493   807 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9"
dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
 from {username='slave'} at 127.0.0.1:47032
I20260924 14:32:30.500051  1054 raft_consensus.cc:686] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Becoming Leader. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:30.500835  1054 consensus_queue.cc:237] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.505246   807 raft_consensus.cc:482] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:30.505669   807 raft_consensus.cc:3037] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.512418   807 raft_consensus.cc:504] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.515134   730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:30.515911   730 raft_consensus.cc:3037] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.515890   881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9" candidate_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
I20260924 14:32:30.515210   628 catalog_manager.cc:5495] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 reported cstate change: term changed from 1 to 2, leader changed from bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194) to ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:30.516700   881 raft_consensus.cc:3032] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:30.517108   881 raft_consensus.cc:729] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:30.517899   881 consensus_queue.cc:260] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.519048   881 raft_consensus.cc:3037] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:30.523049   730 raft_consensus.cc:2454] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 2.
I20260924 14:32:30.515132   807 leader_election.cc:288] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 2 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), 4e24a1d2af5241c183194320fdc3915d (127.22.151.195:33303)
I20260924 14:32:30.524992   774 leader_election.cc:302] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace4fb4d0d594edfb5ceb70d49a01d21, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:30.525804  1090 raft_consensus.cc:2781] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:30.526240  1090 raft_consensus.cc:686] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Becoming Leader. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:30.526957  1090 consensus_queue.cc:237] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:30.529304   881 raft_consensus.cc:2454] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf5ce16ee2ca4b4e9ef50f91febaffde in term 2.
I20260924 14:32:30.534922   627 catalog_manager.cc:5495] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde reported cstate change: term changed from 1 to 2, leader changed from 4e24a1d2af5241c183194320fdc3915d (127.22.151.195) to bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194). New cstate: current_term: 2 leader_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:30.756305   807 raft_consensus.cc:1261] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:30.757678  1054 consensus_queue.cc:1035] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:30.758543   881 raft_consensus.cc:1261] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:30.759965  1054 consensus_queue.cc:1035] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:30.849923   881 raft_consensus.cc:1261] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:30.852805  1058 consensus_queue.cc:1035] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:30.854872   807 raft_consensus.cc:1261] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:30.861543  1058 consensus_queue.cc:1035] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:30.900853   881 raft_consensus.cc:1261] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Refusing update from remote peer bf5ce16ee2ca4b4e9ef50f91febaffde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:30.902812  1043 consensus_queue.cc:1035] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:30.925122   730 raft_consensus.cc:1261] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Refusing update from remote peer bf5ce16ee2ca4b4e9ef50f91febaffde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:30.928329  1043 consensus_queue.cc:1035] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:30.931555   881 raft_consensus.cc:1261] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Refusing update from remote peer bf5ce16ee2ca4b4e9ef50f91febaffde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:30.933609  1043 consensus_queue.cc:1035] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:30.951965   730 raft_consensus.cc:1261] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Refusing update from remote peer bf5ce16ee2ca4b4e9ef50f91febaffde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:30.953696  1112 consensus_queue.cc:1035] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:30.971598   807 raft_consensus.cc:1261] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:30.974527  1054 consensus_queue.cc:1035] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:30.985812   882 raft_consensus.cc:1261] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Refusing update from remote peer ace4fb4d0d594edfb5ceb70d49a01d21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:30.991086  1104 consensus_queue.cc:1035] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:33.108527   627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53866:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260924 14:32:33.135634   627 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53866:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260924 14:32:33.157171 23135 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
W20260924 14:32:33.461951  1161 log.cc:924] Time spent T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260924 14:32:33.468348  1143 log.cc:924] Time spent T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260924 14:32:33.486685  1150 log.cc:924] Time spent T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21: Append to log took a long time: real 0.052s	user 0.002s	sys 0.000s
I20260924 14:32:33.528908  1169 tablet.cc:1722] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260924 14:32:33.543700  1163 log.cc:924] Time spent T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d: Append to log took a long time: real 0.063s	user 0.002s	sys 0.000s
I20260924 14:32:33.665870  1177 tablet.cc:1722] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.677425  1179 tablet.cc:1722] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.659114  1175 tablet.cc:1722] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.679040  1173 tablet.cc:1722] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.675338  1172 tablet.cc:1722] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260924 14:32:33.715337   910 tablet_mm_ops.cc:140] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:481a3cffff2f4a21bed6fa7c0febc169)
W20260924 14:32:33.727589   910 tablet_mm_ops.cc:305] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:481a3cffff2f4a21bed6fa7c0febc169)
W20260924 14:32:33.736697   910 tablet_mm_ops.cc:221] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:481a3cffff2f4a21bed6fa7c0febc169)
I20260924 14:32:33.744365  1185 tablet.cc:1722] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.753662   805 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9"
dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
mode: GRACEFUL
new_leader_uuid: "4e24a1d2af5241c183194320fdc3915d"
 from {username='slave'} at 127.0.0.1:41430
I20260924 14:32:33.754861   805 raft_consensus.cc:595] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Received request to transfer leadership to TS 4e24a1d2af5241c183194320fdc3915d
I20260924 14:32:33.760881  1183 tablet.cc:1722] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.772171  1184 tablet.cc:1722] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.774219  1190 tablet.cc:1722] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.795416  1120 raft_consensus.cc:979] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde: : Instructing follower 4e24a1d2af5241c183194320fdc3915d to start an election
I20260924 14:32:33.796239  1120 raft_consensus.cc:1067] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Signalling peer 4e24a1d2af5241c183194320fdc3915d to start an election
I20260924 14:32:33.798585   877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9"
dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
 from {username='slave'} at 127.0.0.1:41848
I20260924 14:32:33.799297   877 raft_consensus.cc:482] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:33.804258   877 raft_consensus.cc:3037] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:32:33.799713  1189 tablet.cc:1722] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.826408  1184 tablet.cc:1722] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.810987  1192 tablet.cc:1722] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.830742  1175 tablet.cc:1722] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.843040  1179 tablet.cc:1722] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.831807  1169 tablet.cc:1722] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.829015  1172 tablet.cc:1722] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.840186  1190 tablet.cc:1722] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.828392  1173 tablet.cc:1722] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.832906  1177 tablet.cc:1722] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.860908   877 raft_consensus.cc:504] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:33.867492   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:33.868211   808 raft_consensus.cc:3032] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:32:33.868681   808 raft_consensus.cc:729] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:33.868095  1185 tablet.cc:1722] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.869604   808 consensus_queue.cc:260] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:33.870899   808 raft_consensus.cc:3037] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:32:33.865204   729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3becd0b7224034901e3967a8a28aa9" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:33.872695   729 raft_consensus.cc:3037] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:32:33.875129  1189 tablet.cc:1722] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.884346   877 leader_election.cc:288] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 3 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:33.885742  1179 tablet.cc:1722] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.889169   729 raft_consensus.cc:2454] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 3.
I20260924 14:32:33.890832   848 leader_election.cc:302] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:33.892115  1168 raft_consensus.cc:2781] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:32:33.892580  1168 raft_consensus.cc:686] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 3 LEADER]: Becoming Leader. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:33.893533  1168 consensus_queue.cc:237] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:33.894102  1183 tablet.cc:1722] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.908324  1173 tablet.cc:1722] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.910774   625 catalog_manager.cc:5495] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d reported cstate change: term changed from 2 to 3, leader changed from bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194) to 4e24a1d2af5241c183194320fdc3915d (127.22.151.195). New cstate: current_term: 3 leader_uuid: "4e24a1d2af5241c183194320fdc3915d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:33.921694   808 raft_consensus.cc:2454] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 3.
I20260924 14:32:33.914489   727 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
mode: GRACEFUL
new_leader_uuid: "4e24a1d2af5241c183194320fdc3915d"
 from {username='slave'} at 127.0.0.1:49312
I20260924 14:32:33.924041   727 raft_consensus.cc:595] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Received request to transfer leadership to TS 4e24a1d2af5241c183194320fdc3915d
I20260924 14:32:33.925817  1177 tablet.cc:1722] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.930651 23135 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260924 14:32:33.916914  1190 tablet.cc:1722] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:33.931372 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:33.941932   729 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af"
dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
mode: GRACEFUL
new_leader_uuid: "4e24a1d2af5241c183194320fdc3915d"
 from {username='slave'} at 127.0.0.1:49312
I20260924 14:32:33.942589   729 raft_consensus.cc:595] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Received request to transfer leadership to TS 4e24a1d2af5241c183194320fdc3915d
I20260924 14:32:33.959604   808 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cafb25778918400688896e81638526b1"
dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
mode: GRACEFUL
new_leader_uuid: "4e24a1d2af5241c183194320fdc3915d"
 from {username='slave'} at 127.0.0.1:41430
I20260924 14:32:33.960264   808 raft_consensus.cc:595] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Received request to transfer leadership to TS 4e24a1d2af5241c183194320fdc3915d
I20260924 14:32:33.962015 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260924 14:32:33.962657 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:33.964149   626 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276: soft_deleted_table [id=2a51041394454728a4fc4ba090ec4ba9] alter complete (version 1)
I20260924 14:32:34.084503  1121 raft_consensus.cc:979] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21: : Instructing follower 4e24a1d2af5241c183194320fdc3915d to start an election
I20260924 14:32:34.085076  1121 raft_consensus.cc:1067] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Signalling peer 4e24a1d2af5241c183194320fdc3915d to start an election
I20260924 14:32:34.086891   877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af"
dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
 from {username='slave'} at 127.0.0.1:41844
I20260924 14:32:34.087589   877 raft_consensus.cc:482] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:34.088023   877 raft_consensus.cc:3037] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:34.092775  1121 raft_consensus.cc:979] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21: : Instructing follower 4e24a1d2af5241c183194320fdc3915d to start an election
I20260924 14:32:34.093498  1154 raft_consensus.cc:1067] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Signalling peer 4e24a1d2af5241c183194320fdc3915d to start an election
I20260924 14:32:34.095175   878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16"
dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
 from {username='slave'} at 127.0.0.1:41844
I20260924 14:32:34.095887   878 raft_consensus.cc:482] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:34.096257   878 raft_consensus.cc:3037] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:32:34.097576   877 raft_consensus.cc:504] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:34.100630   877 leader_election.cc:288] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 2 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:34.101919   729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:34.103010   729 raft_consensus.cc:3032] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:34.103046   808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d0cbc9dc474a7a9f609fcad1afd0af" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:34.103435   729 raft_consensus.cc:729] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:34.103782   808 raft_consensus.cc:3037] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:34.104342   729 consensus_queue.cc:260] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:34.105962   729 raft_consensus.cc:3037] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:34.112254   878 raft_consensus.cc:504] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:34.114415   729 raft_consensus.cc:2454] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 2.
I20260924 14:32:34.115609   848 leader_election.cc:302] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:34.116468  1168 raft_consensus.cc:2781] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:34.116936  1168 raft_consensus.cc:686] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 2 LEADER]: Becoming Leader. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:34.117491   727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:34.118376   727 raft_consensus.cc:3032] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Stepping down as leader of term 2
I20260924 14:32:34.117990  1168 consensus_queue.cc:237] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:34.118780   727 raft_consensus.cc:729] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ace4fb4d0d594edfb5ceb70d49a01d21, State: Running, Role: LEADER
I20260924 14:32:34.119774   727 consensus_queue.cc:260] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:34.120831   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70d16e3eab34d8b8d14dc793f6b3d16" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:34.121333   727 raft_consensus.cc:3037] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:32:34.121578   806 raft_consensus.cc:3037] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Advancing to term 3
I20260924 14:32:34.112931   808 raft_consensus.cc:2454] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 2.
I20260924 14:32:34.130962   626 catalog_manager.cc:5495] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d reported cstate change: term changed from 1 to 2, leader changed from ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193) to 4e24a1d2af5241c183194320fdc3915d (127.22.151.195). New cstate: current_term: 2 leader_uuid: "4e24a1d2af5241c183194320fdc3915d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:34.116726   878 leader_election.cc:288] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 3 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:34.135236   806 raft_consensus.cc:2454] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 3.
I20260924 14:32:34.137034   847 leader_election.cc:302] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, bf5ce16ee2ca4b4e9ef50f91febaffde; no voters: 
I20260924 14:32:34.137574   727 raft_consensus.cc:2454] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 3.
I20260924 14:32:34.138026  1168 raft_consensus.cc:2781] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 3 FOLLOWER]: Leader election won for term 3
I20260924 14:32:34.138605  1168 raft_consensus.cc:686] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 3 LEADER]: Becoming Leader. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:34.140069  1168 consensus_queue.cc:237] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:34.147418   626 catalog_manager.cc:5495] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d reported cstate change: term changed from 2 to 3, leader changed from ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193) to 4e24a1d2af5241c183194320fdc3915d (127.22.151.195). New cstate: current_term: 3 leader_uuid: "4e24a1d2af5241c183194320fdc3915d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:34.274731   806 raft_consensus.cc:1261] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 3 FOLLOWER]: Refusing update from remote peer 4e24a1d2af5241c183194320fdc3915d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260924 14:32:34.275918  1168 consensus_queue.cc:1035] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:32:34.283789   727 raft_consensus.cc:1261] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 3 FOLLOWER]: Refusing update from remote peer 4e24a1d2af5241c183194320fdc3915d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260924 14:32:34.285107  1168 consensus_queue.cc:1035] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:32:34.294322  1156 raft_consensus.cc:979] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde: : Instructing follower 4e24a1d2af5241c183194320fdc3915d to start an election
I20260924 14:32:34.295069  1180 raft_consensus.cc:1067] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Signalling peer 4e24a1d2af5241c183194320fdc3915d to start an election
I20260924 14:32:34.297084   877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cafb25778918400688896e81638526b1"
dest_uuid: "4e24a1d2af5241c183194320fdc3915d"
 from {username='slave'} at 127.0.0.1:41848
I20260924 14:32:34.297963   877 raft_consensus.cc:482] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:34.298347   877 raft_consensus.cc:3037] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:34.304492   877 raft_consensus.cc:504] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:34.306473   877 leader_election.cc:288] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 2 election: Requested vote from peers ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407), bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655)
I20260924 14:32:34.307341   727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cafb25778918400688896e81638526b1" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21"
I20260924 14:32:34.308072   727 raft_consensus.cc:3037] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:34.309842   806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cafb25778918400688896e81638526b1" candidate_uuid: "4e24a1d2af5241c183194320fdc3915d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde"
I20260924 14:32:34.310480   806 raft_consensus.cc:3032] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:34.310760   806 raft_consensus.cc:729] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bf5ce16ee2ca4b4e9ef50f91febaffde, State: Running, Role: LEADER
I20260924 14:32:34.311543   806 consensus_queue.cc:260] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:34.312649   806 raft_consensus.cc:3037] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:34.314150   727 raft_consensus.cc:2454] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 2.
I20260924 14:32:34.315502   848 leader_election.cc:302] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e24a1d2af5241c183194320fdc3915d, ace4fb4d0d594edfb5ceb70d49a01d21; no voters: 
I20260924 14:32:34.316448  1168 raft_consensus.cc:2781] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:34.317183  1168 raft_consensus.cc:686] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 2 LEADER]: Becoming Leader. State: Replica: 4e24a1d2af5241c183194320fdc3915d, State: Running, Role: LEADER
I20260924 14:32:34.318099  1168 consensus_queue.cc:237] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } }
I20260924 14:32:34.319201   806 raft_consensus.cc:2454] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e24a1d2af5241c183194320fdc3915d in term 2.
I20260924 14:32:34.325284   625 catalog_manager.cc:5495] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d reported cstate change: term changed from 1 to 2, leader changed from bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194) to 4e24a1d2af5241c183194320fdc3915d (127.22.151.195). New cstate: current_term: 2 leader_uuid: "4e24a1d2af5241c183194320fdc3915d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e24a1d2af5241c183194320fdc3915d" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33303 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:34.501178   727 raft_consensus.cc:1261] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Refusing update from remote peer 4e24a1d2af5241c183194320fdc3915d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:34.502367  1168 consensus_queue.cc:1035] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:34.521130   806 raft_consensus.cc:1261] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Refusing update from remote peer 4e24a1d2af5241c183194320fdc3915d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:34.522426  1168 consensus_queue.cc:1035] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:34.540365   806 raft_consensus.cc:1261] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 3 FOLLOWER]: Refusing update from remote peer 4e24a1d2af5241c183194320fdc3915d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260924 14:32:34.541677  1204 consensus_queue.cc:1035] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:32:34.549275   727 raft_consensus.cc:1261] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 3 FOLLOWER]: Refusing update from remote peer 4e24a1d2af5241c183194320fdc3915d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260924 14:32:34.550619  1168 consensus_queue.cc:1035] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260924 14:32:34.701128   727 raft_consensus.cc:1261] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Refusing update from remote peer 4e24a1d2af5241c183194320fdc3915d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:34.702633  1200 consensus_queue.cc:1035] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace4fb4d0d594edfb5ceb70d49a01d21" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:34.712567   806 raft_consensus.cc:1261] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Refusing update from remote peer 4e24a1d2af5241c183194320fdc3915d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:34.714986  1168 consensus_queue.cc:1035] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf5ce16ee2ca4b4e9ef50f91febaffde" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:34.963786 23135 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20260924 14:32:34.970201 23135 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260924 14:32:34.970715 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:34.975500 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:34.975931 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:35.977046 23135 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20260924 14:32:35.984081 23135 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260924 14:32:35.984678 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:35.989573 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:35.989916 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:36.990875 23135 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20260924 14:32:36.995862 23135 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260924 14:32:36.996222 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:37.002687 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:37.003191 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:38.004293 23135 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20260924 14:32:38.010079 23135 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260924 14:32:38.010522 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:38.015422 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:38.015863 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:39.016753 23135 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20260924 14:32:39.021868 23135 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260924 14:32:39.022282 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:39.027635 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:39.028090 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:40.029095 23135 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20260924 14:32:40.035771 23135 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260924 14:32:40.036221 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:40.041937 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:40.042361 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260924 14:32:40.855720   753 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
I20260924 14:32:41.043715 23135 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20260924 14:32:41.053380 23135 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260924 14:32:41.053938 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:41.060539 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:41.060966 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:42.061952 23135 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20260924 14:32:42.067305 23135 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260924 14:32:42.067924 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:42.073742 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:42.074177 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:43.075141 23135 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20260924 14:32:43.080286 23135 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260924 14:32:43.080772 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:43.085687 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:43.086141 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:44.159786 23135 tablet_server.cc:178] TabletServer@127.22.151.193:0 shutting down...
I20260924 14:32:44.212208 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:32:44.213073 23135 tablet_replica.cc:330] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.214290 23135 raft_consensus.cc:2229] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.215080 23135 raft_consensus.cc:2258] T 0c3becd0b7224034901e3967a8a28aa9 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.217905 23135 tablet_replica.cc:330] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.218641 23135 raft_consensus.cc:2229] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.219539 23135 raft_consensus.cc:2258] T 072183f2e81447e39727d1d0735c1935 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.222890 23135 tablet_replica.cc:330] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
W20260924 14:32:44.223721   848 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 403 similar messages]
I20260924 14:32:44.223874 23135 raft_consensus.cc:2229] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.224813 23135 raft_consensus.cc:2258] T 83cd7cae43ae489daf7c1c6b9baa5a44 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:44.227164   848 consensus_peers.cc:487] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d -> Peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Couldn't send request to peer ace4fb4d0d594edfb5ceb70d49a01d21. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:44.227818 23135 tablet_replica.cc:330] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.228489 23135 raft_consensus.cc:2229] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.229471 23135 raft_consensus.cc:2258] T ff761eb6473d48e5984c882cd2a79f08 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.231905 23135 tablet_replica.cc:330] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.232546 23135 raft_consensus.cc:2229] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.233583 23135 raft_consensus.cc:2258] T 28e3e80523cc4c4fabe9188b05b82f3e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.236510 23135 tablet_replica.cc:330] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.237167 23135 raft_consensus.cc:2229] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.237713 23135 raft_consensus.cc:2258] T cafb25778918400688896e81638526b1 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.240118 23135 tablet_replica.cc:330] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.240722 23135 raft_consensus.cc:2229] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.241219 23135 raft_consensus.cc:2258] T b70d16e3eab34d8b8d14dc793f6b3d16 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.243713 23135 tablet_replica.cc:330] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.244464 23135 raft_consensus.cc:2229] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.245481 23135 raft_consensus.cc:2258] T 2ed05026e64d41969eae5584bc00ecfb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.248855 23135 tablet_replica.cc:330] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.249807 23135 raft_consensus.cc:2229] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.250562 23135 raft_consensus.cc:2258] T 6ecaeb9d96a24ef48cba480a8967ee2e P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:44.252126   774 consensus_peers.cc:487] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde -> Peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Couldn't send request to peer ace4fb4d0d594edfb5ceb70d49a01d21. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:44.253939 23135 tablet_replica.cc:330] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.254913 23135 raft_consensus.cc:2229] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.256219 23135 raft_consensus.cc:2258] T 6b5138cabfa044f5847754cfd426e9e5 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.259359 23135 tablet_replica.cc:330] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.260008 23135 raft_consensus.cc:2229] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.260778 23135 raft_consensus.cc:2258] T 481a3cffff2f4a21bed6fa7c0febc169 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.263186 23135 tablet_replica.cc:330] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.263779 23135 raft_consensus.cc:2229] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.264199 23135 raft_consensus.cc:2258] T c9fbbb43543c46609c18b6b74d2a0c7d P ace4fb4d0d594edfb5ceb70d49a01d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.266510 23135 tablet_replica.cc:330] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.267009 23135 raft_consensus.cc:2229] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.267436 23135 raft_consensus.cc:2258] T 2bc6db8074954aa786d6b4df26708dcb P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.270249 23135 tablet_replica.cc:330] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.271078 23135 raft_consensus.cc:2229] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.271726 23135 raft_consensus.cc:2258] T 300151d841e346df92555439cc0f57b8 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.275010 23135 tablet_replica.cc:330] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.275620 23135 raft_consensus.cc:2229] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.276175 23135 raft_consensus.cc:2258] T 7d68635ab99d499696c0d9719093cdf6 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.278414 23135 tablet_replica.cc:330] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.278894 23135 raft_consensus.cc:2229] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.279868 23135 raft_consensus.cc:2258] T d5d35424e0f84f9ebd9f02523e4e9526 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.282035 23135 tablet_replica.cc:330] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.282563 23135 raft_consensus.cc:2229] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.283550 23135 raft_consensus.cc:2258] T 015d56000be4414ca2ba6701f347bef0 P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.286630 23135 tablet_replica.cc:330] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21: stopping tablet replica
I20260924 14:32:44.287159 23135 raft_consensus.cc:2229] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.287786 23135 raft_consensus.cc:2258] T 20d0cbc9dc474a7a9f609fcad1afd0af P ace4fb4d0d594edfb5ceb70d49a01d21 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:44.299356   774 consensus_peers.cc:487] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde -> Peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Couldn't send request to peer ace4fb4d0d594edfb5ceb70d49a01d21. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:44.342723 23135 tablet_server.cc:194] TabletServer@127.22.151.193:0 shutdown complete.
W20260924 14:32:44.355517   774 consensus_peers.cc:487] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde -> Peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Couldn't send request to peer ace4fb4d0d594edfb5ceb70d49a01d21. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:40407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:32:44.363413   848 consensus_peers.cc:487] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d -> Peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Couldn't send request to peer ace4fb4d0d594edfb5ceb70d49a01d21. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:40407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:44.394024 23135 tablet_server.cc:178] TabletServer@127.22.151.194:0 shutting down...
I20260924 14:32:44.434450 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:32:44.435271 23135 tablet_replica.cc:330] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.436276 23135 raft_consensus.cc:2229] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.437414 23135 raft_consensus.cc:2258] T 83cd7cae43ae489daf7c1c6b9baa5a44 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.440088 23135 tablet_replica.cc:330] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.440798 23135 raft_consensus.cc:2229] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.441758 23135 raft_consensus.cc:2258] T 072183f2e81447e39727d1d0735c1935 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.444411 23135 tablet_replica.cc:330] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.445055 23135 raft_consensus.cc:2229] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.445607 23135 raft_consensus.cc:2258] T 28e3e80523cc4c4fabe9188b05b82f3e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.448535 23135 tablet_replica.cc:330] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.449411 23135 raft_consensus.cc:2229] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.450163 23135 raft_consensus.cc:2258] T 2ed05026e64d41969eae5584bc00ecfb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:44.451500   848 consensus_peers.cc:487] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d -> Peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Couldn't send request to peer ace4fb4d0d594edfb5ceb70d49a01d21. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:40407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:44.452921 23135 tablet_replica.cc:330] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.453675 23135 raft_consensus.cc:2229] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.454258 23135 raft_consensus.cc:2258] T 0c3becd0b7224034901e3967a8a28aa9 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.457185 23135 tablet_replica.cc:330] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.457898 23135 raft_consensus.cc:2229] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.458480 23135 raft_consensus.cc:2258] T 015d56000be4414ca2ba6701f347bef0 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.461273 23135 tablet_replica.cc:330] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.461920 23135 raft_consensus.cc:2229] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.462446 23135 raft_consensus.cc:2258] T ff761eb6473d48e5984c882cd2a79f08 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.465754 23135 tablet_replica.cc:330] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.466581 23135 raft_consensus.cc:2229] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.467182 23135 raft_consensus.cc:2258] T 481a3cffff2f4a21bed6fa7c0febc169 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:44.467381   848 consensus_peers.cc:487] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d -> Peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Couldn't send request to peer ace4fb4d0d594edfb5ceb70d49a01d21. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:40407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:44.469973 23135 tablet_replica.cc:330] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.470912 23135 raft_consensus.cc:2229] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.472087 23135 raft_consensus.cc:2258] T c9fbbb43543c46609c18b6b74d2a0c7d P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.475003 23135 tablet_replica.cc:330] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.475751 23135 raft_consensus.cc:2229] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.476749 23135 raft_consensus.cc:2258] T 2bc6db8074954aa786d6b4df26708dcb P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.479161 23135 tablet_replica.cc:330] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.479804 23135 raft_consensus.cc:2229] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.480324 23135 raft_consensus.cc:2258] T d5d35424e0f84f9ebd9f02523e4e9526 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.482810 23135 tablet_replica.cc:330] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.483557 23135 raft_consensus.cc:2229] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.484242 23135 raft_consensus.cc:2258] T 7d68635ab99d499696c0d9719093cdf6 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.487808 23135 tablet_replica.cc:330] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.488793 23135 raft_consensus.cc:2229] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.489444 23135 raft_consensus.cc:2258] T 20d0cbc9dc474a7a9f609fcad1afd0af P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:44.492211   847 consensus_peers.cc:487] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d -> Peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Couldn't send request to peer bf5ce16ee2ca4b4e9ef50f91febaffde. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:44.493072 23135 tablet_replica.cc:330] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.493762 23135 raft_consensus.cc:2229] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.494706 23135 raft_consensus.cc:2258] T 300151d841e346df92555439cc0f57b8 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.497342 23135 tablet_replica.cc:330] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.497956 23135 raft_consensus.cc:2229] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.498528 23135 raft_consensus.cc:2258] T b70d16e3eab34d8b8d14dc793f6b3d16 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.501438 23135 tablet_replica.cc:330] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.502110 23135 raft_consensus.cc:2229] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.502967 23135 raft_consensus.cc:2258] T 6ecaeb9d96a24ef48cba480a8967ee2e P bf5ce16ee2ca4b4e9ef50f91febaffde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.505517 23135 tablet_replica.cc:330] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.506112 23135 raft_consensus.cc:2229] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.506659 23135 raft_consensus.cc:2258] T 6b5138cabfa044f5847754cfd426e9e5 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.509115 23135 tablet_replica.cc:330] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde: stopping tablet replica
I20260924 14:32:44.509595 23135 raft_consensus.cc:2229] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.510074 23135 raft_consensus.cc:2258] T cafb25778918400688896e81638526b1 P bf5ce16ee2ca4b4e9ef50f91febaffde [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:44.520664   848 consensus_peers.cc:487] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d -> Peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Couldn't send request to peer ace4fb4d0d594edfb5ceb70d49a01d21. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:40407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:44.556104 23135 tablet_server.cc:194] TabletServer@127.22.151.194:0 shutdown complete.
W20260924 14:32:44.584633   848 consensus_peers.cc:487] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d -> Peer ace4fb4d0d594edfb5ceb70d49a01d21 (127.22.151.193:40407): Couldn't send request to peer ace4fb4d0d594edfb5ceb70d49a01d21. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:40407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:44.600009 23135 tablet_server.cc:178] TabletServer@127.22.151.195:0 shutting down...
I20260924 14:32:44.633582 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:32:44.634464 23135 tablet_replica.cc:330] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.635408 23135 raft_consensus.cc:2229] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.636135 23135 raft_consensus.cc:2258] T 2ed05026e64d41969eae5584bc00ecfb P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.638800 23135 tablet_replica.cc:330] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.639761 23135 raft_consensus.cc:2229] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.640535 23135 raft_consensus.cc:2258] T ff761eb6473d48e5984c882cd2a79f08 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.643412 23135 tablet_replica.cc:330] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.644157 23135 raft_consensus.cc:2229] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.645072 23135 raft_consensus.cc:2258] T b70d16e3eab34d8b8d14dc793f6b3d16 P 4e24a1d2af5241c183194320fdc3915d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.647275 23135 tablet_replica.cc:330] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.647924 23135 raft_consensus.cc:2229] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.648484 23135 raft_consensus.cc:2258] T 28e3e80523cc4c4fabe9188b05b82f3e P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.650811 23135 tablet_replica.cc:330] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.651650 23135 raft_consensus.cc:2229] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.652259 23135 raft_consensus.cc:2258] T 83cd7cae43ae489daf7c1c6b9baa5a44 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:32:44.653808   847 consensus_peers.cc:487] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d -> Peer bf5ce16ee2ca4b4e9ef50f91febaffde (127.22.151.194:39655): Couldn't send request to peer bf5ce16ee2ca4b4e9ef50f91febaffde. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:39655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:32:44.655000 23135 tablet_replica.cc:330] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.655704 23135 raft_consensus.cc:2229] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.656215 23135 raft_consensus.cc:2258] T 072183f2e81447e39727d1d0735c1935 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.658416 23135 tablet_replica.cc:330] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.659003 23135 raft_consensus.cc:2229] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 3 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.659927 23135 raft_consensus.cc:2258] T 0c3becd0b7224034901e3967a8a28aa9 P 4e24a1d2af5241c183194320fdc3915d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.662050 23135 tablet_replica.cc:330] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.662618 23135 raft_consensus.cc:2229] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.663045 23135 raft_consensus.cc:2258] T c9fbbb43543c46609c18b6b74d2a0c7d P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.665321 23135 tablet_replica.cc:330] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.665921 23135 raft_consensus.cc:2229] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.666808 23135 raft_consensus.cc:2258] T 481a3cffff2f4a21bed6fa7c0febc169 P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.669039 23135 tablet_replica.cc:330] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.669555 23135 raft_consensus.cc:2229] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.670466 23135 raft_consensus.cc:2258] T cafb25778918400688896e81638526b1 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.672639 23135 tablet_replica.cc:330] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.673228 23135 raft_consensus.cc:2229] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.673744 23135 raft_consensus.cc:2258] T 6ecaeb9d96a24ef48cba480a8967ee2e P 4e24a1d2af5241c183194320fdc3915d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.675834 23135 tablet_replica.cc:330] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.676345 23135 raft_consensus.cc:2229] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.676800 23135 raft_consensus.cc:2258] T 6b5138cabfa044f5847754cfd426e9e5 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.678879 23135 tablet_replica.cc:330] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.679381 23135 raft_consensus.cc:2229] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.679881 23135 raft_consensus.cc:2258] T 015d56000be4414ca2ba6701f347bef0 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.681967 23135 tablet_replica.cc:330] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.682760 23135 raft_consensus.cc:2229] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.683879 23135 raft_consensus.cc:2258] T 7d68635ab99d499696c0d9719093cdf6 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.686182 23135 tablet_replica.cc:330] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.686738 23135 raft_consensus.cc:2229] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.687290 23135 raft_consensus.cc:2258] T d5d35424e0f84f9ebd9f02523e4e9526 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.689767 23135 tablet_replica.cc:330] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.690359 23135 raft_consensus.cc:2229] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.690863 23135 raft_consensus.cc:2258] T 300151d841e346df92555439cc0f57b8 P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.693416 23135 tablet_replica.cc:330] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.693939 23135 raft_consensus.cc:2229] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:32:44.694424 23135 raft_consensus.cc:2258] T 2bc6db8074954aa786d6b4df26708dcb P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.696885 23135 tablet_replica.cc:330] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d: stopping tablet replica
I20260924 14:32:44.697404 23135 raft_consensus.cc:2229] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.698410 23135 raft_consensus.cc:2258] T 20d0cbc9dc474a7a9f609fcad1afd0af P 4e24a1d2af5241c183194320fdc3915d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.742324 23135 tablet_server.cc:194] TabletServer@127.22.151.195:0 shutdown complete.
I20260924 14:32:44.779896 23135 master.cc:537] Master@127.22.151.254:39009 shutting down...
I20260924 14:32:44.809476 23135 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:32:44.810132 23135 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:32:44.810491 23135 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8133d61326764837995824b7933a9276: stopping tablet replica
I20260924 14:32:44.832017 23135 master.cc:552] Master@127.22.151.254:39009 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21236 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260924 14:32:44.891769 23135 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.22.151.254:46621
I20260924 14:32:44.893085 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:32:44.899406  1239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:44.900454  1242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:44.900722 23135 server_base.cc:1034] running on GCE node
W20260924 14:32:44.900622  1240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:44.902166 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:32:44.902423 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:32:44.902608 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260364902594 us; error 0 us; skew 500 ppm
I20260924 14:32:44.903182 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:32:44.905851 23135 webserver.cc:460] Webserver started at http://127.22.151.254:39245/ using document root <none> and password file <none>
I20260924 14:32:44.906394 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:32:44.906587 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:32:44.906906 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:32:44.908453 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/master-0-root/instance:
uuid: "58bfe2506c3a4639a24234a3d914debe"
format_stamp: "Formatted at 2026-09-24 14:32:44 on dist-test-slave-pm5x"
I20260924 14:32:44.913179 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260924 14:32:44.916814  1247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:44.917645 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260924 14:32:44.917990 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/master-0-root
uuid: "58bfe2506c3a4639a24234a3d914debe"
format_stamp: "Formatted at 2026-09-24 14:32:44 on dist-test-slave-pm5x"
I20260924 14:32:44.918290 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:32:44.942824 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:32:44.944190 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:32:44.983773 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.254:46621
I20260924 14:32:44.983947  1298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.254:46621 every 8 connection(s)
I20260924 14:32:44.987854  1300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.002107  1300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe: Bootstrap starting.
I20260924 14:32:45.007289  1300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.012059  1300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe: No bootstrap required, opened a new log
I20260924 14:32:45.014470  1300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bfe2506c3a4639a24234a3d914debe" member_type: VOTER }
I20260924 14:32:45.014988  1300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:45.015295  1300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58bfe2506c3a4639a24234a3d914debe, State: Initialized, Role: FOLLOWER
I20260924 14:32:45.016088  1300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bfe2506c3a4639a24234a3d914debe" member_type: VOTER }
I20260924 14:32:45.016620  1300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 14:32:45.016884  1300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 14:32:45.017186  1300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:45.022845  1300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bfe2506c3a4639a24234a3d914debe" member_type: VOTER }
I20260924 14:32:45.023456  1300 leader_election.cc:302] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58bfe2506c3a4639a24234a3d914debe; no voters: 
I20260924 14:32:45.024650  1300 leader_election.cc:288] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260924 14:32:45.025151  1303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:45.026634  1303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 1 LEADER]: Becoming Leader. State: Replica: 58bfe2506c3a4639a24234a3d914debe, State: Running, Role: LEADER
I20260924 14:32:45.027262  1303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bfe2506c3a4639a24234a3d914debe" member_type: VOTER }
I20260924 14:32:45.027935  1300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [sys.catalog]: configured and running, proceeding with master startup.
I20260924 14:32:45.032932  1304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58bfe2506c3a4639a24234a3d914debe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bfe2506c3a4639a24234a3d914debe" member_type: VOTER } }
I20260924 14:32:45.033272  1305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58bfe2506c3a4639a24234a3d914debe. Latest consensus state: current_term: 1 leader_uuid: "58bfe2506c3a4639a24234a3d914debe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bfe2506c3a4639a24234a3d914debe" member_type: VOTER } }
I20260924 14:32:45.033967  1304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [sys.catalog]: This master's current role is: LEADER
I20260924 14:32:45.034492  1305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [sys.catalog]: This master's current role is: LEADER
I20260924 14:32:45.037457  1310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 14:32:45.042565  1310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 14:32:45.048007 23135 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260924 14:32:45.052695  1310 catalog_manager.cc:1347] Generated new cluster ID: 34c2e3a028c84f09a238141ded86257a
I20260924 14:32:45.053016  1310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 14:32:45.078327  1310 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 14:32:45.080242  1310 catalog_manager.cc:1504] Loading token signing keys...
I20260924 14:32:45.094515  1310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe: Generated new TSK 0
I20260924 14:32:45.095240  1310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 14:32:45.116101 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:32:45.122473  1321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:45.123811  1322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:45.125744  1324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:45.126118 23135 server_base.cc:1034] running on GCE node
I20260924 14:32:45.127244 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:32:45.127447 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:32:45.127643 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260365127609 us; error 0 us; skew 500 ppm
I20260924 14:32:45.128264 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:32:45.130868 23135 webserver.cc:460] Webserver started at http://127.22.151.193:38415/ using document root <none> and password file <none>
I20260924 14:32:45.131366 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:32:45.131615 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:32:45.131901 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:32:45.133122 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-0-root/instance:
uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
format_stamp: "Formatted at 2026-09-24 14:32:45 on dist-test-slave-pm5x"
I20260924 14:32:45.138093 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260924 14:32:45.142215  1329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:45.143199 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260924 14:32:45.143600 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-0-root
uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
format_stamp: "Formatted at 2026-09-24 14:32:45 on dist-test-slave-pm5x"
I20260924 14:32:45.143893 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:32:45.168668 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:32:45.170032 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:32:45.171591 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:32:45.173878 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:32:45.174090 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:45.174358 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:32:45.174515 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:45.216830 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.193:43153
I20260924 14:32:45.216951  1391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.193:43153 every 8 connection(s)
I20260924 14:32:45.221601 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:32:45.229417  1396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:45.230731  1397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:45.234269  1399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:45.234445 23135 server_base.cc:1034] running on GCE node
I20260924 14:32:45.235846 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:32:45.236115 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:32:45.236333 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260365236307 us; error 0 us; skew 500 ppm
I20260924 14:32:45.237097 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:32:45.239204  1392 heartbeater.cc:347] Connected to a master server at 127.22.151.254:46621
I20260924 14:32:45.239750  1392 heartbeater.cc:464] Registering TS with master...
I20260924 14:32:45.240005 23135 webserver.cc:460] Webserver started at http://127.22.151.194:39259/ using document root <none> and password file <none>
I20260924 14:32:45.240670 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:32:45.240728  1392 heartbeater.cc:511] Master 127.22.151.254:46621 requested a full tablet report, sending...
I20260924 14:32:45.240851 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:32:45.241458 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:32:45.242928 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-1-root/instance:
uuid: "91f8d84a8caf421d99048c427fc3b5a6"
format_stamp: "Formatted at 2026-09-24 14:32:45 on dist-test-slave-pm5x"
I20260924 14:32:45.243535  1264 ts_manager.cc:194] Registered new tserver with Master: a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153)
I20260924 14:32:45.245893  1264 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51490
I20260924 14:32:45.248190 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260924 14:32:45.252566  1404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:45.253590 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260924 14:32:45.253957 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-1-root
uuid: "91f8d84a8caf421d99048c427fc3b5a6"
format_stamp: "Formatted at 2026-09-24 14:32:45 on dist-test-slave-pm5x"
I20260924 14:32:45.254297 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:32:45.295861 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:32:45.297171 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:32:45.298620 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:32:45.300925 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:32:45.301141 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:45.301381 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:32:45.301546 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:45.345489 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.194:38407
I20260924 14:32:45.345654  1468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.194:38407 every 8 connection(s)
I20260924 14:32:45.350435 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 14:32:45.358664  1472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 14:32:45.360518  1473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:45.365592  1469 heartbeater.cc:347] Connected to a master server at 127.22.151.254:46621
I20260924 14:32:45.366189  1469 heartbeater.cc:464] Registering TS with master...
I20260924 14:32:45.367158  1469 heartbeater.cc:511] Master 127.22.151.254:46621 requested a full tablet report, sending...
I20260924 14:32:45.369990  1264 ts_manager.cc:194] Registered new tserver with Master: 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:45.372994  1264 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51504
W20260924 14:32:45.373704  1475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 14:32:45.375733 23135 server_base.cc:1034] running on GCE node
I20260924 14:32:45.376847 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260924 14:32:45.377084 23135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260924 14:32:45.377275 23135 hybrid_clock.cc:648] HybridClock initialized: now 1790260365377257 us; error 0 us; skew 500 ppm
I20260924 14:32:45.377866 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 14:32:45.380770 23135 webserver.cc:460] Webserver started at http://127.22.151.195:43685/ using document root <none> and password file <none>
I20260924 14:32:45.381242 23135 fs_manager.cc:359] Metadata directory not provided
I20260924 14:32:45.381412 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 14:32:45.381615 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 14:32:45.382856 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-2-root/instance:
uuid: "0fb563febbcf4233b7a125899b66ce05"
format_stamp: "Formatted at 2026-09-24 14:32:45 on dist-test-slave-pm5x"
I20260924 14:32:45.387637 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260924 14:32:45.391235  1480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:45.392149 23135 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260924 14:32:45.392680 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-2-root
uuid: "0fb563febbcf4233b7a125899b66ce05"
format_stamp: "Formatted at 2026-09-24 14:32:45 on dist-test-slave-pm5x"
I20260924 14:32:45.393194 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdFx40I/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1790260019751168-23135-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 14:32:45.426507 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
I20260924 14:32:45.427915 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 14:32:45.429291 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 14:32:45.431651 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 14:32:45.431882 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:45.432113 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 14:32:45.432276 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260924 14:32:45.473609 23135 rpc_server.cc:307] RPC server started. Bound to: 127.22.151.195:33825
I20260924 14:32:45.473783  1544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.151.195:33825 every 8 connection(s)
I20260924 14:32:45.490362  1545 heartbeater.cc:347] Connected to a master server at 127.22.151.254:46621
I20260924 14:32:45.490931  1545 heartbeater.cc:464] Registering TS with master...
I20260924 14:32:45.491707  1545 heartbeater.cc:511] Master 127.22.151.254:46621 requested a full tablet report, sending...
I20260924 14:32:45.493678  1264 ts_manager.cc:194] Registered new tserver with Master: 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:45.494293 23135 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017311927s
I20260924 14:32:45.495612  1264 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51518
I20260924 14:32:45.497079 23135 test_util.cc:268] Using random seed: 2032407263
I20260924 14:32:45.522987  1264 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51530:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260924 14:32:45.525933  1264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 14:32:45.618074  1507 tablet_service.cc:1467] Processing CreateTablet for tablet c843297774c94a31990bd30dc7bb622d (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:45.619753  1507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c843297774c94a31990bd30dc7bb622d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.620417  1508 tablet_service.cc:1467] Processing CreateTablet for tablet e4d0e3cd411a44ba8e4f9fd59b6e0fb5 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:45.621506  1508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4d0e3cd411a44ba8e4f9fd59b6e0fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.624585  1506 tablet_service.cc:1467] Processing CreateTablet for tablet e4216c32baf34f488c831ce881873cfa (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:45.626076  1506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4216c32baf34f488c831ce881873cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.625715  1503 tablet_service.cc:1467] Processing CreateTablet for tablet c8a559653d744c3ab37c5b1b2b1f1b17 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:45.627527  1504 tablet_service.cc:1467] Processing CreateTablet for tablet 7f656b71445647da9ffe7e0ed4e51ca8 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:45.628332  1503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a559653d744c3ab37c5b1b2b1f1b17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.628906  1504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f656b71445647da9ffe7e0ed4e51ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.628777  1505 tablet_service.cc:1467] Processing CreateTablet for tablet 5701334222ec4b839d1dba26d7b14599 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:45.624585  1502 tablet_service.cc:1467] Processing CreateTablet for tablet 96caf2a8ba5d4d5bafb9c6dd0ab6d310 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:45.630251  1505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5701334222ec4b839d1dba26d7b14599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.634842  1502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96caf2a8ba5d4d5bafb9c6dd0ab6d310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.635102  1500 tablet_service.cc:1467] Processing CreateTablet for tablet 27b3ccc05ab7495baec6265cd9c3c35d (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:45.636473  1500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27b3ccc05ab7495baec6265cd9c3c35d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.640717  1501 tablet_service.cc:1467] Processing CreateTablet for tablet 2c8e875b068d47f4a5a6ca1915609625 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:45.642164  1501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c8e875b068d47f4a5a6ca1915609625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.645748  1433 tablet_service.cc:1467] Processing CreateTablet for tablet e4d0e3cd411a44ba8e4f9fd59b6e0fb5 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:45.647194  1433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4d0e3cd411a44ba8e4f9fd59b6e0fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.648703  1432 tablet_service.cc:1467] Processing CreateTablet for tablet c843297774c94a31990bd30dc7bb622d (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:45.651044  1431 tablet_service.cc:1467] Processing CreateTablet for tablet e4216c32baf34f488c831ce881873cfa (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:45.653402  1430 tablet_service.cc:1467] Processing CreateTablet for tablet 5701334222ec4b839d1dba26d7b14599 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:45.655557  1426 tablet_service.cc:1467] Processing CreateTablet for tablet 96caf2a8ba5d4d5bafb9c6dd0ab6d310 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:45.657718  1427 tablet_service.cc:1467] Processing CreateTablet for tablet c8a559653d744c3ab37c5b1b2b1f1b17 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:45.660135  1428 tablet_service.cc:1467] Processing CreateTablet for tablet 7f656b71445647da9ffe7e0ed4e51ca8 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:45.661871  1430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5701334222ec4b839d1dba26d7b14599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.663590  1425 tablet_service.cc:1467] Processing CreateTablet for tablet 2c8e875b068d47f4a5a6ca1915609625 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:45.663825  1424 tablet_service.cc:1467] Processing CreateTablet for tablet 27b3ccc05ab7495baec6265cd9c3c35d (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:45.665450  1424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27b3ccc05ab7495baec6265cd9c3c35d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.666102  1431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4216c32baf34f488c831ce881873cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.667623  1432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c843297774c94a31990bd30dc7bb622d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.668464  1428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f656b71445647da9ffe7e0ed4e51ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.681226  1426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96caf2a8ba5d4d5bafb9c6dd0ab6d310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.691910  1425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c8e875b068d47f4a5a6ca1915609625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.695375  1427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a559653d744c3ab37c5b1b2b1f1b17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.726620  1560 tablet_bootstrap.cc:492] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:45.734091  1560 tablet_bootstrap.cc:654] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.744436  1560 tablet_bootstrap.cc:492] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:45.745148  1560 ts_tablet_manager.cc:1397] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260924 14:32:45.748509  1560 raft_consensus.cc:348] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.749143  1560 raft_consensus.cc:374] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:45.749421  1560 raft_consensus.cc:729] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:45.750182  1560 consensus_queue.cc:260] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.753346  1545 heartbeater.cc:503] Master 127.22.151.254:46621 was elected leader, sending a full tablet report...
I20260924 14:32:45.753366  1560 ts_tablet_manager.cc:1428] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:32:45.754812  1560 tablet_bootstrap.cc:492] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:45.761225  1560 tablet_bootstrap.cc:654] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.767900  1560 tablet_bootstrap.cc:492] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:45.768389  1560 ts_tablet_manager.cc:1397] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260924 14:32:45.771204  1560 raft_consensus.cc:348] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.771997  1560 raft_consensus.cc:374] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:45.772331  1560 raft_consensus.cc:729] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:45.773087  1560 consensus_queue.cc:260] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.774981  1560 ts_tablet_manager.cc:1428] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:32:45.775892  1560 tablet_bootstrap.cc:492] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:45.782122  1560 tablet_bootstrap.cc:654] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.802979  1565 tablet_bootstrap.cc:492] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:45.805271  1560 tablet_bootstrap.cc:492] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:45.805927  1560 ts_tablet_manager.cc:1397] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.015s
I20260924 14:32:45.809089  1560 raft_consensus.cc:348] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.809924  1560 raft_consensus.cc:374] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:45.810351  1560 raft_consensus.cc:729] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:45.811088  1560 consensus_queue.cc:260] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.813416  1560 ts_tablet_manager.cc:1428] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:32:45.814499  1560 tablet_bootstrap.cc:492] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:45.820910  1560 tablet_bootstrap.cc:654] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.826984  1560 tablet_bootstrap.cc:492] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:45.827585  1560 ts_tablet_manager.cc:1397] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.013s	user 0.013s	sys 0.000s
I20260924 14:32:45.830154  1560 raft_consensus.cc:348] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.830958  1560 raft_consensus.cc:374] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:45.831264  1560 raft_consensus.cc:729] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:45.832049  1560 consensus_queue.cc:260] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.834517  1560 ts_tablet_manager.cc:1428] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260924 14:32:45.835664  1560 tablet_bootstrap.cc:492] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:45.842528  1560 tablet_bootstrap.cc:654] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.849603  1560 tablet_bootstrap.cc:492] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:45.865139  1565 tablet_bootstrap.cc:654] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.871340  1565 tablet_bootstrap.cc:492] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:45.871925  1565 ts_tablet_manager.cc:1397] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.071s	user 0.016s	sys 0.008s
I20260924 14:32:45.874809  1565 raft_consensus.cc:348] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.875612  1565 raft_consensus.cc:374] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:45.875900  1565 raft_consensus.cc:729] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:45.876596  1565 consensus_queue.cc:260] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.879213  1469 heartbeater.cc:503] Master 127.22.151.254:46621 was elected leader, sending a full tablet report...
I20260924 14:32:45.879644  1565 ts_tablet_manager.cc:1428] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:32:45.880807  1565 tablet_bootstrap.cc:492] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:45.886386  1565 tablet_bootstrap.cc:654] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.892480  1565 tablet_bootstrap.cc:492] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:45.893030  1565 ts_tablet_manager.cc:1397] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.012s	user 0.006s	sys 0.004s
I20260924 14:32:45.895831  1565 raft_consensus.cc:348] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.896638  1565 raft_consensus.cc:374] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:45.896965  1565 raft_consensus.cc:729] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:45.897822  1565 consensus_queue.cc:260] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.898411  1562 raft_consensus.cc:482] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:45.898996  1562 raft_consensus.cc:504] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.900382  1565 ts_tablet_manager.cc:1428] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:32:45.901503  1565 tablet_bootstrap.cc:492] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:45.904017  1562 leader_election.cc:288] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:45.908078  1565 tablet_bootstrap.cc:654] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.914057  1565 tablet_bootstrap.cc:492] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:45.914476  1569 raft_consensus.cc:482] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:45.914638  1565 ts_tablet_manager.cc:1397] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.013s	user 0.007s	sys 0.003s
I20260924 14:32:45.915107  1569 raft_consensus.cc:504] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.917774  1565 raft_consensus.cc:348] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.918661  1565 raft_consensus.cc:374] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:45.918983  1565 raft_consensus.cc:729] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:45.919909  1565 consensus_queue.cc:260] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.922665  1569 leader_election.cc:288] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:45.925307  1565 ts_tablet_manager.cc:1428] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260924 14:32:45.926385  1565 tablet_bootstrap.cc:492] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:45.932312  1565 tablet_bootstrap.cc:654] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.932698  1356 tablet_service.cc:1467] Processing CreateTablet for tablet c843297774c94a31990bd30dc7bb622d (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:45.934047  1356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c843297774c94a31990bd30dc7bb622d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.934513  1355 tablet_service.cc:1467] Processing CreateTablet for tablet e4216c32baf34f488c831ce881873cfa (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:45.935959  1355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4216c32baf34f488c831ce881873cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.936589  1354 tablet_service.cc:1467] Processing CreateTablet for tablet 5701334222ec4b839d1dba26d7b14599 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:45.937553  1354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5701334222ec4b839d1dba26d7b14599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.937928  1565 tablet_bootstrap.cc:492] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:45.938465  1565 ts_tablet_manager.cc:1397] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.012s	user 0.006s	sys 0.003s
I20260924 14:32:45.940515  1353 tablet_service.cc:1467] Processing CreateTablet for tablet 7f656b71445647da9ffe7e0ed4e51ca8 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:45.941965  1353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f656b71445647da9ffe7e0ed4e51ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.941427  1565 raft_consensus.cc:348] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.932704  1357 tablet_service.cc:1467] Processing CreateTablet for tablet e4d0e3cd411a44ba8e4f9fd59b6e0fb5 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:45.942371  1565 raft_consensus.cc:374] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:45.942721  1565 raft_consensus.cc:729] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:45.943097  1357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4d0e3cd411a44ba8e4f9fd59b6e0fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.940569  1351 tablet_service.cc:1467] Processing CreateTablet for tablet 96caf2a8ba5d4d5bafb9c6dd0ab6d310 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:45.943491  1565 consensus_queue.cc:260] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.944435  1351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96caf2a8ba5d4d5bafb9c6dd0ab6d310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.946096  1565 ts_tablet_manager.cc:1428] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:32:45.947201  1565 tablet_bootstrap.cc:492] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:45.946930  1349 tablet_service.cc:1467] Processing CreateTablet for tablet 27b3ccc05ab7495baec6265cd9c3c35d (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:45.948277  1349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27b3ccc05ab7495baec6265cd9c3c35d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.954317  1565 tablet_bootstrap.cc:654] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:45.958621  1562 raft_consensus.cc:482] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:45.959627  1562 raft_consensus.cc:504] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.960896  1352 tablet_service.cc:1467] Processing CreateTablet for tablet c8a559653d744c3ab37c5b1b2b1f1b17 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:45.962610  1352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a559653d744c3ab37c5b1b2b1f1b17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.965065  1350 tablet_service.cc:1467] Processing CreateTablet for tablet 2c8e875b068d47f4a5a6ca1915609625 (DEFAULT_TABLE table=test-workload [id=267218d6b8114b31b88548b578d814f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:45.965914  1562 leader_election.cc:288] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:45.966499  1350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c8e875b068d47f4a5a6ca1915609625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:45.970431  1560 ts_tablet_manager.cc:1397] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.135s	user 0.002s	sys 0.012s
I20260924 14:32:45.979614  1562 raft_consensus.cc:482] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:45.983115  1562 raft_consensus.cc:504] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.981853  1560 raft_consensus.cc:348] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.985314  1560 raft_consensus.cc:374] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:45.985802  1560 raft_consensus.cc:729] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:45.987481  1560 consensus_queue.cc:260] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:45.996776  1562 leader_election.cc:288] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:46.039290  1560 ts_tablet_manager.cc:1428] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.068s	user 0.009s	sys 0.004s
I20260924 14:32:46.040477  1560 tablet_bootstrap.cc:492] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
W20260924 14:32:46.048028  1313 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:32:46.052873  1560 tablet_bootstrap.cc:654] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.054005  1565 tablet_bootstrap.cc:492] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:46.054610  1565 ts_tablet_manager.cc:1397] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.108s	user 0.036s	sys 0.028s
I20260924 14:32:46.055979  1442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96caf2a8ba5d4d5bafb9c6dd0ab6d310" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:46.057150  1442 raft_consensus.cc:2454] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:46.062929  1482 leader_election.cc:302] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
W20260924 14:32:46.063978  1482 leader_election.cc:341] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407): Not found: Tablet not found: c843297774c94a31990bd30dc7bb622d
I20260924 14:32:46.065868  1579 raft_consensus.cc:2781] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:46.066454  1579 raft_consensus.cc:482] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:32:46.066357  1482 leader_election.cc:341] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407): Not found: Tablet not found: 5701334222ec4b839d1dba26d7b14599
I20260924 14:32:46.072094  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96caf2a8ba5d4d5bafb9c6dd0ab6d310" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:46.079767  1565 raft_consensus.cc:348] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.085860  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5701334222ec4b839d1dba26d7b14599" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:46.069386  1579 raft_consensus.cc:3037] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.087877  1565 raft_consensus.cc:374] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.092334  1565 raft_consensus.cc:729] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.096009  1560 tablet_bootstrap.cc:492] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:46.095220  1565 consensus_queue.cc:260] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.096596  1560 ts_tablet_manager.cc:1397] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.056s	user 0.009s	sys 0.002s
I20260924 14:32:46.099685  1560 raft_consensus.cc:348] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.100409  1560 raft_consensus.cc:374] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.100685  1560 raft_consensus.cc:729] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.101403  1560 consensus_queue.cc:260] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.103191  1560 ts_tablet_manager.cc:1428] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:32:46.104274  1560 tablet_bootstrap.cc:492] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:46.108350  1579 raft_consensus.cc:504] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.109753  1518 raft_consensus.cc:2379] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f8d84a8caf421d99048c427fc3b5a6 in current term 1: Already voted for candidate 0fb563febbcf4233b7a125899b66ce05 in this term.
I20260924 14:32:46.110566  1560 tablet_bootstrap.cc:654] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.111397  1578 tablet_bootstrap.cc:492] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:46.111636  1565 ts_tablet_manager.cc:1428] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.057s	user 0.010s	sys 0.002s
I20260924 14:32:46.121033  1579 leader_election.cc:288] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
W20260924 14:32:46.126576  1481 leader_election.cc:341] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:46.120527  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96caf2a8ba5d4d5bafb9c6dd0ab6d310" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:46.135576  1481 leader_election.cc:302] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05; no voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:46.136049  1578 tablet_bootstrap.cc:654] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.135915  1443 raft_consensus.cc:3037] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:32:46.136895  1481 leader_election.cc:341] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Not found: Tablet not found: 96caf2a8ba5d4d5bafb9c6dd0ab6d310
W20260924 14:32:46.137912  1481 leader_election.cc:341] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Not found: Tablet not found: 96caf2a8ba5d4d5bafb9c6dd0ab6d310
I20260924 14:32:46.138527  1579 raft_consensus.cc:2726] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:46.131318  1560 tablet_bootstrap.cc:492] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:46.121646  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c843297774c94a31990bd30dc7bb622d" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:46.140312  1560 ts_tablet_manager.cc:1397] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.015s
W20260924 14:32:46.128768  1405 leader_election.cc:341] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Not found: Tablet not found: 96caf2a8ba5d4d5bafb9c6dd0ab6d310
W20260924 14:32:46.141371  1481 leader_election.cc:341] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:46.141315  1405 leader_election.cc:302] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6; no voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:46.141971  1481 leader_election.cc:302] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05; no voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:46.143054  1579 raft_consensus.cc:2726] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:46.132265  1565 tablet_bootstrap.cc:492] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:46.144441  1443 raft_consensus.cc:2454] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:46.145190  1574 raft_consensus.cc:2726] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:46.145927  1482 leader_election.cc:302] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:46.143239  1560 raft_consensus.cc:348] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.147011  1560 raft_consensus.cc:374] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.147316  1560 raft_consensus.cc:729] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.147431  1579 raft_consensus.cc:2781] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:46.148345  1579 raft_consensus.cc:686] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Leader. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:46.148193  1560 consensus_queue.cc:260] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.149549  1579 consensus_queue.cc:237] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.150141  1565 tablet_bootstrap.cc:654] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.150240  1578 tablet_bootstrap.cc:492] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:46.150638  1560 ts_tablet_manager.cc:1428] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260924 14:32:46.150992  1578 ts_tablet_manager.cc:1397] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.003s
I20260924 14:32:46.151805  1560 tablet_bootstrap.cc:492] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:46.154659  1578 raft_consensus.cc:348] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.155887  1578 raft_consensus.cc:374] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.156427  1578 raft_consensus.cc:729] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.157508  1578 consensus_queue.cc:260] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.161366  1560 tablet_bootstrap.cc:654] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.165810  1392 heartbeater.cc:503] Master 127.22.151.254:46621 was elected leader, sending a full tablet report...
I20260924 14:32:46.166399  1578 ts_tablet_manager.cc:1428] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260924 14:32:46.167570  1578 tablet_bootstrap.cc:492] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:46.166721  1264 catalog_manager.cc:5495] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "0fb563febbcf4233b7a125899b66ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:46.168638  1560 tablet_bootstrap.cc:492] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:46.169167  1560 ts_tablet_manager.cc:1397] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.004s
I20260924 14:32:46.171782  1560 raft_consensus.cc:348] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.172449  1560 raft_consensus.cc:374] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.172704  1560 raft_consensus.cc:729] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.173349  1560 consensus_queue.cc:260] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.174633  1578 tablet_bootstrap.cc:654] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.175501  1560 ts_tablet_manager.cc:1428] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260924 14:32:46.176563  1560 tablet_bootstrap.cc:492] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:46.178860  1565 tablet_bootstrap.cc:492] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:46.179461  1565 ts_tablet_manager.cc:1397] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.008s
I20260924 14:32:46.182199  1560 tablet_bootstrap.cc:654] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.182500  1565 raft_consensus.cc:348] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.183307  1565 raft_consensus.cc:374] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.183629  1565 raft_consensus.cc:729] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.184492  1565 consensus_queue.cc:260] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.192631  1565 ts_tablet_manager.cc:1428] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260924 14:32:46.194208  1565 tablet_bootstrap.cc:492] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:46.203145  1565 tablet_bootstrap.cc:654] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.211412  1578 tablet_bootstrap.cc:492] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:46.212143  1578 ts_tablet_manager.cc:1397] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.045s	user 0.021s	sys 0.022s
I20260924 14:32:46.215078  1578 raft_consensus.cc:348] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.215945  1578 raft_consensus.cc:374] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.216298  1578 raft_consensus.cc:729] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.217347  1578 consensus_queue.cc:260] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.219973  1578 ts_tablet_manager.cc:1428] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260924 14:32:46.220917  1578 tablet_bootstrap.cc:492] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:46.221098  1565 tablet_bootstrap.cc:492] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:46.221740  1565 ts_tablet_manager.cc:1397] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.011s
I20260924 14:32:46.223265  1560 tablet_bootstrap.cc:492] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:46.223933  1560 ts_tablet_manager.cc:1397] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.048s	user 0.025s	sys 0.016s
I20260924 14:32:46.224678  1565 raft_consensus.cc:348] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.225589  1565 raft_consensus.cc:374] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.225878  1565 raft_consensus.cc:729] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.226814  1578 tablet_bootstrap.cc:654] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.226624  1565 consensus_queue.cc:260] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.226824  1560 raft_consensus.cc:348] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.228130  1560 raft_consensus.cc:374] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.228507  1560 raft_consensus.cc:729] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.229301  1565 ts_tablet_manager.cc:1428] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260924 14:32:46.229708  1560 consensus_queue.cc:260] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.232391  1560 ts_tablet_manager.cc:1428] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260924 14:32:46.230464  1565 tablet_bootstrap.cc:492] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:46.239634  1579 raft_consensus.cc:482] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:46.240302  1579 raft_consensus.cc:504] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.241997  1578 tablet_bootstrap.cc:492] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:46.242419  1579 leader_election.cc:288] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153)
I20260924 14:32:46.242581  1578 ts_tablet_manager.cc:1397] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.001s
I20260924 14:32:46.244448  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4d0e3cd411a44ba8e4f9fd59b6e0fb5" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:46.245178  1443 raft_consensus.cc:2454] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:46.245325  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4d0e3cd411a44ba8e4f9fd59b6e0fb5" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:46.246071  1367 raft_consensus.cc:2454] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:46.245787  1578 raft_consensus.cc:348] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.246647  1578 raft_consensus.cc:374] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.246953  1578 raft_consensus.cc:729] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.246989  1482 leader_election.cc:302] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:46.247884  1578 consensus_queue.cc:260] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.246241  1565 tablet_bootstrap.cc:654] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.250263  1578 ts_tablet_manager.cc:1428] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260924 14:32:46.251387  1578 tablet_bootstrap.cc:492] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:46.260003  1578 tablet_bootstrap.cc:654] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.263607  1562 raft_consensus.cc:2781] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:46.264078  1562 raft_consensus.cc:482] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:46.264431  1562 raft_consensus.cc:3037] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.268026  1579 raft_consensus.cc:482] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:46.268565  1579 raft_consensus.cc:504] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.271919  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4216c32baf34f488c831ce881873cfa" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:46.272879  1443 raft_consensus.cc:2454] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:46.274153  1562 raft_consensus.cc:504] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.274477  1482 leader_election.cc:302] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:46.274950  1579 leader_election.cc:288] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:46.272065  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4216c32baf34f488c831ce881873cfa" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:46.276141  1579 raft_consensus.cc:2781] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:46.276713  1579 raft_consensus.cc:482] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260924 14:32:46.276919  1481 leader_election.cc:341] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:46.276996  1579 raft_consensus.cc:3037] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.277669  1562 leader_election.cc:288] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153)
I20260924 14:32:46.278110  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4d0e3cd411a44ba8e4f9fd59b6e0fb5" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:46.278738  1443 raft_consensus.cc:3037] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.278947  1366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4d0e3cd411a44ba8e4f9fd59b6e0fb5" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:46.279592  1366 raft_consensus.cc:3037] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.280414  1578 tablet_bootstrap.cc:492] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:46.280920  1578 ts_tablet_manager.cc:1397] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.004s
I20260924 14:32:46.283733  1578 raft_consensus.cc:348] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.284515  1578 raft_consensus.cc:374] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.284350  1579 raft_consensus.cc:504] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.284940  1578 raft_consensus.cc:729] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.285710  1578 consensus_queue.cc:260] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.287539  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4216c32baf34f488c831ce881873cfa" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:46.287830  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4216c32baf34f488c831ce881873cfa" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:46.288201  1367 raft_consensus.cc:3037] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.288475  1578 ts_tablet_manager.cc:1428] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260924 14:32:46.288534  1441 raft_consensus.cc:3037] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.290709  1578 tablet_bootstrap.cc:492] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:46.292061  1366 raft_consensus.cc:2454] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:46.293273  1481 leader_election.cc:302] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:46.294260  1562 raft_consensus.cc:2781] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:46.286480  1443 raft_consensus.cc:2454] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:46.297669  1578 tablet_bootstrap.cc:654] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.299752  1565 tablet_bootstrap.cc:492] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:46.299665  1441 raft_consensus.cc:2454] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:46.300375  1565 ts_tablet_manager.cc:1397] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.070s	user 0.024s	sys 0.011s
I20260924 14:32:46.301088  1482 leader_election.cc:302] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:46.305020  1565 raft_consensus.cc:348] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.305994  1565 raft_consensus.cc:374] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.306300  1565 raft_consensus.cc:729] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.307125  1565 consensus_queue.cc:260] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.310002  1565 ts_tablet_manager.cc:1428] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260924 14:32:46.310469  1579 leader_election.cc:288] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:46.312088  1596 raft_consensus.cc:2781] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:46.313192  1596 raft_consensus.cc:686] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Leader. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:46.313714  1367 raft_consensus.cc:2454] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:46.312642  1565 tablet_bootstrap.cc:492] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:46.315539  1562 raft_consensus.cc:686] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Leader. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:46.316716  1596 consensus_queue.cc:237] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.316846  1562 consensus_queue.cc:237] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.322741  1565 tablet_bootstrap.cc:654] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.322881  1585 raft_consensus.cc:482] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:46.323415  1585 raft_consensus.cc:504] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.328555  1585 leader_election.cc:288] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.334182  1578 tablet_bootstrap.cc:492] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:46.335017  1578 ts_tablet_manager.cc:1397] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.015s
I20260924 14:32:46.338043  1578 raft_consensus.cc:348] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.338900  1578 raft_consensus.cc:374] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.339224  1578 raft_consensus.cc:729] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.339952  1578 consensus_queue.cc:260] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.342095  1578 ts_tablet_manager.cc:1428] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260924 14:32:46.342943  1578 tablet_bootstrap.cc:492] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:46.345678  1565 tablet_bootstrap.cc:492] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:46.346385  1565 ts_tablet_manager.cc:1397] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.001s
I20260924 14:32:46.349009  1578 tablet_bootstrap.cc:654] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.349869  1596 raft_consensus.cc:482] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:46.349218  1264 catalog_manager.cc:5495] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "0fb563febbcf4233b7a125899b66ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:46.349905  1565 raft_consensus.cc:348] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.351207  1565 raft_consensus.cc:374] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.350482  1596 raft_consensus.cc:504] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.351578  1565 raft_consensus.cc:729] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.353259  1596 leader_election.cc:288] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:46.352702  1565 consensus_queue.cc:260] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.356323  1565 ts_tablet_manager.cc:1428] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260924 14:32:46.359401  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27b3ccc05ab7495baec6265cd9c3c35d" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:46.360107  1441 raft_consensus.cc:2454] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:46.360486  1578 tablet_bootstrap.cc:492] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:46.361040  1578 ts_tablet_manager.cc:1397] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260924 14:32:46.361624  1482 leader_election.cc:302] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:46.363062  1596 raft_consensus.cc:2781] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:46.363202  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27b3ccc05ab7495baec6265cd9c3c35d" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:46.363497  1596 raft_consensus.cc:482] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:46.363533  1578 raft_consensus.cc:348] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.364130  1367 raft_consensus.cc:2454] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:46.364185  1596 raft_consensus.cc:3037] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.364507  1578 raft_consensus.cc:374] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.364862  1578 raft_consensus.cc:729] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.365542  1578 consensus_queue.cc:260] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.367249  1578 ts_tablet_manager.cc:1428] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260924 14:32:46.368062  1578 tablet_bootstrap.cc:492] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:46.376695  1596 raft_consensus.cc:504] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.378453  1578 tablet_bootstrap.cc:654] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.378798  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5701334222ec4b839d1dba26d7b14599" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:46.379513  1441 raft_consensus.cc:2454] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:46.379935  1262 catalog_manager.cc:5495] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "0fb563febbcf4233b7a125899b66ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:46.380895  1331 leader_election.cc:302] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:46.381803  1585 raft_consensus.cc:2781] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:46.382181  1585 raft_consensus.cc:482] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:46.382519  1585 raft_consensus.cc:3037] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.385879  1596 leader_election.cc:288] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:46.389132  1578 tablet_bootstrap.cc:492] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:46.389766  1578 ts_tablet_manager.cc:1397] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.022s	user 0.002s	sys 0.012s
I20260924 14:32:46.390506  1585 raft_consensus.cc:504] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.392388  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27b3ccc05ab7495baec6265cd9c3c35d" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:46.393007  1367 raft_consensus.cc:3037] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.393399  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5701334222ec4b839d1dba26d7b14599" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:46.393177  1578 raft_consensus.cc:348] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.393862  1585 leader_election.cc:288] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.394147  1443 raft_consensus.cc:3037] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.398687  1574 raft_consensus.cc:482] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:46.399164  1574 raft_consensus.cc:504] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.399928  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27b3ccc05ab7495baec6265cd9c3c35d" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:46.400581  1441 raft_consensus.cc:3037] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.401049  1574 leader_election.cc:288] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.401479  1443 raft_consensus.cc:2454] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:46.402079  1366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c8e875b068d47f4a5a6ca1915609625" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:46.402768  1366 raft_consensus.cc:2454] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 0.
I20260924 14:32:46.394131  1578 raft_consensus.cc:374] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.402678  1331 leader_election.cc:302] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:46.403872  1578 raft_consensus.cc:729] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.402667  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c8e875b068d47f4a5a6ca1915609625" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:46.404371  1405 leader_election.cc:302] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:46.404536  1518 raft_consensus.cc:2454] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 0.
I20260924 14:32:46.404783  1578 consensus_queue.cc:260] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.405390  1574 raft_consensus.cc:2781] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:46.405920  1574 raft_consensus.cc:482] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:46.406302  1574 raft_consensus.cc:3037] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.408910  1367 raft_consensus.cc:2454] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:46.410255  1481 leader_election.cc:302] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:46.411137  1596 raft_consensus.cc:2781] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:46.411465  1441 raft_consensus.cc:2454] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:46.411700  1596 raft_consensus.cc:686] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Leader. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:46.413285  1596 consensus_queue.cc:237] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.415198  1585 raft_consensus.cc:2781] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:46.416936  1574 raft_consensus.cc:504] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.420826  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c8e875b068d47f4a5a6ca1915609625" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:46.421788  1367 raft_consensus.cc:3037] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.430007  1585 raft_consensus.cc:686] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:46.431347  1585 consensus_queue.cc:237] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.435839  1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5701334222ec4b839d1dba26d7b14599" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:46.436494  1517 raft_consensus.cc:3037] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.437927  1574 leader_election.cc:288] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.438720  1574 raft_consensus.cc:482] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:46.439302  1574 raft_consensus.cc:504] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.434999  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5701334222ec4b839d1dba26d7b14599" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:46.441033  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c8e875b068d47f4a5a6ca1915609625" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:46.441717  1516 raft_consensus.cc:3037] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.441964  1574 leader_election.cc:288] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.444478  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a559653d744c3ab37c5b1b2b1f1b17" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:46.445240  1518 raft_consensus.cc:2454] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 0.
I20260924 14:32:46.445833  1367 raft_consensus.cc:2454] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 1.
I20260924 14:32:46.447016  1408 leader_election.cc:302] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:46.448225  1405 leader_election.cc:302] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:46.448820  1578 ts_tablet_manager.cc:1428] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.059s	user 0.018s	sys 0.026s
I20260924 14:32:46.449231  1610 raft_consensus.cc:2781] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:46.449656  1610 raft_consensus.cc:482] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:46.450238  1578 tablet_bootstrap.cc:492] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:46.451555  1516 raft_consensus.cc:2454] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 1.
I20260924 14:32:46.454936  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a559653d744c3ab37c5b1b2b1f1b17" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:46.454538  1264 catalog_manager.cc:5495] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
W20260924 14:32:46.456341  1405 leader_election.cc:341] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:46.456504  1574 raft_consensus.cc:2781] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:46.457173  1578 tablet_bootstrap.cc:654] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.456336  1517 raft_consensus.cc:2454] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:46.461805  1585 raft_consensus.cc:482] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:46.461163  1262 catalog_manager.cc:5495] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "0fb563febbcf4233b7a125899b66ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:46.462338  1585 raft_consensus.cc:504] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.465349  1585 leader_election.cc:288] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.466359  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f656b71445647da9ffe7e0ed4e51ca8" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:46.467346  1441 raft_consensus.cc:2454] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:46.467607  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f656b71445647da9ffe7e0ed4e51ca8" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:46.468343  1516 raft_consensus.cc:2454] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:46.469092  1331 leader_election.cc:302] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:46.470342  1585 raft_consensus.cc:2781] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:46.470945  1585 raft_consensus.cc:482] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:46.471374  1585 raft_consensus.cc:3037] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.449970  1610 raft_consensus.cc:3037] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.483911  1574 raft_consensus.cc:686] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Becoming Leader. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:46.484995  1578 tablet_bootstrap.cc:492] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:46.484032  1610 raft_consensus.cc:504] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.485574  1578 ts_tablet_manager.cc:1397] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.003s
I20260924 14:32:46.485183  1574 consensus_queue.cc:237] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.488418  1578 raft_consensus.cc:348] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.488991  1578 raft_consensus.cc:374] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.489221  1578 raft_consensus.cc:729] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.489832  1578 consensus_queue.cc:260] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.491555  1578 ts_tablet_manager.cc:1428] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260924 14:32:46.492496  1578 tablet_bootstrap.cc:492] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:46.482525  1585 raft_consensus.cc:504] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.497798  1578 tablet_bootstrap.cc:654] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:46.498718  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f656b71445647da9ffe7e0ed4e51ca8" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:46.499619  1441 raft_consensus.cc:3037] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.508674  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a559653d744c3ab37c5b1b2b1f1b17" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:46.509307  1367 raft_consensus.cc:3037] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.513190  1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a559653d744c3ab37c5b1b2b1f1b17" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:46.513942  1517 raft_consensus.cc:3037] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.514479  1441 raft_consensus.cc:2454] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:46.516458  1367 raft_consensus.cc:2454] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 1.
I20260924 14:32:46.517676  1405 leader_election.cc:302] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:46.518743  1612 raft_consensus.cc:2781] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:46.522184  1578 tablet_bootstrap.cc:492] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:46.522689  1578 ts_tablet_manager.cc:1397] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.006s
I20260924 14:32:46.523581  1331 leader_election.cc:302] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:46.525166  1585 leader_election.cc:288] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.525885  1585 raft_consensus.cc:2781] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:46.526396  1585 raft_consensus.cc:686] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:46.526454  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f656b71445647da9ffe7e0ed4e51ca8" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:46.527395  1516 raft_consensus.cc:3037] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.527527  1585 consensus_queue.cc:237] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.530439  1578 raft_consensus.cc:348] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.531400  1578 raft_consensus.cc:374] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:46.531847  1578 raft_consensus.cc:729] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:46.532732  1578 consensus_queue.cc:260] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.535614  1578 ts_tablet_manager.cc:1428] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260924 14:32:46.545286  1610 leader_election.cc:288] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.548302  1517 raft_consensus.cc:2454] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 1.
I20260924 14:32:46.548877  1516 raft_consensus.cc:2454] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:46.553908  1262 catalog_manager.cc:5495] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194). New cstate: current_term: 1 leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:46.555029  1612 raft_consensus.cc:686] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Becoming Leader. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:46.556102  1612 consensus_queue.cc:237] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.559401  1585 raft_consensus.cc:482] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:46.559991  1585 raft_consensus.cc:504] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.562871  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96caf2a8ba5d4d5bafb9c6dd0ab6d310" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:46.562865  1585 leader_election.cc:288] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.563808  1441 raft_consensus.cc:2379] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in current term 1: Already voted for candidate 0fb563febbcf4233b7a125899b66ce05 in this term.
I20260924 14:32:46.563207  1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96caf2a8ba5d4d5bafb9c6dd0ab6d310" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:46.563231  1264 catalog_manager.cc:5495] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:46.565760  1333 leader_election.cc:302] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6
I20260924 14:32:46.567867  1585 raft_consensus.cc:3037] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.574739  1261 catalog_manager.cc:5495] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194). New cstate: current_term: 1 leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:46.578049  1585 raft_consensus.cc:2726] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:46.655663  1612 raft_consensus.cc:482] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:46.656317  1612 raft_consensus.cc:504] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.658077  1612 leader_election.cc:288] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.659190  1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c843297774c94a31990bd30dc7bb622d" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:46.659102  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c843297774c94a31990bd30dc7bb622d" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:46.660133  1517 raft_consensus.cc:2454] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 0.
I20260924 14:32:46.660161  1367 raft_consensus.cc:2454] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 0.
I20260924 14:32:46.661365  1408 leader_election.cc:302] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:46.662070  1612 raft_consensus.cc:2781] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:46.662442  1612 raft_consensus.cc:482] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:46.662708  1612 raft_consensus.cc:3037] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.668073  1612 raft_consensus.cc:504] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.669612  1612 leader_election.cc:288] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:46.670463  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c843297774c94a31990bd30dc7bb622d" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:46.670704  1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c843297774c94a31990bd30dc7bb622d" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:46.671183  1367 raft_consensus.cc:3037] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.671248  1517 raft_consensus.cc:3037] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:46.678321  1367 raft_consensus.cc:2454] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 1.
I20260924 14:32:46.678325  1517 raft_consensus.cc:2454] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 1.
I20260924 14:32:46.679759  1408 leader_election.cc:302] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:46.680439  1612 raft_consensus.cc:2781] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:46.680881  1612 raft_consensus.cc:686] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Becoming Leader. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:46.681564  1612 consensus_queue.cc:237] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:46.690584  1261 catalog_manager.cc:5495] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194). New cstate: current_term: 1 leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:46.696352  1596 consensus_queue.cc:1035] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:46.708102  1596 consensus_queue.cc:1035] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:46.820086  1585 consensus_queue.cc:1035] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:46.830943  1608 consensus_queue.cc:1035] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 14:32:46.850229  1465 debug-util.cc:398] Leaking SignalData structure 0x7b0800203c80 after lost signal to thread 23138
W20260924 14:32:46.851370  1465 debug-util.cc:398] Leaking SignalData structure 0x7b080015aba0 after lost signal to thread 1298
I20260924 14:32:46.855975  1596 consensus_queue.cc:1035] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:46.871400  1579 consensus_queue.cc:1035] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:46.879273  1579 consensus_queue.cc:1035] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:46.890282  1616 consensus_queue.cc:1035] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260924 14:32:46.897848  1616 consensus_queue.cc:1035] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:46.904567  1579 consensus_queue.cc:1035] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:46.932775  1596 consensus_queue.cc:1035] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:46.938824  1579 consensus_queue.cc:1035] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:46.970667  1517 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e4d0e3cd411a44ba8e4f9fd59b6e0fb5"
dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
mode: GRACEFUL
new_leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
 from {username='slave'} at 127.0.0.1:37318
I20260924 14:32:46.971630  1517 raft_consensus.cc:595] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Received request to transfer leadership to TS a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:46.995587  1443 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c843297774c94a31990bd30dc7bb622d"
dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
mode: GRACEFUL
new_leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
 from {username='slave'} at 127.0.0.1:55082
I20260924 14:32:46.996167  1443 raft_consensus.cc:595] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Received request to transfer leadership to TS a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:47.002667  1517 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e4216c32baf34f488c831ce881873cfa"
dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
mode: GRACEFUL
new_leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
 from {username='slave'} at 127.0.0.1:37318
I20260924 14:32:47.003401  1517 raft_consensus.cc:595] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Received request to transfer leadership to TS a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:47.023692  1366 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7f656b71445647da9ffe7e0ed4e51ca8"
dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
mode: GRACEFUL
new_leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
 from {username='slave'} at 127.0.0.1:40618
I20260924 14:32:47.024353  1366 raft_consensus.cc:595] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Received request to transfer leadership to TS 91f8d84a8caf421d99048c427fc3b5a6
I20260924 14:32:47.032691  1517 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96caf2a8ba5d4d5bafb9c6dd0ab6d310"
dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
mode: GRACEFUL
new_leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
 from {username='slave'} at 127.0.0.1:37318
I20260924 14:32:47.033401  1517 raft_consensus.cc:595] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Received request to transfer leadership to TS 91f8d84a8caf421d99048c427fc3b5a6
I20260924 14:32:47.062857  1610 consensus_queue.cc:1035] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:47.074746  1610 consensus_queue.cc:1035] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:47.133292  1579 raft_consensus.cc:979] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05: : Instructing follower 91f8d84a8caf421d99048c427fc3b5a6 to start an election
I20260924 14:32:47.133876  1562 raft_consensus.cc:1067] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Signalling peer 91f8d84a8caf421d99048c427fc3b5a6 to start an election
I20260924 14:32:47.135743  1443 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "96caf2a8ba5d4d5bafb9c6dd0ab6d310"
dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
 from {username='slave'} at 127.0.0.1:55068
I20260924 14:32:47.136397  1443 raft_consensus.cc:482] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:47.136817  1443 raft_consensus.cc:3037] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.145570  1443 raft_consensus.cc:504] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.147917  1443 leader_election.cc:288] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:47.148859  1366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96caf2a8ba5d4d5bafb9c6dd0ab6d310" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:47.149148  1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96caf2a8ba5d4d5bafb9c6dd0ab6d310" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:47.149581  1366 raft_consensus.cc:3037] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.149885  1517 raft_consensus.cc:3032] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:47.150297  1517 raft_consensus.cc:729] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:47.151207  1517 consensus_queue.cc:260] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.152719  1517 raft_consensus.cc:3037] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.155433  1366 raft_consensus.cc:2454] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 2.
I20260924 14:32:47.157047  1405 leader_election.cc:302] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:47.157786  1610 raft_consensus.cc:2781] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:47.158231  1610 raft_consensus.cc:686] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 LEADER]: Becoming Leader. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:47.159126  1608 consensus_queue.cc:1035] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:47.159101  1610 consensus_queue.cc:237] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.161165  1517 raft_consensus.cc:2454] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 2.
I20260924 14:32:47.168056  1262 catalog_manager.cc:5495] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 reported cstate change: term changed from 1 to 2, leader changed from 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195) to 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194). New cstate: current_term: 2 leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:47.170472  1608 consensus_queue.cc:1035] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:47.173415  1607 raft_consensus.cc:979] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2: : Instructing follower 91f8d84a8caf421d99048c427fc3b5a6 to start an election
I20260924 14:32:47.173995  1607 raft_consensus.cc:1067] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Signalling peer 91f8d84a8caf421d99048c427fc3b5a6 to start an election
I20260924 14:32:47.176726  1443 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7f656b71445647da9ffe7e0ed4e51ca8"
dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
 from {username='slave'} at 127.0.0.1:55072
I20260924 14:32:47.177315  1443 raft_consensus.cc:482] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:47.177666  1443 raft_consensus.cc:3037] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.189040  1443 raft_consensus.cc:504] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.190074  1441 raft_consensus.cc:1226] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Rejecting Update request from peer a622d8c7c7304ddc8ab146e1a8e3f5f2 for earlier term 1. Current term is 2. Ops: []
I20260924 14:32:47.195720  1366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f656b71445647da9ffe7e0ed4e51ca8" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:47.196405  1366 raft_consensus.cc:3032] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:47.196707  1366 raft_consensus.cc:729] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:47.197515  1366 consensus_queue.cc:260] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.199167  1443 leader_election.cc:288] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:47.200331  1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f656b71445647da9ffe7e0ed4e51ca8" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:47.201022  1517 raft_consensus.cc:3037] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
W20260924 14:32:47.201583  1585 consensus_queue.cc:1159] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91f8d84a8caf421d99048c427fc3b5a6" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 0 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a622d8c7c7304ddc8ab146e1a8e3f5f2 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260924 14:32:47.199056  1366 raft_consensus.cc:3037] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.212153  1366 raft_consensus.cc:2454] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 2.
I20260924 14:32:47.213325  1405 leader_election.cc:302] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:47.213255  1610 consensus_queue.cc:1035] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:47.214067  1517 raft_consensus.cc:2454] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 2.
I20260924 14:32:47.214458  1616 raft_consensus.cc:2781] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:47.215008  1616 raft_consensus.cc:686] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 LEADER]: Becoming Leader. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:47.215909  1616 consensus_queue.cc:237] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.225262  1262 catalog_manager.cc:5495] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 reported cstate change: term changed from 1 to 2, leader changed from a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193) to 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194). New cstate: current_term: 2 leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:47.229722  1616 consensus_queue.cc:1035] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:47.243855  1612 raft_consensus.cc:979] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6: : Instructing follower a622d8c7c7304ddc8ab146e1a8e3f5f2 to start an election
I20260924 14:32:47.244385  1612 raft_consensus.cc:1067] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Signalling peer a622d8c7c7304ddc8ab146e1a8e3f5f2 to start an election
I20260924 14:32:47.246600  1366 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c843297774c94a31990bd30dc7bb622d"
dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
 from {username='slave'} at 127.0.0.1:40612
I20260924 14:32:47.247247  1366 raft_consensus.cc:482] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:47.247648  1366 raft_consensus.cc:3037] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.255753  1366 raft_consensus.cc:504] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.258316  1366 leader_election.cc:288] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 2 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:47.258889  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c843297774c94a31990bd30dc7bb622d" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:47.259675  1443 raft_consensus.cc:3032] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:47.258919  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c843297774c94a31990bd30dc7bb622d" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:47.260037  1443 raft_consensus.cc:729] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:47.260618  1516 raft_consensus.cc:3037] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.261142  1443 consensus_queue.cc:260] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.262598  1443 raft_consensus.cc:3037] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.267608  1516 raft_consensus.cc:2454] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 2.
I20260924 14:32:47.268654  1333 leader_election.cc:302] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:47.269371  1585 raft_consensus.cc:2781] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:47.269873  1585 raft_consensus.cc:686] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:47.270336  1443 raft_consensus.cc:2454] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 2.
I20260924 14:32:47.270753  1585 consensus_queue.cc:237] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.279218  1263 catalog_manager.cc:5495] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 1 to 2, leader changed from 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194) to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:47.363907  1579 raft_consensus.cc:979] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05: : Instructing follower a622d8c7c7304ddc8ab146e1a8e3f5f2 to start an election
I20260924 14:32:47.364392  1562 raft_consensus.cc:1067] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Signalling peer a622d8c7c7304ddc8ab146e1a8e3f5f2 to start an election
I20260924 14:32:47.365953  1366 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e4d0e3cd411a44ba8e4f9fd59b6e0fb5"
dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
 from {username='slave'} at 127.0.0.1:40600
I20260924 14:32:47.366467  1366 raft_consensus.cc:482] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:47.366770  1366 raft_consensus.cc:3037] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.372304  1366 raft_consensus.cc:504] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.374078  1366 leader_election.cc:288] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 2 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:47.375044  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4d0e3cd411a44ba8e4f9fd59b6e0fb5" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:47.375294  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4d0e3cd411a44ba8e4f9fd59b6e0fb5" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:47.375895  1443 raft_consensus.cc:3037] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.375962  1516 raft_consensus.cc:3032] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:47.376422  1516 raft_consensus.cc:729] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:47.377251  1516 consensus_queue.cc:260] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.378646  1516 raft_consensus.cc:3037] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.383284  1443 raft_consensus.cc:2454] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 2.
I20260924 14:32:47.384743  1331 leader_election.cc:302] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:47.385699  1585 raft_consensus.cc:2781] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:47.385896  1516 raft_consensus.cc:2454] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 2.
I20260924 14:32:47.386209  1585 raft_consensus.cc:686] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:47.387024  1585 consensus_queue.cc:237] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.394829  1263 catalog_manager.cc:5495] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 1 to 2, leader changed from 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195) to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:47.432114  1579 raft_consensus.cc:979] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05: : Instructing follower a622d8c7c7304ddc8ab146e1a8e3f5f2 to start an election
I20260924 14:32:47.432615  1562 raft_consensus.cc:1067] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Signalling peer a622d8c7c7304ddc8ab146e1a8e3f5f2 to start an election
I20260924 14:32:47.434190  1365 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e4216c32baf34f488c831ce881873cfa"
dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
 from {username='slave'} at 127.0.0.1:40600
I20260924 14:32:47.434721  1365 raft_consensus.cc:482] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:47.435037  1365 raft_consensus.cc:3037] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.440721  1365 raft_consensus.cc:504] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.444645  1365 leader_election.cc:288] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 2 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:47.443563  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4216c32baf34f488c831ce881873cfa" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:47.446425  1443 raft_consensus.cc:3037] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.447345  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4216c32baf34f488c831ce881873cfa" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:47.448148  1516 raft_consensus.cc:3032] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:47.448530  1516 raft_consensus.cc:729] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:47.449496  1516 consensus_queue.cc:260] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.451239  1516 raft_consensus.cc:3037] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:47.454727  1443 raft_consensus.cc:2454] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 2.
I20260924 14:32:47.456127  1331 leader_election.cc:302] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:47.457100  1607 raft_consensus.cc:2781] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:47.457131  1516 raft_consensus.cc:2454] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 2.
I20260924 14:32:47.457660  1607 raft_consensus.cc:686] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:47.458562  1607 consensus_queue.cc:237] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:47.466133  1263 catalog_manager.cc:5495] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 1 to 2, leader changed from 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195) to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:47.608299  1365 raft_consensus.cc:1261] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Refusing update from remote peer 91f8d84a8caf421d99048c427fc3b5a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:47.609715  1616 consensus_queue.cc:1035] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:47.624737  1516 raft_consensus.cc:1261] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Refusing update from remote peer 91f8d84a8caf421d99048c427fc3b5a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:47.626137  1616 consensus_queue.cc:1035] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:47.649653  1365 raft_consensus.cc:1261] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Refusing update from remote peer 91f8d84a8caf421d99048c427fc3b5a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:47.651006  1616 consensus_queue.cc:1035] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:47.659144  1516 raft_consensus.cc:1261] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Refusing update from remote peer 91f8d84a8caf421d99048c427fc3b5a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:47.660651  1574 consensus_queue.cc:1035] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:47.748245  1443 raft_consensus.cc:1261] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Refusing update from remote peer a622d8c7c7304ddc8ab146e1a8e3f5f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:47.749745  1607 consensus_queue.cc:1035] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:47.761394  1516 raft_consensus.cc:1261] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Refusing update from remote peer a622d8c7c7304ddc8ab146e1a8e3f5f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:47.763688  1608 consensus_queue.cc:1035] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:47.797943  1443 raft_consensus.cc:1261] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Refusing update from remote peer a622d8c7c7304ddc8ab146e1a8e3f5f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:47.799700  1644 consensus_queue.cc:1035] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:47.810105  1516 raft_consensus.cc:1261] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Refusing update from remote peer a622d8c7c7304ddc8ab146e1a8e3f5f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:47.811589  1608 consensus_queue.cc:1035] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:47.954924  1516 raft_consensus.cc:1261] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Refusing update from remote peer a622d8c7c7304ddc8ab146e1a8e3f5f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:47.956445  1608 consensus_queue.cc:1035] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:47.967788  1443 raft_consensus.cc:1261] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Refusing update from remote peer a622d8c7c7304ddc8ab146e1a8e3f5f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:47.974400  1608 consensus_queue.cc:1035] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:50.041311 23135 test_util.cc:268] Using random seed: 2036951490
I20260924 14:32:50.100586  1261 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51544:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260924 14:32:50.103817  1261 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 14:32:50.165974  1351 tablet_service.cc:1467] Processing CreateTablet for tablet bebe60ffa3d94d20bbf6c4c843ba76a7 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:50.166800  1432 tablet_service.cc:1467] Processing CreateTablet for tablet bebe60ffa3d94d20bbf6c4c843ba76a7 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:50.167484  1351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bebe60ffa3d94d20bbf6c4c843ba76a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.168331  1432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bebe60ffa3d94d20bbf6c4c843ba76a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.168191  1504 tablet_service.cc:1467] Processing CreateTablet for tablet bebe60ffa3d94d20bbf6c4c843ba76a7 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260924 14:32:50.169607  1504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bebe60ffa3d94d20bbf6c4c843ba76a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.182399  1352 tablet_service.cc:1467] Processing CreateTablet for tablet c9d3660f3bdf43fbb2b66c128da37339 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:50.184000  1352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9d3660f3bdf43fbb2b66c128da37339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.184980  1506 tablet_service.cc:1467] Processing CreateTablet for tablet c9d3660f3bdf43fbb2b66c128da37339 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:50.186489  1506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9d3660f3bdf43fbb2b66c128da37339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.189060  1430 tablet_service.cc:1467] Processing CreateTablet for tablet c9d3660f3bdf43fbb2b66c128da37339 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260924 14:32:50.190532  1430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9d3660f3bdf43fbb2b66c128da37339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.205668  1353 tablet_service.cc:1467] Processing CreateTablet for tablet eab0e342bab24c059a1ecf79cdc277b4 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:50.207232  1353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eab0e342bab24c059a1ecf79cdc277b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.209304  1424 tablet_service.cc:1467] Processing CreateTablet for tablet eab0e342bab24c059a1ecf79cdc277b4 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:50.210774  1424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eab0e342bab24c059a1ecf79cdc277b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.211112  1501 tablet_service.cc:1467] Processing CreateTablet for tablet eab0e342bab24c059a1ecf79cdc277b4 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260924 14:32:50.212560  1501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eab0e342bab24c059a1ecf79cdc277b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.216684  1661 tablet_bootstrap.cc:492] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:50.218708  1354 tablet_service.cc:1467] Processing CreateTablet for tablet 3577172d729540129939ac66bbad3c4f (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:50.220829  1354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3577172d729540129939ac66bbad3c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.237491  1500 tablet_service.cc:1467] Processing CreateTablet for tablet 3577172d729540129939ac66bbad3c4f (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:50.237295  1432 tablet_service.cc:1467] Processing CreateTablet for tablet 3577172d729540129939ac66bbad3c4f (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260924 14:32:50.239084  1500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3577172d729540129939ac66bbad3c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.239534  1432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3577172d729540129939ac66bbad3c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.247529  1433 tablet_service.cc:1467] Processing CreateTablet for tablet 006c6555ed6b44099b1fce0d5c205a33 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:50.248891  1433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 006c6555ed6b44099b1fce0d5c205a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.250773  1352 tablet_service.cc:1467] Processing CreateTablet for tablet 006c6555ed6b44099b1fce0d5c205a33 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:50.252281  1352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 006c6555ed6b44099b1fce0d5c205a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.258358  1661 tablet_bootstrap.cc:654] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.268838  1351 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0b4d3e5e7145a9b1d2ccc5e62347aa (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:50.269260  1502 tablet_service.cc:1467] Processing CreateTablet for tablet 006c6555ed6b44099b1fce0d5c205a33 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260924 14:32:50.272267  1351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e0b4d3e5e7145a9b1d2ccc5e62347aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.272408  1502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 006c6555ed6b44099b1fce0d5c205a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.272735  1428 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0b4d3e5e7145a9b1d2ccc5e62347aa (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:50.274086  1428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e0b4d3e5e7145a9b1d2ccc5e62347aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.286621  1350 tablet_service.cc:1467] Processing CreateTablet for tablet 260345b02b31461d9c29791d5e54d98f (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:50.288115  1350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260345b02b31461d9c29791d5e54d98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.287942  1427 tablet_service.cc:1467] Processing CreateTablet for tablet 260345b02b31461d9c29791d5e54d98f (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:50.289290  1427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260345b02b31461d9c29791d5e54d98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.290268  1356 tablet_service.cc:1467] Processing CreateTablet for tablet 10709e8d44eb49f1ac2288975ee83633 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:50.291621  1356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10709e8d44eb49f1ac2288975ee83633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.294365  1503 tablet_service.cc:1467] Processing CreateTablet for tablet 260345b02b31461d9c29791d5e54d98f (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260924 14:32:50.295749  1503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260345b02b31461d9c29791d5e54d98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.298166  1431 tablet_service.cc:1467] Processing CreateTablet for tablet 10709e8d44eb49f1ac2288975ee83633 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:50.300475  1355 tablet_service.cc:1467] Processing CreateTablet for tablet 5acf2170d0f8454e9c0344b6e5a446a4 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:50.300966  1431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10709e8d44eb49f1ac2288975ee83633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.301781  1355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5acf2170d0f8454e9c0344b6e5a446a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.301764  1507 tablet_service.cc:1467] Processing CreateTablet for tablet 10709e8d44eb49f1ac2288975ee83633 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260924 14:32:50.303122  1507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10709e8d44eb49f1ac2288975ee83633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.325688  1424 tablet_service.cc:1467] Processing CreateTablet for tablet 5acf2170d0f8454e9c0344b6e5a446a4 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:50.329159  1508 tablet_service.cc:1467] Processing CreateTablet for tablet 5acf2170d0f8454e9c0344b6e5a446a4 (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260924 14:32:50.330561  1508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5acf2170d0f8454e9c0344b6e5a446a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.288757  1506 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0b4d3e5e7145a9b1d2ccc5e62347aa (DEFAULT_TABLE table=soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260924 14:32:50.329231  1424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5acf2170d0f8454e9c0344b6e5a446a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.362946  1663 tablet_bootstrap.cc:492] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:50.364750  1506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e0b4d3e5e7145a9b1d2ccc5e62347aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 14:32:50.384104  1662 tablet_bootstrap.cc:492] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:50.389441  1663 tablet_bootstrap.cc:654] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.407873  1661 tablet_bootstrap.cc:492] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:50.408579  1661 ts_tablet_manager.cc:1397] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.192s	user 0.017s	sys 0.016s
I20260924 14:32:50.407868  1663 tablet_bootstrap.cc:492] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:50.410648  1663 ts_tablet_manager.cc:1397] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.009s
I20260924 14:32:50.411546  1661 raft_consensus.cc:348] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.412493  1661 raft_consensus.cc:374] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.412843  1661 raft_consensus.cc:729] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.413729  1661 consensus_queue.cc:260] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.414978  1662 tablet_bootstrap.cc:654] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.416477  1661 ts_tablet_manager.cc:1428] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260924 14:32:50.417652  1661 tablet_bootstrap.cc:492] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:50.413731  1663 raft_consensus.cc:348] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.419436  1663 raft_consensus.cc:374] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.419816  1663 raft_consensus.cc:729] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.420639  1663 consensus_queue.cc:260] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.423137  1663 ts_tablet_manager.cc:1428] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260924 14:32:50.424911  1663 tablet_bootstrap.cc:492] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:50.427955  1661 tablet_bootstrap.cc:654] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.438235  1663 tablet_bootstrap.cc:654] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.440243  1661 tablet_bootstrap.cc:492] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:50.441325  1662 tablet_bootstrap.cc:492] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:50.443547  1662 ts_tablet_manager.cc:1397] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.060s	user 0.012s	sys 0.004s
I20260924 14:32:50.443626  1661 ts_tablet_manager.cc:1397] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.001s
I20260924 14:32:50.446751  1662 raft_consensus.cc:348] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.447008  1661 raft_consensus.cc:348] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.447602  1662 raft_consensus.cc:374] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.447779  1661 raft_consensus.cc:374] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.447950  1662 raft_consensus.cc:729] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.448101  1661 raft_consensus.cc:729] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.448762  1662 consensus_queue.cc:260] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.448905  1661 consensus_queue.cc:260] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.451184  1662 ts_tablet_manager.cc:1428] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260924 14:32:50.451507  1661 ts_tablet_manager.cc:1428] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:32:50.452531  1661 tablet_bootstrap.cc:492] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:50.452531  1662 tablet_bootstrap.cc:492] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:50.459196  1661 tablet_bootstrap.cc:654] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.461030  1662 tablet_bootstrap.cc:654] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.470590  1663 tablet_bootstrap.cc:492] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:50.471258  1663 ts_tablet_manager.cc:1397] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.002s
I20260924 14:32:50.471710  1661 tablet_bootstrap.cc:492] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:50.472383  1661 ts_tablet_manager.cc:1397] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260924 14:32:50.474351  1663 raft_consensus.cc:348] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.475206  1663 raft_consensus.cc:374] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.475656  1663 raft_consensus.cc:729] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.475507  1661 raft_consensus.cc:348] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.476513  1661 raft_consensus.cc:374] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.476886  1661 raft_consensus.cc:729] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.476646  1663 consensus_queue.cc:260] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.477672  1661 consensus_queue.cc:260] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.479257  1663 ts_tablet_manager.cc:1428] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260924 14:32:50.480087  1661 ts_tablet_manager.cc:1428] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260924 14:32:50.481165  1661 tablet_bootstrap.cc:492] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:50.483139  1662 tablet_bootstrap.cc:492] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:50.483808  1662 ts_tablet_manager.cc:1397] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.000s
I20260924 14:32:50.487097  1662 raft_consensus.cc:348] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.488085  1662 raft_consensus.cc:374] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.488199  1661 tablet_bootstrap.cc:654] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.488616  1663 tablet_bootstrap.cc:492] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:50.500010  1663 tablet_bootstrap.cc:654] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.488931  1662 raft_consensus.cc:729] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.501718  1662 consensus_queue.cc:260] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.513701  1662 ts_tablet_manager.cc:1428] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260924 14:32:50.515285  1662 tablet_bootstrap.cc:492] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:50.522225  1661 tablet_bootstrap.cc:492] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:50.522552  1663 tablet_bootstrap.cc:492] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:50.522854  1661 ts_tablet_manager.cc:1397] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.024s
I20260924 14:32:50.523217  1663 ts_tablet_manager.cc:1397] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.000s
I20260924 14:32:50.523594  1662 tablet_bootstrap.cc:654] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.526046  1661 raft_consensus.cc:348] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.526575  1663 raft_consensus.cc:348] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.527503  1663 raft_consensus.cc:374] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.527872  1663 raft_consensus.cc:729] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
W20260924 14:32:50.529054  1408 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:33825, user_credentials={real_user=slave}} blocked reactor thread for 115493us
I20260924 14:32:50.528795  1663 consensus_queue.cc:260] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.531572  1663 ts_tablet_manager.cc:1428] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:32:50.526976  1661 raft_consensus.cc:374] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.531951  1661 raft_consensus.cc:729] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.532768  1661 consensus_queue.cc:260] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.535181  1661 ts_tablet_manager.cc:1428] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260924 14:32:50.536197  1663 tablet_bootstrap.cc:492] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:50.543879  1661 tablet_bootstrap.cc:492] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:50.544107  1663 tablet_bootstrap.cc:654] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.545977  1662 tablet_bootstrap.cc:492] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:50.546545  1662 ts_tablet_manager.cc:1397] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.008s
I20260924 14:32:50.549731  1662 raft_consensus.cc:348] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.550546  1662 raft_consensus.cc:374] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.550840  1662 raft_consensus.cc:729] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.551616  1662 consensus_queue.cc:260] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.559816  1661 tablet_bootstrap.cc:654] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.564046  1662 ts_tablet_manager.cc:1428] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260924 14:32:50.565981  1662 tablet_bootstrap.cc:492] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:50.573077  1662 tablet_bootstrap.cc:654] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.583914  1661 tablet_bootstrap.cc:492] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:50.584633  1661 ts_tablet_manager.cc:1397] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.000s
I20260924 14:32:50.587800  1661 raft_consensus.cc:348] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.589506  1661 raft_consensus.cc:374] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.590440  1661 raft_consensus.cc:729] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.594955  1661 consensus_queue.cc:260] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.602679  1661 ts_tablet_manager.cc:1428] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260924 14:32:50.604919  1596 raft_consensus.cc:482] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:50.605499  1596 raft_consensus.cc:504] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.606909  1662 tablet_bootstrap.cc:492] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:50.607511  1662 ts_tablet_manager.cc:1397] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.007s
I20260924 14:32:50.609364  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bebe60ffa3d94d20bbf6c4c843ba76a7" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:50.610383  1365 raft_consensus.cc:2454] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:50.610477  1662 raft_consensus.cc:348] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.604878  1661 tablet_bootstrap.cc:492] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:50.611351  1662 raft_consensus.cc:374] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.615649  1481 leader_election.cc:302] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:50.615211  1662 raft_consensus.cc:729] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.619853  1608 raft_consensus.cc:482] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:50.619616  1662 consensus_queue.cc:260] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.620517  1608 raft_consensus.cc:504] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.622658  1662 ts_tablet_manager.cc:1428] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260924 14:32:50.607630  1596 leader_election.cc:288] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:50.623811  1662 tablet_bootstrap.cc:492] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:50.623915  1596 raft_consensus.cc:2781] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:50.619163  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bebe60ffa3d94d20bbf6c4c843ba76a7" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:50.624359  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab0e342bab24c059a1ecf79cdc277b4" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:50.625131  1441 raft_consensus.cc:2454] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:50.624317  1596 raft_consensus.cc:482] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:50.628788  1331 leader_election.cc:302] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:50.629280  1596 raft_consensus.cc:3037] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.625136  1443 raft_consensus.cc:2454] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:50.635636  1678 raft_consensus.cc:2781] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:50.636135  1678 raft_consensus.cc:482] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:50.636461  1663 tablet_bootstrap.cc:492] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:50.637158  1663 ts_tablet_manager.cc:1397] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.101s	user 0.030s	sys 0.066s
I20260924 14:32:50.637815  1608 leader_election.cc:288] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:50.636545  1678 raft_consensus.cc:3037] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.640530  1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab0e342bab24c059a1ecf79cdc277b4" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:50.640354  1663 raft_consensus.cc:348] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.641428  1663 raft_consensus.cc:374] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.641423  1517 raft_consensus.cc:2454] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:50.641855  1663 raft_consensus.cc:729] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.642761  1663 consensus_queue.cc:260] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.645577  1663 ts_tablet_manager.cc:1428] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260924 14:32:50.654930  1663 tablet_bootstrap.cc:492] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
W20260924 14:32:50.660794  1481 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.151.193:43153, user_credentials={real_user=slave}} blocked reactor thread for 45250.3us
I20260924 14:32:50.662535  1678 raft_consensus.cc:504] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.664222  1596 raft_consensus.cc:504] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.671228  1680 raft_consensus.cc:482] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:50.671917  1680 raft_consensus.cc:504] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.673103  1661 tablet_bootstrap.cc:654] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.674181  1680 leader_election.cc:288] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:50.675568  1662 tablet_bootstrap.cc:654] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.675346  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d3660f3bdf43fbb2b66c128da37339" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:50.675843  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d3660f3bdf43fbb2b66c128da37339" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:50.676120  1443 raft_consensus.cc:2454] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:50.676757  1365 raft_consensus.cc:2454] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:50.677573  1482 leader_election.cc:302] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:50.681188  1663 tablet_bootstrap.cc:654] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.681712  1680 raft_consensus.cc:2781] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:50.682199  1680 raft_consensus.cc:482] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:50.682641  1680 raft_consensus.cc:3037] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.684908  1596 leader_election.cc:288] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:50.686084  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bebe60ffa3d94d20bbf6c4c843ba76a7" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:50.686839  1443 raft_consensus.cc:3037] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.695118  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bebe60ffa3d94d20bbf6c4c843ba76a7" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:50.696139  1365 raft_consensus.cc:3037] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.696770  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab0e342bab24c059a1ecf79cdc277b4" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:50.697543  1441 raft_consensus.cc:3037] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.706350  1441 raft_consensus.cc:2454] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:50.709518  1443 raft_consensus.cc:2454] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:50.710690  1662 tablet_bootstrap.cc:492] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:50.723304  1662 ts_tablet_manager.cc:1397] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.100s	user 0.014s	sys 0.004s
I20260924 14:32:50.723528  1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab0e342bab24c059a1ecf79cdc277b4" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:50.723979  1331 leader_election.cc:302] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:50.715184  1661 tablet_bootstrap.cc:492] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:50.725162  1681 raft_consensus.cc:2781] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:50.725708  1681 raft_consensus.cc:686] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:50.712112  1365 raft_consensus.cc:2454] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:50.722398  1678 leader_election.cc:288] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:50.726608  1681 consensus_queue.cc:237] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.726336  1680 raft_consensus.cc:504] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.724192  1517 raft_consensus.cc:3037] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.728394  1481 leader_election.cc:302] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:50.729234  1596 raft_consensus.cc:2781] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:50.729926  1680 leader_election.cc:288] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:50.730613  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d3660f3bdf43fbb2b66c128da37339" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:50.726444  1662 raft_consensus.cc:348] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.732079  1662 raft_consensus.cc:374] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.732350  1662 raft_consensus.cc:729] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.733098  1662 consensus_queue.cc:260] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.717331  1663 tablet_bootstrap.cc:492] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:50.731293  1365 raft_consensus.cc:3037] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.735811  1663 ts_tablet_manager.cc:1397] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.081s	user 0.007s	sys 0.011s
I20260924 14:32:50.725703  1661 ts_tablet_manager.cc:1397] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.121s	user 0.036s	sys 0.016s
I20260924 14:32:50.742439  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d3660f3bdf43fbb2b66c128da37339" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:50.743168  1441 raft_consensus.cc:3037] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.744218  1662 ts_tablet_manager.cc:1428] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260924 14:32:50.738823  1663 raft_consensus.cc:348] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.745064  1663 raft_consensus.cc:374] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.745384  1663 raft_consensus.cc:729] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.746266  1662 tablet_bootstrap.cc:492] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:50.746384  1663 consensus_queue.cc:260] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.749382  1517 raft_consensus.cc:2454] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:50.749807  1596 raft_consensus.cc:686] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Leader. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:50.750772  1596 consensus_queue.cc:237] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.751523  1663 ts_tablet_manager.cc:1428] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260924 14:32:50.752665  1663 tablet_bootstrap.cc:492] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:50.753268  1365 raft_consensus.cc:2454] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:50.748333  1661 raft_consensus.cc:348] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.757323  1661 raft_consensus.cc:374] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.757637  1661 raft_consensus.cc:729] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.756832  1662 tablet_bootstrap.cc:654] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.758471  1481 leader_election.cc:302] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:50.758366  1661 consensus_queue.cc:260] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.759871  1680 raft_consensus.cc:2781] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:50.760370  1680 raft_consensus.cc:686] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Leader. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:50.761184  1680 consensus_queue.cc:237] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.766208  1661 ts_tablet_manager.cc:1428] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260924 14:32:50.767383  1661 tablet_bootstrap.cc:492] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:50.770871  1441 raft_consensus.cc:2454] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:50.775660  1263 catalog_manager.cc:5495] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "0fb563febbcf4233b7a125899b66ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:50.782691  1663 tablet_bootstrap.cc:654] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.791004  1662 tablet_bootstrap.cc:492] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:50.791669  1662 ts_tablet_manager.cc:1397] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.015s
I20260924 14:32:50.795045  1662 raft_consensus.cc:348] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.796063  1662 raft_consensus.cc:374] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.796368  1662 raft_consensus.cc:729] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.797124  1662 consensus_queue.cc:260] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.799578  1663 tablet_bootstrap.cc:492] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:50.801680  1662 ts_tablet_manager.cc:1428] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260924 14:32:50.803649  1663 ts_tablet_manager.cc:1397] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.008s
I20260924 14:32:50.805330  1264 catalog_manager.cc:5495] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:50.809083  1673 raft_consensus.cc:482] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:50.809715  1673 raft_consensus.cc:504] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.813601  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0b4d3e5e7145a9b1d2ccc5e62347aa" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:50.814512  1673 leader_election.cc:288] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:50.814420  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0b4d3e5e7145a9b1d2ccc5e62347aa" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
W20260924 14:32:50.814958  1408 leader_election.cc:341] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:50.815301  1365 raft_consensus.cc:2454] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 0.
I20260924 14:32:50.807554  1663 raft_consensus.cc:348] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.816078  1663 raft_consensus.cc:374] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.816397  1663 raft_consensus.cc:729] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.816854  1405 leader_election.cc:302] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 0fb563febbcf4233b7a125899b66ce05
I20260924 14:32:50.807992  1662 tablet_bootstrap.cc:492] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:50.818238  1648 raft_consensus.cc:2781] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:50.818615  1648 raft_consensus.cc:482] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:50.818957  1648 raft_consensus.cc:3037] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.817082  1663 consensus_queue.cc:260] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.826445  1663 ts_tablet_manager.cc:1428] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260924 14:32:50.827570  1663 tablet_bootstrap.cc:492] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:50.828112  1662 tablet_bootstrap.cc:654] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.832079  1648 raft_consensus.cc:504] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.834872  1663 tablet_bootstrap.cc:654] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.794209  1661 tablet_bootstrap.cc:654] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.853215  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0b4d3e5e7145a9b1d2ccc5e62347aa" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:50.853906  1365 raft_consensus.cc:3037] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.857053  1263 catalog_manager.cc:5495] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "0fb563febbcf4233b7a125899b66ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:50.861675  1365 raft_consensus.cc:2454] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 1.
I20260924 14:32:50.863041  1405 leader_election.cc:302] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:50.864066  1673 raft_consensus.cc:2781] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:50.864751  1673 raft_consensus.cc:686] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Becoming Leader. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:50.864372  1648 leader_election.cc:288] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:50.865972  1673 consensus_queue.cc:237] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.869366  1661 tablet_bootstrap.cc:492] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:50.872133  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0b4d3e5e7145a9b1d2ccc5e62347aa" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
W20260924 14:32:50.874076  1408 leader_election.cc:341] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:50.877894  1661 ts_tablet_manager.cc:1397] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.111s	user 0.007s	sys 0.005s
I20260924 14:32:50.880296  1680 raft_consensus.cc:482] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:50.881031  1680 raft_consensus.cc:504] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.884917  1680 leader_election.cc:288] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:50.884840  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10709e8d44eb49f1ac2288975ee83633" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:50.887055  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10709e8d44eb49f1ac2288975ee83633" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:50.887019  1661 raft_consensus.cc:348] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.888388  1661 raft_consensus.cc:374] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260924 14:32:50.888011  1482 leader_election.cc:341] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:50.888041  1365 raft_consensus.cc:2454] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:50.888861  1661 raft_consensus.cc:729] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.889870  1481 leader_election.cc:302] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 91f8d84a8caf421d99048c427fc3b5a6
I20260924 14:32:50.890851  1662 tablet_bootstrap.cc:492] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:50.891165  1680 raft_consensus.cc:2781] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:50.891392  1662 ts_tablet_manager.cc:1397] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.024s
I20260924 14:32:50.891702  1680 raft_consensus.cc:482] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:50.890971  1661 consensus_queue.cc:260] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.892060  1680 raft_consensus.cc:3037] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.894248  1661 ts_tablet_manager.cc:1428] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260924 14:32:50.898602  1662 raft_consensus.cc:348] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.899374  1662 raft_consensus.cc:374] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.899680  1662 raft_consensus.cc:729] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.905053  1680 raft_consensus.cc:504] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.909130  1663 tablet_bootstrap.cc:492] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:50.906423  1264 catalog_manager.cc:5495] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194). New cstate: current_term: 1 leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:50.912988  1663 ts_tablet_manager.cc:1397] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.086s	user 0.032s	sys 0.022s
I20260924 14:32:50.913445  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10709e8d44eb49f1ac2288975ee83633" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:50.914106  1365 raft_consensus.cc:3037] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.917338  1663 raft_consensus.cc:348] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.918290  1663 raft_consensus.cc:374] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:50.918591  1663 raft_consensus.cc:729] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:50.919710  1608 raft_consensus.cc:482] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:50.919416  1663 consensus_queue.cc:260] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.920231  1608 raft_consensus.cc:504] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.921950  1663 ts_tablet_manager.cc:1428] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260924 14:32:50.922359  1608 leader_election.cc:288] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:50.923101  1663 tablet_bootstrap.cc:492] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:50.924291  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260345b02b31461d9c29791d5e54d98f" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:50.924988  1516 raft_consensus.cc:2454] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:50.926964  1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260345b02b31461d9c29791d5e54d98f" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:50.927736  1443 raft_consensus.cc:2454] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:50.929113  1333 leader_election.cc:302] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:50.929662  1663 tablet_bootstrap.cc:654] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.930867  1608 raft_consensus.cc:2781] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:50.931298  1608 raft_consensus.cc:482] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:50.931649  1608 raft_consensus.cc:3037] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.909915  1662 consensus_queue.cc:260] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.895573  1661 tablet_bootstrap.cc:492] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
I20260924 14:32:50.910954  1680 leader_election.cc:288] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:50.911867  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10709e8d44eb49f1ac2288975ee83633" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:50.945461  1441 raft_consensus.cc:3037] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.954720  1661 tablet_bootstrap.cc:654] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:50.962510  1365 raft_consensus.cc:2454] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:50.965874  1481 leader_election.cc:302] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:50.965641  1608 raft_consensus.cc:504] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.966843  1680 raft_consensus.cc:2781] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:50.967331  1680 raft_consensus.cc:686] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Leader. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:50.968152  1680 consensus_queue.cc:237] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:50.969337  1442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260345b02b31461d9c29791d5e54d98f" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:50.969995  1442 raft_consensus.cc:3037] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.971055  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260345b02b31461d9c29791d5e54d98f" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:50.971717  1516 raft_consensus.cc:3037] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:50.982153  1442 raft_consensus.cc:2454] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:50.978884  1441 raft_consensus.cc:2454] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 1.
I20260924 14:32:51.033130  1331 leader_election.cc:302] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:50.986148  1663 tablet_bootstrap.cc:492] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:50.990278  1661 tablet_bootstrap.cc:492] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:51.036031  1663 ts_tablet_manager.cc:1397] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.113s	user 0.010s	sys 0.030s
I20260924 14:32:51.036301  1661 ts_tablet_manager.cc:1397] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.141s	user 0.019s	sys 0.009s
I20260924 14:32:50.981772  1516 raft_consensus.cc:2454] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:51.036386  1596 raft_consensus.cc:482] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:51.042487  1681 raft_consensus.cc:2781] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:51.040133  1596 raft_consensus.cc:504] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.042762  1673 raft_consensus.cc:482] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:51.043919  1685 raft_consensus.cc:482] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:51.047032  1681 raft_consensus.cc:686] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:51.047721  1685 raft_consensus.cc:504] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.048568  1596 leader_election.cc:288] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:51.049074  1681 consensus_queue.cc:237] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.050807  1685 leader_election.cc:288] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:50.968075  1608 leader_election.cc:288] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:51.051111  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3577172d729540129939ac66bbad3c4f" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:51.051795  1673 raft_consensus.cc:504] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.054435  1673 leader_election.cc:288] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:51.055600  1366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
W20260924 14:32:51.058912  1481 leader_election.cc:341] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:51.060299  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
W20260924 14:32:51.062508  1405 leader_election.cc:341] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:51.069020  1442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:51.069586  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3577172d729540129939ac66bbad3c4f" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:51.069876  1442 raft_consensus.cc:2454] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 0.
I20260924 14:32:51.070366  1663 raft_consensus.cc:348] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.072250  1663 raft_consensus.cc:374] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:51.071821  1661 raft_consensus.cc:348] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.072618  1663 raft_consensus.cc:729] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:51.072783  1661 raft_consensus.cc:374] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:51.071967  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:51.073635  1516 raft_consensus.cc:2454] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 0.
I20260924 14:32:51.073529  1663 consensus_queue.cc:260] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.073764  1482 leader_election.cc:302] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:51.073119  1661 raft_consensus.cc:729] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:51.075779  1685 raft_consensus.cc:2781] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:51.076339  1685 raft_consensus.cc:482] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:51.077144  1685 raft_consensus.cc:3037] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:51.076224  1661 consensus_queue.cc:260] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.079282  1661 ts_tablet_manager.cc:1428] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.042s	user 0.010s	sys 0.001s
I20260924 14:32:51.083236  1661 tablet_bootstrap.cc:492] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Bootstrap starting.
W20260924 14:32:51.084712  1482 leader_election.cc:341] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:51.085330  1482 leader_election.cc:302] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05; no voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:51.086398  1680 raft_consensus.cc:2726] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:51.088140  1263 catalog_manager.cc:5495] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195). New cstate: current_term: 1 leader_uuid: "0fb563febbcf4233b7a125899b66ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:51.089339  1262 catalog_manager.cc:5495] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:51.091177  1661 tablet_bootstrap.cc:654] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Neither blocks nor log segments found. Creating new log.
W20260924 14:32:51.080179  1481 leader_election.cc:341] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:51.098464  1663 ts_tablet_manager.cc:1428] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.061s	user 0.004s	sys 0.008s
I20260924 14:32:51.100914  1663 tablet_bootstrap.cc:492] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05: Bootstrap starting.
I20260924 14:32:51.110100  1663 tablet_bootstrap.cc:654] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:51.113900  1408 leader_election.cc:302] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:51.115123  1673 raft_consensus.cc:2781] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:51.115623  1673 raft_consensus.cc:482] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:51.116022  1673 raft_consensus.cc:3037] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
W20260924 14:32:51.120667  1313 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:32:51.131493  1662 ts_tablet_manager.cc:1428] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.233s	user 0.004s	sys 0.012s
I20260924 14:32:51.133388  1662 tablet_bootstrap.cc:492] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:51.145102  1673 raft_consensus.cc:504] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.148854  1673 leader_election.cc:288] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:51.148916  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
W20260924 14:32:51.151809  1405 leader_election.cc:341] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:51.152168  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:51.152136  1685 raft_consensus.cc:504] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.154302  1516 raft_consensus.cc:2379] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91f8d84a8caf421d99048c427fc3b5a6 in current term 1: Already voted for candidate 0fb563febbcf4233b7a125899b66ce05 in this term.
I20260924 14:32:51.155805  1408 leader_election.cc:302] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6; no voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:51.156255  1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:51.157984  1648 raft_consensus.cc:2726] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260924 14:32:51.161201  1481 leader_election.cc:341] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260924 14:32:51.166143  1663 tablet_bootstrap.cc:492] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05: No bootstrap required, opened a new log
I20260924 14:32:51.166719  1663 ts_tablet_manager.cc:1397] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05: Time spent bootstrapping tablet: real 0.066s	user 0.013s	sys 0.019s
I20260924 14:32:51.167505  1685 leader_election.cc:288] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:51.168854  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:51.169731  1441 raft_consensus.cc:2379] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fb563febbcf4233b7a125899b66ce05 in current term 1: Already voted for candidate 91f8d84a8caf421d99048c427fc3b5a6 in this term.
I20260924 14:32:51.169605  1663 raft_consensus.cc:348] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.170274  1663 raft_consensus.cc:374] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:51.170538  1663 raft_consensus.cc:729] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Initialized, Role: FOLLOWER
I20260924 14:32:51.171116  1482 leader_election.cc:302] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05; no voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:51.171248  1663 consensus_queue.cc:260] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.173171  1680 raft_consensus.cc:2726] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 14:32:51.173359  1661 tablet_bootstrap.cc:492] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2: No bootstrap required, opened a new log
I20260924 14:32:51.173974  1661 ts_tablet_manager.cc:1397] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.011s
I20260924 14:32:51.175417  1663 ts_tablet_manager.cc:1428] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260924 14:32:51.177649  1661 raft_consensus.cc:348] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.183544  1661 raft_consensus.cc:374] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:51.183820  1661 raft_consensus.cc:729] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Initialized, Role: FOLLOWER
I20260924 14:32:51.184443  1662 tablet_bootstrap.cc:654] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:51.184504  1661 consensus_queue.cc:260] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.185766  1681 consensus_queue.cc:1035] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:51.190927  1661 ts_tablet_manager.cc:1428] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.005s
I20260924 14:32:51.190672  1680 consensus_queue.cc:1035] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:51.226752  1681 raft_consensus.cc:482] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:51.227809  1681 raft_consensus.cc:504] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.235874  1596 consensus_queue.cc:1035] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:51.238195  1685 consensus_queue.cc:1035] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:51.250450  1681 leader_election.cc:288] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:51.252960  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5acf2170d0f8454e9c0344b6e5a446a4" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:51.257149  1516 raft_consensus.cc:2454] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:51.258059  1662 tablet_bootstrap.cc:492] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:51.258803  1662 ts_tablet_manager.cc:1397] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.126s	user 0.008s	sys 0.012s
I20260924 14:32:51.259397  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5acf2170d0f8454e9c0344b6e5a446a4" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:51.259589  1333 leader_election.cc:302] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:51.251413  1681 consensus_queue.cc:1035] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260924 14:32:51.267277  1608 raft_consensus.cc:2781] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:51.267983  1608 raft_consensus.cc:482] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:51.268448  1608 raft_consensus.cc:3037] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:51.265038  1662 raft_consensus.cc:348] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.270274  1662 raft_consensus.cc:374] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:51.270583  1662 raft_consensus.cc:729] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:51.272529  1662 consensus_queue.cc:260] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.274580  1662 ts_tablet_manager.cc:1428] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260924 14:32:51.276172  1662 tablet_bootstrap.cc:492] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6: Bootstrap starting.
I20260924 14:32:51.276173  1441 raft_consensus.cc:2454] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:51.308777  1608 raft_consensus.cc:504] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.320075  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5acf2170d0f8454e9c0344b6e5a446a4" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:51.320768  1441 raft_consensus.cc:3037] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:51.321550  1662 tablet_bootstrap.cc:654] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6: Neither blocks nor log segments found. Creating new log.
I20260924 14:32:51.342661  1685 consensus_queue.cc:1035] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:51.345223  1517 raft_consensus.cc:3037] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:51.369083  1662 tablet_bootstrap.cc:492] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6: No bootstrap required, opened a new log
I20260924 14:32:51.369686  1662 ts_tablet_manager.cc:1397] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6: Time spent bootstrapping tablet: real 0.094s	user 0.018s	sys 0.022s
I20260924 14:32:51.376629  1662 raft_consensus.cc:348] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.377487  1662 raft_consensus.cc:374] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 14:32:51.377774  1662 raft_consensus.cc:729] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Initialized, Role: FOLLOWER
I20260924 14:32:51.378618  1662 consensus_queue.cc:260] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.384546  1441 raft_consensus.cc:2454] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:51.386041  1331 leader_election.cc:302] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:51.387213  1678 raft_consensus.cc:2781] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:51.387817  1678 raft_consensus.cc:686] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:51.388895  1678 consensus_queue.cc:237] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.389672  1662 ts_tablet_manager.cc:1428] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260924 14:32:51.398931  1648 consensus_queue.cc:1035] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:51.406752  1608 leader_election.cc:288] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:51.408038  1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5acf2170d0f8454e9c0344b6e5a446a4" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:51.408900  1516 raft_consensus.cc:3037] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:51.414577  1648 consensus_queue.cc:1035] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:51.420265  1516 raft_consensus.cc:2454] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:51.438439  1264 catalog_manager.cc:5495] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:51.445389  1685 consensus_queue.cc:1035] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:51.468272  1685 consensus_queue.cc:1035] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:51.498586  1678 raft_consensus.cc:482] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:51.499112  1678 raft_consensus.cc:504] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.501283  1678 leader_election.cc:288] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:51.502202  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3577172d729540129939ac66bbad3c4f" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:51.502478  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3577172d729540129939ac66bbad3c4f" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:51.503082  1441 raft_consensus.cc:2454] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:51.503201  1518 raft_consensus.cc:2454] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 0.
I20260924 14:32:51.504317  1331 leader_election.cc:302] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:51.505368  1678 raft_consensus.cc:2781] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 14:32:51.505759  1678 raft_consensus.cc:482] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:51.506039  1678 raft_consensus.cc:3037] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:51.513550  1678 raft_consensus.cc:504] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.515295  1678 leader_election.cc:288] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:51.516378  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3577172d729540129939ac66bbad3c4f" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:51.516600  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3577172d729540129939ac66bbad3c4f" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:51.517027  1441 raft_consensus.cc:3037] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:51.517335  1518 raft_consensus.cc:3037] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:51.523725  1441 raft_consensus.cc:2454] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:51.524945  1331 leader_election.cc:302] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:51.525686  1678 raft_consensus.cc:2781] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 14:32:51.526127  1678 raft_consensus.cc:686] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:51.527417  1518 raft_consensus.cc:2454] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 1.
I20260924 14:32:51.527010  1678 consensus_queue.cc:237] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.536896  1262 catalog_manager.cc:5495] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 1 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:51.552616  1678 consensus_queue.cc:1035] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:51.567036  1717 consensus_queue.cc:1035] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260924 14:32:51.583714  1716 raft_consensus.cc:482] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:51.584239  1716 raft_consensus.cc:504] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:51.587356  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6" is_pre_election: true
I20260924 14:32:51.587599  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:51.587790  1716 leader_election.cc:288] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:51.588476  1441 raft_consensus.cc:2379] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in current term 1: Already voted for candidate 91f8d84a8caf421d99048c427fc3b5a6 in this term.
I20260924 14:32:51.588548  1518 raft_consensus.cc:2379] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in current term 1: Already voted for candidate 0fb563febbcf4233b7a125899b66ce05 in this term.
I20260924 14:32:51.589874  1331 leader_election.cc:302] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6
I20260924 14:32:51.590580  1716 raft_consensus.cc:3037] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 14:32:51.597316  1716 raft_consensus.cc:2726] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 14:32:51.789508  1717 consensus_queue.cc:1035] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:51.802284  1717 consensus_queue.cc:1035] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:52.019273  1717 consensus_queue.cc:1035] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:52.037021  1717 consensus_queue.cc:1035] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260924 14:32:52.131824  1313 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260924 14:32:52.803623  1734 raft_consensus.cc:482] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 14:32:52.804296  1734 raft_consensus.cc:504] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:52.807233  1734 leader_election.cc:288] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:52.807518  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05" is_pre_election: true
I20260924 14:32:52.808197  1518 raft_consensus.cc:2454] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 1.
I20260924 14:32:52.807364  1366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" is_pre_election: true
I20260924 14:32:52.808985  1366 raft_consensus.cc:2454] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 1.
I20260924 14:32:52.809391  1408 leader_election.cc:302] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:52.810238  1648 raft_consensus.cc:2781] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 14:32:52.810587  1648 raft_consensus.cc:482] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 14:32:52.810868  1648 raft_consensus.cc:3037] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:52.817885  1648 raft_consensus.cc:504] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:52.820533  1648 leader_election.cc:288] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:52.820739  1366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:52.821465  1366 raft_consensus.cc:3037] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:52.824913  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006c6555ed6b44099b1fce0d5c205a33" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:52.825474  1518 raft_consensus.cc:3037] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:52.830200  1366 raft_consensus.cc:2454] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 2.
I20260924 14:32:52.831761  1405 leader_election.cc:302] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:52.832629  1648 raft_consensus.cc:2781] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:52.833249  1518 raft_consensus.cc:2454] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 2.
I20260924 14:32:52.835071  1648 raft_consensus.cc:686] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 LEADER]: Becoming Leader. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:52.835897  1648 consensus_queue.cc:237] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:52.845113  1264 catalog_manager.cc:5495] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 reported cstate change: term changed from 0 to 2, leader changed from <none> to 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194). New cstate: current_term: 2 leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:52.908018  1518 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bebe60ffa3d94d20bbf6c4c843ba76a7"
dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
mode: GRACEFUL
new_leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
 from {username='slave'} at 127.0.0.1:37318
I20260924 14:32:52.908761  1518 raft_consensus.cc:595] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Received request to transfer leadership to TS a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:52.914384  1518 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c9d3660f3bdf43fbb2b66c128da37339"
dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
mode: GRACEFUL
new_leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
 from {username='slave'} at 127.0.0.1:37318
I20260924 14:32:52.914908  1518 raft_consensus.cc:595] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Received request to transfer leadership to TS a622d8c7c7304ddc8ab146e1a8e3f5f2
I20260924 14:32:52.930665  1366 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "260345b02b31461d9c29791d5e54d98f"
dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
mode: GRACEFUL
new_leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
 from {username='slave'} at 127.0.0.1:40618
I20260924 14:32:52.931241  1366 raft_consensus.cc:595] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Received request to transfer leadership to TS 91f8d84a8caf421d99048c427fc3b5a6
I20260924 14:32:52.936375  1518 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "10709e8d44eb49f1ac2288975ee83633"
dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
mode: GRACEFUL
new_leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
 from {username='slave'} at 127.0.0.1:37318
I20260924 14:32:52.936944  1518 raft_consensus.cc:595] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Received request to transfer leadership to TS 91f8d84a8caf421d99048c427fc3b5a6
I20260924 14:32:52.942662  1366 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5acf2170d0f8454e9c0344b6e5a446a4"
dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
mode: GRACEFUL
new_leader_uuid: "0fb563febbcf4233b7a125899b66ce05"
 from {username='slave'} at 127.0.0.1:40618
I20260924 14:32:52.943354  1366 raft_consensus.cc:595] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Received request to transfer leadership to TS 0fb563febbcf4233b7a125899b66ce05
I20260924 14:32:52.986182  1741 raft_consensus.cc:979] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05: : Instructing follower a622d8c7c7304ddc8ab146e1a8e3f5f2 to start an election
I20260924 14:32:52.986716  1713 raft_consensus.cc:1067] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Signalling peer a622d8c7c7304ddc8ab146e1a8e3f5f2 to start an election
I20260924 14:32:52.988579  1366 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bebe60ffa3d94d20bbf6c4c843ba76a7"
dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
 from {username='slave'} at 127.0.0.1:40600
I20260924 14:32:52.989148  1366 raft_consensus.cc:482] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:52.989833  1366 raft_consensus.cc:3037] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:52.996160  1366 raft_consensus.cc:504] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:52.997964  1366 leader_election.cc:288] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 2 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:52.998929  1442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bebe60ffa3d94d20bbf6c4c843ba76a7" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:52.999219  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bebe60ffa3d94d20bbf6c4c843ba76a7" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:52.999831  1442 raft_consensus.cc:3037] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.000032  1518 raft_consensus.cc:3032] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:53.000681  1518 raft_consensus.cc:729] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:53.002223  1518 consensus_queue.cc:260] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.004179  1518 raft_consensus.cc:3037] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.008195  1442 raft_consensus.cc:2454] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 2.
I20260924 14:32:53.010052  1331 leader_election.cc:302] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:53.010887  1678 raft_consensus.cc:2781] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:53.011368  1678 raft_consensus.cc:686] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:53.012357  1678 consensus_queue.cc:237] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.013424  1518 raft_consensus.cc:2454] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 2.
I20260924 14:32:53.024113  1264 catalog_manager.cc:5495] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 1 to 2, leader changed from 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195) to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:53.191934  1741 raft_consensus.cc:979] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05: : Instructing follower 91f8d84a8caf421d99048c427fc3b5a6 to start an election
I20260924 14:32:53.192416  1713 raft_consensus.cc:1067] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Signalling peer 91f8d84a8caf421d99048c427fc3b5a6 to start an election
I20260924 14:32:53.194027  1442 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "10709e8d44eb49f1ac2288975ee83633"
dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
 from {username='slave'} at 127.0.0.1:55068
I20260924 14:32:53.194707  1442 raft_consensus.cc:482] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:53.195076  1442 raft_consensus.cc:3037] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.201629  1442 raft_consensus.cc:504] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.203264  1442 leader_election.cc:288] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:53.204747  1366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10709e8d44eb49f1ac2288975ee83633" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:53.204869  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10709e8d44eb49f1ac2288975ee83633" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:53.205675  1366 raft_consensus.cc:3037] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.205704  1518 raft_consensus.cc:3032] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:53.206430  1518 raft_consensus.cc:729] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:53.207369  1518 consensus_queue.cc:260] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.209066  1518 raft_consensus.cc:3037] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.214154  1366 raft_consensus.cc:2454] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 2.
I20260924 14:32:53.215629  1405 leader_election.cc:302] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f8d84a8caf421d99048c427fc3b5a6, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:53.216228  1518 raft_consensus.cc:2454] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 2.
I20260924 14:32:53.216522  1648 raft_consensus.cc:2781] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:53.217128  1648 raft_consensus.cc:686] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 LEADER]: Becoming Leader. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:53.218097  1648 consensus_queue.cc:237] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.226329  1264 catalog_manager.cc:5495] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 reported cstate change: term changed from 1 to 2, leader changed from 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195) to 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194). New cstate: current_term: 2 leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:53.280107  1518 raft_consensus.cc:1261] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Refusing update from remote peer 91f8d84a8caf421d99048c427fc3b5a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260924 14:32:53.281795  1648 consensus_queue.cc:1035] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:53.293639  1366 raft_consensus.cc:1261] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Refusing update from remote peer 91f8d84a8caf421d99048c427fc3b5a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260924 14:32:53.296046  1648 consensus_queue.cc:1035] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 14:32:53.330523  1741 raft_consensus.cc:979] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05: : Instructing follower a622d8c7c7304ddc8ab146e1a8e3f5f2 to start an election
I20260924 14:32:53.331073  1713 raft_consensus.cc:1067] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Signalling peer a622d8c7c7304ddc8ab146e1a8e3f5f2 to start an election
I20260924 14:32:53.332590  1366 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c9d3660f3bdf43fbb2b66c128da37339"
dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
 from {username='slave'} at 127.0.0.1:40600
I20260924 14:32:53.333112  1366 raft_consensus.cc:482] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:53.333442  1366 raft_consensus.cc:3037] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.338989  1366 raft_consensus.cc:504] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.340848  1366 leader_election.cc:288] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 2 election: Requested vote from peers 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:53.341703  1442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d3660f3bdf43fbb2b66c128da37339" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:53.341958  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d3660f3bdf43fbb2b66c128da37339" candidate_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:53.342388  1442 raft_consensus.cc:3037] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.342679  1518 raft_consensus.cc:3032] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:53.343088  1518 raft_consensus.cc:729] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:53.344785  1518 consensus_queue.cc:260] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.346894  1518 raft_consensus.cc:3037] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.346961  1717 raft_consensus.cc:979] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2: : Instructing follower 91f8d84a8caf421d99048c427fc3b5a6 to start an election
I20260924 14:32:53.347543  1678 raft_consensus.cc:1067] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Signalling peer 91f8d84a8caf421d99048c427fc3b5a6 to start an election
I20260924 14:32:53.349234  1441 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "260345b02b31461d9c29791d5e54d98f"
dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
 from {username='slave'} at 127.0.0.1:55072
I20260924 14:32:53.349809  1441 raft_consensus.cc:482] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:53.350165  1441 raft_consensus.cc:3037] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.354136  1518 raft_consensus.cc:2454] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 2.
I20260924 14:32:53.355332  1333 leader_election.cc:302] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, a622d8c7c7304ddc8ab146e1a8e3f5f2; no voters: 
I20260924 14:32:53.356264  1717 raft_consensus.cc:2781] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:53.356709  1717 raft_consensus.cc:686] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 LEADER]: Becoming Leader. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:53.356940  1442 raft_consensus.cc:2454] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622d8c7c7304ddc8ab146e1a8e3f5f2 in term 2.
I20260924 14:32:53.357673  1717 consensus_queue.cc:237] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.357738  1441 raft_consensus.cc:504] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.360900  1366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260345b02b31461d9c29791d5e54d98f" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:53.361340  1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260345b02b31461d9c29791d5e54d98f" candidate_uuid: "91f8d84a8caf421d99048c427fc3b5a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
I20260924 14:32:53.361654  1366 raft_consensus.cc:3032] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:53.362084  1518 raft_consensus.cc:3037] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.362246  1366 raft_consensus.cc:729] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:53.363360  1366 consensus_queue.cc:260] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.364274  1366 raft_consensus.cc:3037] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.367614  1441 leader_election.cc:288] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825)
I20260924 14:32:53.367624  1264 catalog_manager.cc:5495] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 reported cstate change: term changed from 1 to 2, leader changed from 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195) to a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193). New cstate: current_term: 2 leader_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:53.369302  1518 raft_consensus.cc:2454] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 2.
I20260924 14:32:53.370416  1408 leader_election.cc:302] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:53.371321  1748 raft_consensus.cc:2781] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:53.371881  1748 raft_consensus.cc:686] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 LEADER]: Becoming Leader. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:53.372686  1748 consensus_queue.cc:237] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.374715  1366 raft_consensus.cc:2454] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f8d84a8caf421d99048c427fc3b5a6 in term 2.
I20260924 14:32:53.383603  1264 catalog_manager.cc:5495] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 reported cstate change: term changed from 1 to 2, leader changed from a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193) to 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194). New cstate: current_term: 2 leader_uuid: "91f8d84a8caf421d99048c427fc3b5a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: UNKNOWN } } }
I20260924 14:32:53.388967  1678 raft_consensus.cc:979] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: : Instructing follower 0fb563febbcf4233b7a125899b66ce05 to start an election
I20260924 14:32:53.389496  1678 raft_consensus.cc:1067] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Signalling peer 0fb563febbcf4233b7a125899b66ce05 to start an election
I20260924 14:32:53.391398  1518 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5acf2170d0f8454e9c0344b6e5a446a4"
dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
 from {username='slave'} at 127.0.0.1:37310
I20260924 14:32:53.392067  1518 raft_consensus.cc:482] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:53.392391  1518 raft_consensus.cc:3037] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.398468  1518 raft_consensus.cc:504] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.400378  1518 leader_election.cc:288] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 2 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:53.401183  1366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5acf2170d0f8454e9c0344b6e5a446a4" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:53.401501  1441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5acf2170d0f8454e9c0344b6e5a446a4" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:53.401895  1366 raft_consensus.cc:3032] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:53.402244  1366 raft_consensus.cc:729] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:53.402367  1441 raft_consensus.cc:3037] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.402854  1366 consensus_queue.cc:260] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.404036  1366 raft_consensus.cc:3037] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:53.408016  1441 raft_consensus.cc:2454] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 2.
I20260924 14:32:53.409536  1482 leader_election.cc:302] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:53.409567  1366 raft_consensus.cc:2454] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 2.
I20260924 14:32:53.410854  1713 raft_consensus.cc:2781] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:53.411351  1713 raft_consensus.cc:686] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 2 LEADER]: Becoming Leader. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:53.412163  1713 consensus_queue.cc:237] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:53.422427  1264 catalog_manager.cc:5495] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 reported cstate change: term changed from 1 to 2, leader changed from a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193) to 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "0fb563febbcf4233b7a125899b66ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:53.465837  1518 raft_consensus.cc:1261] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Refusing update from remote peer a622d8c7c7304ddc8ab146e1a8e3f5f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:53.467331  1678 consensus_queue.cc:1035] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:53.476783  1442 raft_consensus.cc:1261] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Refusing update from remote peer a622d8c7c7304ddc8ab146e1a8e3f5f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:53.478133  1742 consensus_queue.cc:1035] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:53.674499  1517 raft_consensus.cc:1261] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Refusing update from remote peer 91f8d84a8caf421d99048c427fc3b5a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:53.676044  1748 consensus_queue.cc:1035] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:53.687166  1365 raft_consensus.cc:1261] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Refusing update from remote peer 91f8d84a8caf421d99048c427fc3b5a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:53.689532  1748 consensus_queue.cc:1035] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:53.802062  1442 raft_consensus.cc:1261] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Refusing update from remote peer a622d8c7c7304ddc8ab146e1a8e3f5f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:53.803548  1742 consensus_queue.cc:1035] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:53.812517  1517 raft_consensus.cc:1261] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Refusing update from remote peer a622d8c7c7304ddc8ab146e1a8e3f5f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:53.813845  1742 consensus_queue.cc:1035] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:53.849436  1365 raft_consensus.cc:1261] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Refusing update from remote peer 0fb563febbcf4233b7a125899b66ce05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:53.851051  1713 consensus_queue.cc:1035] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:53.861346  1442 raft_consensus.cc:1261] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Refusing update from remote peer 0fb563febbcf4233b7a125899b66ce05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:53.863235  1741 consensus_queue.cc:1035] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:53.882632  1365 raft_consensus.cc:1261] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Refusing update from remote peer 91f8d84a8caf421d99048c427fc3b5a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:53.884038  1748 consensus_queue.cc:1035] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:53.893054  1517 raft_consensus.cc:1261] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Refusing update from remote peer 91f8d84a8caf421d99048c427fc3b5a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:53.895140  1648 consensus_queue.cc:1035] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260924 14:32:55.946684  1261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51544:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260924 14:32:55.966837  1261 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51544:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260924 14:32:56.004184 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260924 14:32:56.211338  1797 log.cc:924] Time spent T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260924 14:32:56.255729  1796 log.cc:924] Time spent T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260924 14:32:56.311666  1798 log.cc:924] Time spent T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Append to log took a long time: real 0.102s	user 0.002s	sys 0.000s
I20260924 14:32:56.335708  1823 tablet.cc:1722] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260924 14:32:56.376842  1408 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:33825, user_credentials={real_user=slave}} blocked reactor thread for 96812.5us
W20260924 14:32:56.384912  1804 log.cc:924] Time spent T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6: Append to log took a long time: real 0.131s	user 0.002s	sys 0.000s
W20260924 14:32:56.395881  1331 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.194:38407, user_credentials={real_user=slave}} blocked reactor thread for 111596us
I20260924 14:32:56.437611  1830 tablet.cc:1722] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.448518  1837 tablet.cc:1722] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260924 14:32:56.499675  1408 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.195:33825, user_credentials={real_user=slave}} blocked reactor thread for 49771.3us
W20260924 14:32:56.517928  1833 log.cc:924] Time spent T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260924 14:32:56.517540  1843 tablet.cc:1722] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260924 14:32:56.519692  1831 log.cc:924] Time spent T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260924 14:32:56.520890  1827 log.cc:924] Time spent T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260924 14:32:56.435304  1820 tablet.cc:1722] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260924 14:32:56.502529  1405 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.151.193:43153, user_credentials={real_user=slave}} blocked reactor thread for 64602.5us
I20260924 14:32:56.432389  1825 tablet.cc:1722] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.509359  1835 tablet.cc:1722] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.546586  1834 tablet.cc:1722] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.444188  1828 tablet.cc:1722] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.549861  1442 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2c8e875b068d47f4a5a6ca1915609625"
dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
mode: GRACEFUL
new_leader_uuid: "0fb563febbcf4233b7a125899b66ce05"
 from {username='slave'} at 127.0.0.1:36134
I20260924 14:32:56.550611  1442 raft_consensus.cc:595] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Received request to transfer leadership to TS 0fb563febbcf4233b7a125899b66ce05
I20260924 14:32:56.557351  1847 tablet.cc:1722] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.555655  1846 tablet.cc:1722] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.519032  1839 tablet.cc:1722] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.590848  1261 catalog_manager.cc:5587] TS 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195:33825) does not have the latest schema for tablet 5acf2170d0f8454e9c0344b6e5a446a4 (table soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]). Expected version 1 got 0
I20260924 14:32:56.591857  1830 tablet.cc:1722] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.599900  1823 tablet.cc:1722] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.610749  1264 catalog_manager.cc:5587] TS a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153) does not have the latest schema for tablet 3577172d729540129939ac66bbad3c4f (table soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]). Expected version 1 got 0
I20260924 14:32:56.616639  1262 catalog_manager.cc:5587] TS 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407) does not have the latest schema for tablet 9e0b4d3e5e7145a9b1d2ccc5e62347aa (table soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f]). Expected version 1 got 0
I20260924 14:32:56.623292  1843 tablet.cc:1722] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.630586  1837 tablet.cc:1722] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.633143  1839 tablet.cc:1722] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.640803  1846 tablet.cc:1722] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.646757  1820 tablet.cc:1722] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.635233  1835 tablet.cc:1722] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.646804  1847 tablet.cc:1722] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.655287  1825 tablet.cc:1722] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.657754  1834 tablet.cc:1722] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.641255  1828 tablet.cc:1722] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.688295  1823 tablet.cc:1722] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.695933  1830 tablet.cc:1722] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.702836  1834 tablet.cc:1722] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260924 14:32:56.725162  1367 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5701334222ec4b839d1dba26d7b14599"
dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
mode: GRACEFUL
new_leader_uuid: "0fb563febbcf4233b7a125899b66ce05"
 from {username='slave'} at 127.0.0.1:50754
I20260924 14:32:56.725888  1367 raft_consensus.cc:595] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Received request to transfer leadership to TS 0fb563febbcf4233b7a125899b66ce05
I20260924 14:32:56.727933 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260924 14:32:56.728826 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:56.735656  1262 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe: soft_deleted_table [id=58f381eb0545436a8c9ee7691705db8f] alter complete (version 1)
I20260924 14:32:56.746043  1829 raft_consensus.cc:979] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6: : Instructing follower 0fb563febbcf4233b7a125899b66ce05 to start an election
I20260924 14:32:56.746598  1829 raft_consensus.cc:1067] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Signalling peer 0fb563febbcf4233b7a125899b66ce05 to start an election
I20260924 14:32:56.748554  1513 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2c8e875b068d47f4a5a6ca1915609625"
dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
 from {username='slave'} at 127.0.0.1:37306
I20260924 14:32:56.749120  1513 raft_consensus.cc:482] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:56.749420  1513 raft_consensus.cc:3037] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:56.756771  1513 raft_consensus.cc:504] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:56.758618  1513 leader_election.cc:288] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 2 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:56.759853  1440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c8e875b068d47f4a5a6ca1915609625" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:56.760558  1440 raft_consensus.cc:3032] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:56.760860  1440 raft_consensus.cc:729] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f8d84a8caf421d99048c427fc3b5a6, State: Running, Role: LEADER
I20260924 14:32:56.761756  1440 consensus_queue.cc:260] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:56.762847  1440 raft_consensus.cc:3037] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:56.764219  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c8e875b068d47f4a5a6ca1915609625" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:56.764806  1367 raft_consensus.cc:3037] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:56.769171  1440 raft_consensus.cc:2454] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 2.
I20260924 14:32:56.770463  1482 leader_election.cc:302] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:56.771265  1842 raft_consensus.cc:2781] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:56.771598  1367 raft_consensus.cc:2454] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 2.
I20260924 14:32:56.771811  1842 raft_consensus.cc:686] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 2 LEADER]: Becoming Leader. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:56.772799  1842 consensus_queue.cc:237] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:56.780997  1262 catalog_manager.cc:5495] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 reported cstate change: term changed from 1 to 2, leader changed from 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194) to 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "0fb563febbcf4233b7a125899b66ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:57.244212  1849 raft_consensus.cc:979] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2: : Instructing follower 0fb563febbcf4233b7a125899b66ce05 to start an election
I20260924 14:32:57.244709  1803 raft_consensus.cc:1067] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Signalling peer 0fb563febbcf4233b7a125899b66ce05 to start an election
I20260924 14:32:57.246158  1513 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5701334222ec4b839d1dba26d7b14599"
dest_uuid: "0fb563febbcf4233b7a125899b66ce05"
 from {username='slave'} at 127.0.0.1:37310
I20260924 14:32:57.246603  1513 raft_consensus.cc:482] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 14:32:57.246872  1513 raft_consensus.cc:3037] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:57.251920  1513 raft_consensus.cc:504] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:57.253386  1513 leader_election.cc:288] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 2 election: Requested vote from peers a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153), 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407)
I20260924 14:32:57.254352  1442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5701334222ec4b839d1dba26d7b14599" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91f8d84a8caf421d99048c427fc3b5a6"
I20260924 14:32:57.255050  1442 raft_consensus.cc:3037] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:57.254925  1367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5701334222ec4b839d1dba26d7b14599" candidate_uuid: "0fb563febbcf4233b7a125899b66ce05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2"
I20260924 14:32:57.255729  1367 raft_consensus.cc:3032] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260924 14:32:57.256060  1367 raft_consensus.cc:729] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a622d8c7c7304ddc8ab146e1a8e3f5f2, State: Running, Role: LEADER
I20260924 14:32:57.256845  1367 consensus_queue.cc:260] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:57.258442  1367 raft_consensus.cc:3037] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Advancing to term 2
I20260924 14:32:57.262650  1442 raft_consensus.cc:2454] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 2.
I20260924 14:32:57.263908  1482 leader_election.cc:302] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb563febbcf4233b7a125899b66ce05, 91f8d84a8caf421d99048c427fc3b5a6; no voters: 
I20260924 14:32:57.264816  1842 raft_consensus.cc:2781] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 14:32:57.265288  1842 raft_consensus.cc:686] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 2 LEADER]: Becoming Leader. State: Replica: 0fb563febbcf4233b7a125899b66ce05, State: Running, Role: LEADER
I20260924 14:32:57.265681  1367 raft_consensus.cc:2454] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb563febbcf4233b7a125899b66ce05 in term 2.
I20260924 14:32:57.266187  1842 consensus_queue.cc:237] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } }
I20260924 14:32:57.276324  1262 catalog_manager.cc:5495] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 reported cstate change: term changed from 1 to 2, leader changed from a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193) to 0fb563febbcf4233b7a125899b66ce05 (127.22.151.195). New cstate: current_term: 2 leader_uuid: "0fb563febbcf4233b7a125899b66ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb563febbcf4233b7a125899b66ce05" member_type: VOTER last_known_addr { host: "127.22.151.195" port: 33825 } health_report { overall_health: HEALTHY } } }
I20260924 14:32:57.306907  1442 raft_consensus.cc:1261] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Refusing update from remote peer 0fb563febbcf4233b7a125899b66ce05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:57.308203  1842 consensus_queue.cc:1035] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:57.318819  1367 raft_consensus.cc:1261] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Refusing update from remote peer 0fb563febbcf4233b7a125899b66ce05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:57.320459  1858 consensus_queue.cc:1035] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:57.698141  1442 raft_consensus.cc:1261] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Refusing update from remote peer 0fb563febbcf4233b7a125899b66ce05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:57.699640  1806 consensus_queue.cc:1035] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f8d84a8caf421d99048c427fc3b5a6" member_type: VOTER last_known_addr { host: "127.22.151.194" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:57.712352  1367 raft_consensus.cc:1261] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Refusing update from remote peer 0fb563febbcf4233b7a125899b66ce05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260924 14:32:57.714604  1858 consensus_queue.cc:1035] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622d8c7c7304ddc8ab146e1a8e3f5f2" member_type: VOTER last_known_addr { host: "127.22.151.193" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260924 14:32:57.729969 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:57.738073 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:57.738678 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:58.739836 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:58.746618 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:58.747143 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:32:59.748155 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:32:59.754087 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:32:59.754593 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:33:00.755666 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:33:00.762549 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:33:00.763149 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:33:01.764354 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:33:01.770622 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:33:01.771193 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:33:02.772171 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:33:02.778760 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:33:02.779409 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:33:03.780535 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:33:03.786389 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:33:03.786978 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:33:04.787938 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:33:04.793715 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:33:04.794204 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:33:05.795050 23135 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260924 14:33:05.801122 23135 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260924 14:33:05.801594 23135 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260924 14:33:07.024343 23135 tablet_server.cc:178] TabletServer@127.22.151.193:0 shutting down...
W20260924 14:33:07.031965  1405 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
W20260924 14:33:07.037339  1405 consensus_peers.cc:487] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 -> Peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Couldn't send request to peer a622d8c7c7304ddc8ab146e1a8e3f5f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:33:07.039153  1481 consensus_peers.cc:487] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 -> Peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Couldn't send request to peer a622d8c7c7304ddc8ab146e1a8e3f5f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:33:07.076857 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:33:07.077910 23135 tablet_replica.cc:330] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.079412 23135 raft_consensus.cc:2229] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.080547 23135 raft_consensus.cc:2258] T 10709e8d44eb49f1ac2288975ee83633 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.084015 23135 tablet_replica.cc:330] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.084702 23135 raft_consensus.cc:2229] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.085248 23135 raft_consensus.cc:2258] T 260345b02b31461d9c29791d5e54d98f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.087798 23135 tablet_replica.cc:330] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.088589 23135 raft_consensus.cc:2229] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.089176 23135 raft_consensus.cc:2258] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.092342 23135 tablet_replica.cc:330] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.093240 23135 raft_consensus.cc:2229] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.094655 23135 raft_consensus.cc:2258] T eab0e342bab24c059a1ecf79cdc277b4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.098075 23135 tablet_replica.cc:330] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.098834 23135 raft_consensus.cc:2229] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.099395 23135 raft_consensus.cc:2258] T 5701334222ec4b839d1dba26d7b14599 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.103152 23135 tablet_replica.cc:330] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.103950 23135 raft_consensus.cc:2229] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.104930 23135 raft_consensus.cc:2258] T c9d3660f3bdf43fbb2b66c128da37339 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.107597 23135 tablet_replica.cc:330] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.108238 23135 raft_consensus.cc:2229] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.109210 23135 raft_consensus.cc:2258] T bebe60ffa3d94d20bbf6c4c843ba76a7 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.112394 23135 tablet_replica.cc:330] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.113080 23135 raft_consensus.cc:2229] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.113639 23135 raft_consensus.cc:2258] T c8a559653d744c3ab37c5b1b2b1f1b17 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.116358 23135 tablet_replica.cc:330] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.117115 23135 raft_consensus.cc:2229] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.117852 23135 raft_consensus.cc:2258] T 5acf2170d0f8454e9c0344b6e5a446a4 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.121551 23135 tablet_replica.cc:330] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.122305 23135 raft_consensus.cc:2229] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.122823 23135 raft_consensus.cc:2258] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.126909 23135 tablet_replica.cc:330] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.127538 23135 raft_consensus.cc:2229] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.128010 23135 raft_consensus.cc:2258] T 7f656b71445647da9ffe7e0ed4e51ca8 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.130668 23135 tablet_replica.cc:330] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.131402 23135 raft_consensus.cc:2229] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.132104 23135 raft_consensus.cc:2258] T 006c6555ed6b44099b1fce0d5c205a33 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.136138 23135 tablet_replica.cc:330] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.136770 23135 raft_consensus.cc:2229] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.137324 23135 raft_consensus.cc:2258] T 2c8e875b068d47f4a5a6ca1915609625 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.140396 23135 tablet_replica.cc:330] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.141343 23135 raft_consensus.cc:2229] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.142895 23135 raft_consensus.cc:2258] T e4216c32baf34f488c831ce881873cfa P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:33:07.144701  1405 consensus_peers.cc:487] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 -> Peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Couldn't send request to peer a622d8c7c7304ddc8ab146e1a8e3f5f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260924 14:33:07.146095 23135 tablet_replica.cc:330] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.146776 23135 raft_consensus.cc:2229] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.147835 23135 raft_consensus.cc:2258] T c843297774c94a31990bd30dc7bb622d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.150364 23135 tablet_replica.cc:330] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.150918 23135 raft_consensus.cc:2229] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.151427 23135 raft_consensus.cc:2258] T 27b3ccc05ab7495baec6265cd9c3c35d P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.153995 23135 tablet_replica.cc:330] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.154599 23135 raft_consensus.cc:2229] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.155674 23135 raft_consensus.cc:2258] T 3577172d729540129939ac66bbad3c4f P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.158372 23135 tablet_replica.cc:330] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2: stopping tablet replica
I20260924 14:33:07.158895 23135 raft_consensus.cc:2229] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.160022 23135 raft_consensus.cc:2258] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P a622d8c7c7304ddc8ab146e1a8e3f5f2 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:33:07.198577  1405 consensus_peers.cc:487] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 -> Peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Couldn't send request to peer a622d8c7c7304ddc8ab146e1a8e3f5f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260924 14:33:07.200569  1481 connection.cc:535] client connection to 127.22.151.193:43153 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 14:33:07.201279  1481 consensus_peers.cc:487] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 -> Peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Couldn't send request to peer a622d8c7c7304ddc8ab146e1a8e3f5f2. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:33:07.216784 23135 tablet_server.cc:194] TabletServer@127.22.151.193:0 shutdown complete.
I20260924 14:33:07.264943 23135 tablet_server.cc:178] TabletServer@127.22.151.194:0 shutting down...
I20260924 14:33:07.299674 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:33:07.300737 23135 tablet_replica.cc:330] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.301967 23135 raft_consensus.cc:2229] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.303238 23135 raft_consensus.cc:2258] T 10709e8d44eb49f1ac2288975ee83633 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.306411 23135 tablet_replica.cc:330] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.307376 23135 raft_consensus.cc:2229] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.308620 23135 raft_consensus.cc:2258] T 260345b02b31461d9c29791d5e54d98f P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:33:07.310735  1405 consensus_peers.cc:487] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 -> Peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Couldn't send request to peer a622d8c7c7304ddc8ab146e1a8e3f5f2. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:43153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:33:07.311833 23135 tablet_replica.cc:330] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.312570 23135 raft_consensus.cc:2229] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.313479 23135 raft_consensus.cc:2258] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.315876 23135 tablet_replica.cc:330] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.316551 23135 raft_consensus.cc:2229] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.317083 23135 raft_consensus.cc:2258] T eab0e342bab24c059a1ecf79cdc277b4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.319615 23135 tablet_replica.cc:330] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.320232 23135 raft_consensus.cc:2229] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.321346 23135 raft_consensus.cc:2258] T c8a559653d744c3ab37c5b1b2b1f1b17 P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 14:33:07.324870  1405 consensus_peers.cc:487] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 -> Peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Couldn't send request to peer a622d8c7c7304ddc8ab146e1a8e3f5f2. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:43153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:33:07.325107 23135 tablet_replica.cc:330] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.325819 23135 raft_consensus.cc:2229] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.326320 23135 raft_consensus.cc:2258] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.328714 23135 tablet_replica.cc:330] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.329284 23135 raft_consensus.cc:2229] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.329766 23135 raft_consensus.cc:2258] T 5701334222ec4b839d1dba26d7b14599 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.332360 23135 tablet_replica.cc:330] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.332955 23135 raft_consensus.cc:2229] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.333505 23135 raft_consensus.cc:2258] T c9d3660f3bdf43fbb2b66c128da37339 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.335944 23135 tablet_replica.cc:330] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.336541 23135 raft_consensus.cc:2229] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.337018 23135 raft_consensus.cc:2258] T 27b3ccc05ab7495baec6265cd9c3c35d P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.339362 23135 tablet_replica.cc:330] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.340004 23135 raft_consensus.cc:2229] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.340514 23135 raft_consensus.cc:2258] T 3577172d729540129939ac66bbad3c4f P 91f8d84a8caf421d99048c427fc3b5a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.342881 23135 tablet_replica.cc:330] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.343524 23135 raft_consensus.cc:2229] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.344008 23135 raft_consensus.cc:2258] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.346307 23135 tablet_replica.cc:330] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.346925 23135 raft_consensus.cc:2229] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.347415 23135 raft_consensus.cc:2258] T c843297774c94a31990bd30dc7bb622d P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.349807 23135 tablet_replica.cc:330] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.350395 23135 raft_consensus.cc:2229] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.350950 23135 raft_consensus.cc:2258] T e4216c32baf34f488c831ce881873cfa P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.353327 23135 tablet_replica.cc:330] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.353888 23135 raft_consensus.cc:2229] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.354429 23135 raft_consensus.cc:2258] T 5acf2170d0f8454e9c0344b6e5a446a4 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.356681 23135 tablet_replica.cc:330] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.357267 23135 raft_consensus.cc:2229] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.358193 23135 raft_consensus.cc:2258] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.360412 23135 tablet_replica.cc:330] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.360975 23135 raft_consensus.cc:2229] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.361950 23135 raft_consensus.cc:2258] T 7f656b71445647da9ffe7e0ed4e51ca8 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.364460 23135 tablet_replica.cc:330] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.365020 23135 raft_consensus.cc:2229] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.365985 23135 raft_consensus.cc:2258] T 006c6555ed6b44099b1fce0d5c205a33 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.369278 23135 tablet_replica.cc:330] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6: stopping tablet replica
I20260924 14:33:07.369858 23135 raft_consensus.cc:2229] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.370399 23135 raft_consensus.cc:2258] T 2c8e875b068d47f4a5a6ca1915609625 P 91f8d84a8caf421d99048c427fc3b5a6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260924 14:33:07.436404  1295 debug-util.cc:398] Leaking SignalData structure 0x7b08001cd500 after lost signal to thread 23138
W20260924 14:33:07.437837  1295 debug-util.cc:398] Leaking SignalData structure 0x7b08000444c0 after lost signal to thread 1298
W20260924 14:33:07.441529  1295 debug-util.cc:398] Leaking SignalData structure 0x7b08001fb4a0 after lost signal to thread 1544
W20260924 14:33:07.443886  1481 consensus_peers.cc:487] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 -> Peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Couldn't send request to peer a622d8c7c7304ddc8ab146e1a8e3f5f2. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:43153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:33:07.456259 23135 tablet_server.cc:194] TabletServer@127.22.151.194:0 shutdown complete.
W20260924 14:33:07.459246  1481 consensus_peers.cc:487] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 -> Peer a622d8c7c7304ddc8ab146e1a8e3f5f2 (127.22.151.193:43153): Couldn't send request to peer a622d8c7c7304ddc8ab146e1a8e3f5f2. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.193:43153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:33:07.471735  1482 consensus_peers.cc:487] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 -> Peer 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407): Couldn't send request to peer 91f8d84a8caf421d99048c427fc3b5a6. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:38407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260924 14:33:07.489704  1482 consensus_peers.cc:487] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 -> Peer 91f8d84a8caf421d99048c427fc3b5a6 (127.22.151.194:38407): Couldn't send request to peer 91f8d84a8caf421d99048c427fc3b5a6. Status: Network error: Client connection negotiation failed: client connection to 127.22.151.194:38407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 14:33:07.524791 23135 tablet_server.cc:178] TabletServer@127.22.151.195:0 shutting down...
I20260924 14:33:07.586117 23135 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260924 14:33:07.587225 23135 tablet_replica.cc:330] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.588512 23135 raft_consensus.cc:2229] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.589365 23135 raft_consensus.cc:2258] T 9e0b4d3e5e7145a9b1d2ccc5e62347aa P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.592993 23135 tablet_replica.cc:330] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.594043 23135 raft_consensus.cc:2229] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.594870 23135 raft_consensus.cc:2258] T 260345b02b31461d9c29791d5e54d98f P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.598433 23135 tablet_replica.cc:330] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.599395 23135 raft_consensus.cc:2229] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.600268 23135 raft_consensus.cc:2258] T 10709e8d44eb49f1ac2288975ee83633 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.603772 23135 tablet_replica.cc:330] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.604681 23135 raft_consensus.cc:2229] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.605419 23135 raft_consensus.cc:2258] T eab0e342bab24c059a1ecf79cdc277b4 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.608716 23135 tablet_replica.cc:330] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.609862 23135 raft_consensus.cc:2229] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.611522 23135 raft_consensus.cc:2258] T 5701334222ec4b839d1dba26d7b14599 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.615336 23135 tablet_replica.cc:330] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.616382 23135 raft_consensus.cc:2229] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.617206 23135 raft_consensus.cc:2258] T c9d3660f3bdf43fbb2b66c128da37339 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.621173 23135 tablet_replica.cc:330] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.622184 23135 raft_consensus.cc:2229] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.622929 23135 raft_consensus.cc:2258] T c8a559653d744c3ab37c5b1b2b1f1b17 P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.626731 23135 tablet_replica.cc:330] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.627630 23135 raft_consensus.cc:2229] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.628340 23135 raft_consensus.cc:2258] T bebe60ffa3d94d20bbf6c4c843ba76a7 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.631701 23135 tablet_replica.cc:330] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.632539 23135 raft_consensus.cc:2229] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.633284 23135 raft_consensus.cc:2258] T 006c6555ed6b44099b1fce0d5c205a33 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.636803 23135 tablet_replica.cc:330] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.637638 23135 raft_consensus.cc:2229] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.638933 23135 raft_consensus.cc:2258] T 2c8e875b068d47f4a5a6ca1915609625 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.642522 23135 tablet_replica.cc:330] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.643322 23135 raft_consensus.cc:2229] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 2 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.644438 23135 raft_consensus.cc:2258] T 5acf2170d0f8454e9c0344b6e5a446a4 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.646943 23135 tablet_replica.cc:330] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.647629 23135 raft_consensus.cc:2229] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.648154 23135 raft_consensus.cc:2258] T 96caf2a8ba5d4d5bafb9c6dd0ab6d310 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.650758 23135 tablet_replica.cc:330] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.651348 23135 raft_consensus.cc:2229] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.651929 23135 raft_consensus.cc:2258] T 7f656b71445647da9ffe7e0ed4e51ca8 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.654418 23135 tablet_replica.cc:330] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.655009 23135 raft_consensus.cc:2229] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.655545 23135 raft_consensus.cc:2258] T e4216c32baf34f488c831ce881873cfa P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.657889 23135 tablet_replica.cc:330] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.658476 23135 raft_consensus.cc:2229] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.659013 23135 raft_consensus.cc:2258] T c843297774c94a31990bd30dc7bb622d P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.661588 23135 tablet_replica.cc:330] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.662125 23135 raft_consensus.cc:2229] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.663152 23135 raft_consensus.cc:2258] T 27b3ccc05ab7495baec6265cd9c3c35d P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.665743 23135 tablet_replica.cc:330] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.666251 23135 raft_consensus.cc:2229] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.666885 23135 raft_consensus.cc:2258] T 3577172d729540129939ac66bbad3c4f P 0fb563febbcf4233b7a125899b66ce05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.669119 23135 tablet_replica.cc:330] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05: stopping tablet replica
I20260924 14:33:07.669579 23135 raft_consensus.cc:2229] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260924 14:33:07.670073 23135 raft_consensus.cc:2258] T e4d0e3cd411a44ba8e4f9fd59b6e0fb5 P 0fb563febbcf4233b7a125899b66ce05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.722589 23135 tablet_server.cc:194] TabletServer@127.22.151.195:0 shutdown complete.
I20260924 14:33:07.782403 23135 master.cc:537] Master@127.22.151.254:46621 shutting down...
I20260924 14:33:07.810472 23135 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 1 LEADER]: Raft consensus shutting down.
I20260924 14:33:07.811379 23135 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 14:33:07.811857 23135 tablet_replica.cc:330] T 00000000000000000000000000000000 P 58bfe2506c3a4639a24234a3d914debe: stopping tablet replica
W20260924 14:33:08.831609 23135 thread.cc:536] Waited for 1000ms trying to join with diag-logger (tid 1295)
W20260924 14:33:09.832159 23135 thread.cc:536] Waited for 2000ms trying to join with diag-logger (tid 1295)
I20260924 14:33:10.397886 23135 master.cc:552] Master@127.22.151.254:46621 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (25567 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (46803 ms total)

[----------] Global test environment tear-down
[==========] 7 tests from 2 test suites ran. (370520 ms total)
[  PASSED  ] 5 tests.
[  SKIPPED ] 1 test, listed below:
[  SKIPPED ] LeaderRebalancerTest.TestMaintenanceMode
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.RestartTserver

 1 FAILED TEST
I20260924 14:33:10.462843 23135 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_mm_ops.cc:221: suppressed but not reported on 2237 messages since previous log ~36 seconds ago
I20260924 14:33:10.463088 23135 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_mm_ops.cc:305: suppressed but not reported on 2237 messages since previous log ~36 seconds ago
I20260924 14:33:10.463243 23135 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_mm_ops.cc:140: suppressed but not reported on 2235 messages since previous log ~36 seconds ago
I20260924 14:33:10.463415 23135 logging.cc:424] LogThrottler Service Queue Overflow: suppressed but not reported on 1 messages since previous log ~145 seconds ago
I20260924 14:33:10.463673 23135 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:137: suppressed but not reported on 1 messages since previous log ~146 seconds ago
I20260924 14:33:10.463842 23135 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/heartbeater.cc:644: suppressed but not reported on 10 messages since previous log ~126 seconds ago
I20260924 14:33:10.463981 23135 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:594: suppressed but not reported on 44 messages since previous log ~231 seconds ago
I20260924 14:33:10.464111 23135 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:542: suppressed but not reported on 44 messages since previous log ~232 seconds ago
I20260924 14:33:10.464318 23135 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~3 seconds ago
I20260924 14:33:10.464593 23135 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet.cc:2369: suppressed but not reported on 107 messages since previous log ~45 seconds ago